Data engineering

Data engineering is the specialized application of software-engineering practices to collect, transform, and store data for analysis and predictive modeling.

Data engineering practices may overlap significantly with business intelligence, data science, and data analysis practices. However, the primary focus of data engineering is to develop the infrastructure, automation, and CI/CD required to store and access the data as required by the business.

Video: How Data Engineering Works

Deeper Knowledge on Data Engineering

Big Data

Challenges related to data variety, velocity, and volume

Data Teams

The make up and measures of effective data teams

Data Products

Ways of making data available

Data Pipelines

Ways of making data available

Broader Topics Related to Data Engineering

Business Intelligence

Methods to bridge the gap between data and business

Information Technology (IT)

The principles, practices, and technologies used to build information systems

Software Engineering

Engineering approaches to software development