Reimagining data engineering in the era of AI

As businesses integrate AI into more of their processes, top executives are recognizing the crucial role of data engineers in actualizing these projects. AI can only perform effectively when there is an abundance of trustworthy and well-handled, high-quality data. This report reveals that data engineers are essential to their organizations, acting as facilitators of AI. By doing so, they contribute significantly to the overall prosperity of the business.

According to findings from a survey involving 400 senior data and technology executives conducted by MIT Technology Review Insights, data engineers have gained influence in domains that extend far beyond their conventional roles as pipeline overseers. The evolution of technology is also altering how data engineers operate, with their focus shifting from fundamental data management responsibilities towards AI-focused tasks.

As their authority expands, so too do the obstacles that data engineers encounter. A significant challenge is tackling increased complexity, as more sophisticated AI models heighten the necessity of managing unstructured data and real-time data streams. Another issue is handling growing workloads; data engineers are being tasked with more responsibilities than ever before, and this trend is unlikely to reverse.

Key insights from the report include:

• Data engineers are vital to the organization. This perspective is shared by 72% of the surveyed technology leaders—and 86% among those in the largest surveyed organizations, where AI maturity is highest. This view is especially prevalent among executives in the financial services and manufacturing sectors.
• AI is transforming the responsibilities of data engineers. The proportion of time data engineers dedicate daily to AI-related projects has nearly doubled over the last two years, rising from an average of 19% in 2023 to 37% in 2025, based on our survey. Respondents anticipate this number will continue to rise to an average of 61% in the next two years. This is also leading to heavier workloads for data engineers; most respondents (77%) acknowledge these are becoming increasingly burdensome.

Access the report.

This content was developed by Insights, the custom content division of MIT Technology Review. It was not authored by the editorial team of MIT Technology Review.

This content was researched, crafted, and composed by human writers, editors, analysts, and illustrators. This encompasses the formulation of surveys and the gathering of data for those surveys. AI tools that may have been utilized were restricted to secondary production methods that underwent thorough human scrutiny.