Cloud Modernization Stalls Without Data Engineering Capacity

McKinsey's research on cloud transformation puts it bluntly. Only 10 percent of programs capture their full value. A separate McKinsey study of industrial companies found that 74 percent fail to hit the savings or business outcomes they promised the board. Those are not the numbers of an immature category. Cloud has been mainstream for nearly two decades. The technology works. The commercial model works. So why do most programs underdeliver?

"Cloud modernization" has become a label for application modernization with a different hosting bill. Teams refactor monoliths into microservices, retire a few legacy systems, move workloads onto managed services, and call it a transformation. Then the data team, still hand-coding pipelines into the same warehouse, becomes the bottleneck for everything the new architecture was supposed to enable. The cloud is modern. The data layer underneath it is not.

This piece walks through what cloud modernization actually involves, where the layers fit together, why so many programs stall in the same place, and how to build a strategy that survives contact with reality.

What is cloud modernization?

Cloud modernization is the ongoing process of optimizing applications, data platforms, and infrastructure to operate as cloud-first systems rather than lift-and-shifted versions of on-premise software. It typically involves replatforming, refactoring, or rearchitecting workloads so they take advantage of elastic compute, managed services, and modern data architectures. Unlike cloud migration, which moves workloads to the cloud as-is, modernization changes how they are built and operated.

Most definitions stop there. The more useful framing is that cloud modernization spans five layers, and the discipline of running a successful program is sequencing those layers correctly.

The five layers of cloud modernization

Skipping a layer is the most common cause of stalled programs. Each one is a real piece of work, and each one has its own success criteria.

Infrastructure modernization

The compute, network, and storage foundation. Moving from physical servers and virtual machines to containerized, serverless, and managed-service models. Typical work includes adopting Infrastructure as Code (Terraform, Pulumi), containerization (Kubernetes, ECS), and automated provisioning. This layer tends to be where programs start, because it has the cleanest cost-saving narrative and the lowest application-level risk.

Platform modernization

Platform modernization is the layer most often confused with infrastructure modernization, and it's the one most teams underestimate. It covers the runtime environments, developer platforms, and operational tooling that sit between raw infrastructure and the applications themselves. Think internal developer platforms (Backstage, Humanitec), managed Kubernetes (EKS, GKE, AKS), service meshes, observability stacks, CI/CD pipelines, and the policy and governance layer that makes the whole thing safe at scale. A modernized platform means application teams can ship without filing tickets against an infrastructure team. Without this layer, cloud architectures look modern on a slide and feel like 2012 in practice.

Application modernization

Refactoring or rearchitecting applications themselves. The 6Rs framework is the standard reference here: rehost, replatform, refactor, repurchase, retire, retain. Most programs start with rehost (lift-and-shift), then progress to replatform and refactor as the team gets more comfortable. Refactoring monoliths into microservices, adopting event-driven architectures, and replacing in-house code with SaaS where it makes sense. This is where the marketing slides usually focus.

Data modernization

The layer that decides whether the rest of the program delivers. Cloud-native applications need data that's fresh, governed, and structured for the way they actually consume it. That means cloud data platforms (Snowflake, Databricks, BigQuery), modern transformation tooling, real-time streaming where the use case demands it, and pipelines that don't break every time an upstream source produces schema drift. Most cloud modernization programs treat this layer as a downstream consequence of the application work. It's actually the critical path. More on this below.

Operating model modernization

The org change. Product teams instead of project teams. Platform engineering instead of central IT ticketing. FinOps instead of monthly cost surprises. DataOps and DevSecOps as default practices rather than buzzwords on a poster. None of this is technical, all of it is required. Programs that skip the operating model rebuild end up with a modern stack run by a legacy organization, and the value capture stays in single digits.

Why cloud modernization programs stall

The pattern at enterprise scale is remarkably consistent. The infrastructure work goes well. The platform work goes well enough. Application teams start shipping faster. Then the program hits the data layer and slows to a crawl.

Here's what that actually looks like:

• A bank refactors its credit decisioning system into microservices on Kubernetes. The new services need real-time data on customer transactions, balances, and risk signals. The data team is still running batch ETL jobs nightly into a warehouse that the old monolith was designed around. The shiny new microservices end up reading from a stale snapshot of yesterday's data.
• A retailer stands up Snowflake and migrates its analytics workload off a legacy appliance. Six months in, the analytics team is producing the same dashboards that worked on the old system. The pipelines feeding Snowflake are hand-coded by three engineers who built them in a hurry, and every new data product takes eight weeks because those three engineers are the bottleneck.
• A healthcare provider sets up a multi-cloud architecture for resilience. The data engineering team now maintains the same pipelines twice, in two different orchestrators, with two different sets of credentials and two different alerting systems. Headcount goes up. Velocity goes down. The board asks why.

The technical pattern underneath all three: application architecture got modern, data architecture didn't. The data team is now the constraint on every initiative the rest of the program was supposed to enable. Every cloud cost optimization conversation eventually turns into a conversation about why the data team needs more headcount.

This is the silent tax of the AI economy. Manual data work scales linearly with ambition. Cloud and AI investments scale exponentially with ambition. The gap between the two is where programs go to die.

How to build a cloud modernization strategy

A workable strategy sequences the five layers around the data critical path, not the application critical path. In post-mortems of stalled programs, the failure point is almost always a sequencing decision made in the first quarter, not a technology choice made later. The order matters.

1. Map the data dependencies before the application roadmap. For every workload you're considering modernizing, ask what data it produces, what data it consumes, and how fresh that data needs to be. Workloads with simple, batch-friendly data needs are good early candidates. Workloads with real-time or cross-system data dependencies need their data layer modernized first or in parallel. Treating this as discovery work, before the application architects start drawing target states, prevents the most expensive class of mistake.

2. Pick the target architecture before the migration sequence. Hybrid, single-cloud, multi-cloud. Each has real consequences for how data moves, where it's governed, and what skills you need to hire. There's no universally right answer, but there is a universally wrong process: deciding workload by workload and ending up with an accidental architecture two years later.

3. Prioritize by business value and data readiness, not technical complexity. The instinct is to start with the easiest workloads. The discipline is to start with the workloads that release the most strategic value once modernized. If your hardest workload is also your highest-value one, sequence the data work to support it early.

4. Modernize the data layer in parallel, not after. This is the single most important sequencing decision in the strategy. If the data team is six months behind the application team, every cloud-native application built in the gap inherits the old data architecture. That technical debt compounds.

5. Build FinOps and DataOps disciplines from day one. Cost governance and data quality are not things you bolt on after the migration finishes. They're the operating disciplines that determine whether the cloud bill goes up linearly or quadratically with usage. McKinsey's research is clear that organizations integrating site reliability engineering, FinOps, and product-team operating models early capture meaningfully more value than those treating them as later-phase concerns.

6. Sequence migrations to avoid the lift-and-shift trap. Lift-and-shift is fine as a tactical first step for some workloads. It becomes a trap when the program stops there and never circles back to refactor. Build the refactor commitment into the migration plan from the start, with named owners and a deadline. Without that, lift-and-shift becomes lift-and-stay.

A strategy built this way is harder to sell internally because it doesn't promise quick wins in the first quarter. It does, however, finish.

Enterprise cloud transformation: when modernization stops being a project

At enterprise scale, cloud modernization stops being a program and becomes an operating-model change. The CDAO and CIO are no longer overseeing a migration. They're rebuilding how the organization builds, ships, and operates technology.

Across Fortune 500 transformation programs, three shifts consistently define this transition.

The first is governance. A modernization program has a steering committee. An operating model has product owners, platform teams, and shared services with their own P&Ls. The mistake most enterprises make is keeping the steering committee structure long after the work has outgrown it.

The second is the platform team model. In a modernized enterprise, the platform team is a product team that serves application engineers as customers. Its product is the developer experience, including the data developer experience. If your data engineers still file tickets to get a new pipeline approved, you have a modernized stack and a legacy operating model.

The third is FinOps as a finance discipline, not a tooling category. Cloud spend at enterprise scale is one of the top three lines on the IT budget, and it grows on its own unless someone is accountable for it. FinOps integrated into the finance function, with engineering teams accountable for unit economics, is what separates enterprises that capture cloud value from those that just pay for it.

CDAOs running this kind of transformation are no longer the gatekeepers of scarce data engineering capacity. They're the architects of how the business uses data to compete. That shift in role is the real outcome of an enterprise cloud transformation that worked.

The role of AI data automation in modern cloud strategy

We’ve covered how cloud modernization can stall at the data layer. The next question is what to do about it.

Hand-coded pipelines, even when written in modern languages and orchestrated with modern tools, scale with engineering headcount. They are brittle, expensive to maintain, and slow to change. In a cloud environment where applications, infrastructure, and platform layers are all moving toward higher levels of automation and self-service, leaving the data layer dependent on manual engineering effort is the structural reason programs stall.

AI data automation, sometimes called autonomous data engineering, is the operational layer that closes that gap. It uses agents to build, monitor, and adapt data pipelines with limited human intervention, freeing the data engineering team from the work of keeping existing pipelines alive and giving them capacity to build new data products. It's the difference between a data team that's the bottleneck of a modernization program and a data team that's the engine of it.

The teams that close this gap don't add headcount. They remove the constraint. That's what Maia is built for. St. James's Place, the UK wealth management firm, ran Maia against a complex ETL migration and cut the engineering effort by roughly two-thirds, redirecting capacity to its SAP and AI roadmap. At Edmund Optics, a two-person data team tripled pipeline development output and absorbed a major systems migration that would otherwise have gone to consultants. And Nature's Touch found a $500,000 annual inventory variance hidden inside spreadsheet logic that its ERP and MRP systems had been processing without question for years. The point is not the tool. The point is that data automation is now a category, and any cloud modernization strategy written in 2026 that doesn't account for it is a strategy that's planning to stall in 18 months. 

The takeaway

The category called "cloud modernization" was built around an application-first definition that no longer matches what enterprises actually need. Applications, infrastructure, and platforms have caught up to a cloud-first operating model. Data engineering, mostly, has not. The programs that finish in the next three years will be the ones that put the data layer on the same trajectory as the rest of the stack, with AI data automation doing the work that human pipeline-builders no longer have time for.

If you're scoping a program right now, the question to bring into the planning room is not "which workloads do we move first." It's "what does our data layer need to look like for the workloads we're modernizing to actually deliver." Answer that one well, and the rest of the strategy mostly writes itself.

Stop sequencing your cloud program around the data team's capacity.