Case studies / Cloud
Cloud · Case study
A petabyte-scale SQL Server platform, rebuilt for AWS
A multi-year re-architecture — not a lift-and-shift — for a financial data and analytics business running 50+ on-premises SQL Server instances and growing by roughly 10TB a month.
The challenge
In 2017, the client's infrastructure, data, and applications were hosted entirely in an on-premises data center. Microsoft SQL Server was the primary data platform — more than 50 instances holding close to a petabyte of data, growing by roughly 10TB every month, all flowing through multiple complex pipelines.
From the outset, this wasn't going to be a simple lift-and-shift from on-premises to the cloud. The goal was to take full advantage of what AWS offered, which meant fundamentally re-architecting the platform around AWS-native services — many of which the client's own team was only beginning to build expertise in.
How we approached it
Prove the approach
DataGrokr joined early, running quick-cycle proofs of concept across candidate AWS services before settling on Amazon Redshift, Amazon EMR, and an S3-based data lake as the target architecture — using transient EMR clusters sized to each workload, with Spot and fleet instances, carefully chosen instance types, and scaling rules tied to data volume.
Redesign and validate at scale
More than 300 complex ETL pipelines were redesigned, rebuilt, and thoroughly tested — validating that output from the new pipelines matched the legacy environment before any cutover. Pipelines ran several times faster on the new platform; some processes that had taken weeks in the legacy environment finished in about 12 hours.
Operate, secure, and advise
DataGrokr helped establish the AWS cloud architecture, networking strategy, system operations, security framework, and DevOps pipelines, growing to a team of roughly 20 engineers who worked directly with internal users — moving beyond execution into an ongoing advisory role on architecture and planning.
Under the hood
The rebuilt platform, and the legacy environment it replaced:
The result
The modernization cut annual platform spend by 40% — from $10M to $6M — and reduced day-to-day (BAU) time-to-delivery by several weeks.
What stood out most was that DataGrokr was never just an execution partner. They consistently advised us on design, planning, and problem-solving, and brought the expertise, flexibility, and collaborative mindset a modernization of this scale requires.
Modernizing a financial data platform without losing what already works?
Facing something similar?
Scope a diagnostic →