Quest1

From Months to Days: A Self-Service Migration Pipeline Between Two SaaS Products

How a practice-management software provider serving the elective medical industry rebuilt customer migration between two of its products — replacing months of engineering-led work with a self-serve, AWS-native pipeline that the service team runs in days, with two-hour turnarounds in sight.

$ cat case-study/details.yaml
Industry
Healthcare
Service
Data Engineering
Partner
AWS
$ read /case-study/01-the-problem.md

The Problem.

The client operates two practice-management products serving overlapping customer segments. The legacy product was built around the workflow of cosmetic and plastic surgery clinics. The modern product is purpose-built for med spas. A meaningful share of the legacy product's customer base is structurally a better fit for the modern product — and moving them onto the right product was both a customer success story and a retention strategy.

The problem was getting them there. The two products were architected for different operating models, and the database structures reflected that. The legacy product's data sits in MySQL, sharded per customer, producing thousands of shards across the customer base. The modern product uses a non-sharded model with tenant segregation expressed through a TenantId column. The shapes of the data, the schemas, and the assumptions baked into them — none of them lined up.

The modern product shipped with an importer tool designed to ingest standardized CSV files into its database. But the importer is a Windows-native application that has to be installed on a service team member's machine, and it expects data already shaped to the importer's exact format. The bridge from a per-customer legacy shard to the importer's CSV format didn't exist. Building it for one customer was a manual, multi-week engineering exercise. Doing it across the customer base, repeatably, with the service team rather than engineering doing the work, was a different problem entirely.

There was also the matter of the customer's images and files — radiology images, before/after photos, documents — which had to move with the structured data, with their metadata intact, into the new tenant's S3 storage in the modern product.

The objective for the engagement: build a tool that compresses migration from months to days (with a two-hour ceiling in sight), runs as self-service for the service team, scales across the customer base, and is extensible to ingest data from other EMR systems in the future.

$ read /case-study/02-what-we-built.md

What we built.

Quest1 designed and built an end-to-end, AWS-native migration platform that takes a customer all the way through from the legacy product to a fully populated tenant on the modern product, with a self-service UI for the service team and structured orchestration underneath.

A self-service UI for the service team, not a developer tool.
The front end is an Angular application, authenticated via AWS Cognito, hosted on S3 and fronted by CloudFront. A service team member searches for a customer, kicks off the migration, and watches the run through to completion — no engineering ticket, no Windows-native importer install on a workstation, no manual CSV preparation. The .NET back end runs as Lambda functions behind API Gateway. Authorization is integrated through Cognito.
A staged data pipeline orchestrated by Step Functions.
The migration workflow runs as a Step Functions state machine that sequences four core stages: read from the source shard, stage to S3, transform to the importer's standard format, and load into the modern product. The exact set of stages and tables to process for a given customer is defined as configuration in DynamoDB rather than hardcoded — so the same orchestration handles different customer shapes without code changes.
AWS Glue with PySpark for the transforms.
Once data is staged in S3, AWS Glue jobs running PySpark transforms cleanse, standardize, and reshape the source records into the importer-compatible format. The team adopted SparkSQL-style transforms to keep the transformation logic accessible — readable by the data engineering team rather than locked behind procedural Python — which paid off as the transform library grew across customer migrations.
A Lambda-based loader.
The transformed data is then written into the modern product's database by a Lambda step that reads from S3 and persists into the destination while preserving tenant isolation through TenantId injection at write time.
File and image migration alongside the structured data.
Customer images, documents, and their metadata move from the legacy-side S3 storage into the modern tenant's S3 bucket as part of the same orchestrated run, so the structured and unstructured halves of a customer's data land together.
Status tracking through DynamoDB and EventBridge.
The same DynamoDB table that holds the per-customer pipeline configuration also tracks each run's progress. EventBridge events emitted by pipeline stages trigger a status-update Lambda that writes back into DynamoDB, giving the UI a live view of where each migration is — without polling, and without coupling the orchestration logic to the status surface.

The networking and platform layer — VPC, subnets, NAT gateway, S3 endpoints, SQS for buffer where stages decouple — was structured to keep the data path private and the failure modes recoverable, so a stalled migration could be resumed rather than restarted from zero.

$ read /case-study/03-the-impact.md

The Impact.

The platform compressed migration from a months-long engineering exercise into a service team task that takes days, with a two-hour ceiling in sight as the next round of improvements lands. The cost structure of customer migration changed from "an engineering project per customer" to "a service team workflow," and that change held up at volume.

In the first month after deployment, more than ten customers were migrated through the platform — comfortably faster than the prior model would have permitted, and a meaningful early signal on the platform's repeatability.

  • A self-service migration tool, owned and operated by the service team, with the engineering team out of the per-customer critical path.
  • A configurable, orchestrated pipeline in Step Functions whose stage definitions and per-customer specifics live in DynamoDB rather than code — extensible to additional data shapes without re-architecting.
  • A SparkSQL-based transform library that's accessible to the data engineering team for extension rather than ownership-locked to one author.
  • A pattern that extends beyond legacy-to-modern migration to ingest data from external EMR systems — the same orchestration carries other source-system shapes through with new transforms.
  • An AWS-native architecture with TCO modeled and consolidated across services rather than fragmented across third-party tools.
$ cat metrics.summary
months → days, headed to two hours
>90% faster migrations
self-serve for the service team
80% less tech team involvement
pattern proven at volume
10+ customers in month one