Data Migration
Data migration is the movement of data from a legacy system into a new one, transformed to fit the target model. The work runs across profiling, mapping, cleansing, load, reconciliation and sign-off, and is normally rehearsed through repeated dry runs before the live load during cutover.
What it contains
A migration is organised around objects — customers, accounts, products, open orders, historical transactions — each with its own scope decision, mapping, rules and sign-off. For each object the work covers profiling the source to find out what is actually there, mapping fields to the target, defining transformation and default rules, cleansing whatever will not load, executing the load, and reconciling the result.
Alongside the object list sit the artefacts: a mapping specification, a cleansing log with owners, a dry run schedule, defect tracking for load failures, reconciliation reports per object, and a sign-off record naming who accepted each one. That set is what the migration and UAT tracker keeps together, on the observation that in practice the same people are running both in the same six weeks.
How it is used
Dry runs are the mechanism that turns migration from a hope into a rehearsed procedure. Each run is executed against full-volume production-like data, timed, and reconciled. The first run typically fails in ways nobody predicted; the value is in the second, third and fourth, where duration becomes predictable and the defect count falls. Timing matters as much as correctness, because the load has to fit inside the cutover window — which is why migration timings feed directly into the cutover runbook rather than sitting in a separate plan.
Reconciliation is the evidence that the load worked: record counts, control totals on financial fields, and sampled record comparison. Tolerances are agreed in advance, per object, along with who accepts the result. The guide on migration planning covers how those tolerances and sign-offs are set before the numbers are known, which is the only point at which the conversation is honest.
The final use is at the go/no-go meeting, where migration readiness is a named criterion: dry runs completed, reconciliation within tolerance, defects closed or accepted, and the load duration proven against the window available.
Where it goes wrong
The dominant failure is discovering data quality late. Profiling is skipped or done on a sample, the real state of the legacy data appears during the first full dry run, and cleansing — which is business work, not technical work — starts far too late to complete.
The second is dry runs at reduced volume. A load that takes two hours for ten per cent of records does not take twenty hours for all of them, but nobody knows what it does take until it runs. Volume rehearsal is what makes the cutover timing credible.
The third is unowned cleansing. Defects are identified, listed, and left with "the business" rather than a named person and a date. The list grows and nothing moves.
The fourth is reconciliation designed after the load. Counts match, but no one agreed in advance what a match meant for a partially migrated object, so acceptance turns into an argument at two in the morning during cutover.
The fifth is scope creep in history. "Migrate everything" is common, expensive, and often unnecessary; deciding what stays in the legacy system as a read-only archive is a decision worth making early.
Related terms
Reconciliation is the comparison that proves the load was complete and accurate. Dry run is a full rehearsal of the load, ideally at production volume. ETL describes the extract, transform and load mechanics. Cleansing is correcting source data before migration. Cutover is the window in which the final live load runs.
Questions
How many dry runs are needed?
Enough that the duration is predictable and the defect count has stabilised. Most programmes plan three or four; the number that matters is how many ran clean, not how many were scheduled.
Should historical data be migrated?
That is a scope decision with a cost attached. Keeping history in a read-only archive is a common alternative to migrating volumes that are rarely accessed.
Who signs off migrated data?
The business owner of each data object, against reconciliation evidence and agreed tolerances. Technical completion of the load is a separate confirmation.
When does cleansing start?
As soon as profiling identifies defects. Cleansing is usually business work with a long lead time, and it is the activity most often started too late to finish.