Course Remote

How to Use Differential Testing During a Legacy Migration

Hugo Teijiz · Remote

At a glance

Type: Course
Organisation: Hugo Teijiz
Location: Remote
Work mode: Remote
Deadline: Rolling / not stated
Posted: 14 Sep 2026

About this course

The most dangerous moment in a legacy migration isn't necessarily when you start writing the new implementation. It's when the new implementation looks finished. The code compiles, the tests pass, the architecture is cleaner, and the new service responds faster. Then everybody starts asking the same question: <blockquote> Can we switch traffic now? </blockquote> That's where confidence becomes difficult. A new implementation can pass its own test suite and still behave differently from the system it is replacing. Maybe rounding changed, or null values are handled differently, or an error became a successful response. Maybe records are sorted differently, or a side effect happens in a different order, or a business rule you never documented was lost during the migration. This is why, during a legacy migration, I like having another source of evidence: run the old and new implementations with the same inputs and compare what they do. That's the basic idea behind differential testing. Instead of asking only if the new system passes its tests, you also ask: given the same input, where does the new system behave differently from the old one? Those differences become evidence. Some are bugs, some are intentional improvements, some are harmless representation differences, and some reveal behavior nobody knew existed. In this tutorial, I'll show you how to use differential testing during a legacy migration to: <ul> <li>Compare old and new implementations </li> <li>Define what should be considered equivalent </li> <li>Normalize outputs before comparing them </li> <li>Handle timestamps and other nondeterministic values </li> <li>Compare errors and side effects </li> <li>Run differential tests automatically </li> <li>Introduce tolerances where exact equality doesn't make sense </li> <li>Analyze mismatches </li> <li>Use shadow traffic in production safely </li> <li>Use AI to classify divergences without letting it decide correctness </li> <li>Determine when the new implementation is ready for cutover </li> </ul> The examples use TypeScript and Vitest, but the approach applies to most languages and migration strategies. The goal isn't to prove that two implementations are internally identical. It's to obtain evidence that they are behaviorally equivalent where equivalence matters. <h2 id="heading-prerequisites">Prerequisites</h2> To follow along here, you should be comfortable with: <ul> <li>TypeScript or a similar language </li> <li>unit and integration testing </li> <li>asynchronous code </li> <li>API and service boundaries </li> <li>legacy modernization </li> <li>basic observability concepts </li> </ul> You should also already have some understanding of the capability being migrated. Ideally, you know its inputs, outputs, important business rules, external contracts, side effects, and known areas of uncertainty. Differential testing works best after you've already created a boundary around the capability you want to migrate. <h2 id="heading-table-of-contents">Table of Contents</h2> <ul> <li><a href="#heading-prerequisites">Prerequisites</a> </li> <li><a href="#heading-what-differential-testing-actually-tells-you">What Differential Testing Actually Tells You</a> </li> <li><a href="#heading-start-with-one-observable-boundary">Start with One Observable Boundary</a> </li> <li><a href="#heading-run-the-legacy-and-new-implementations-with-the-same-input">Run the Legacy and New Implementations with the Same Input</a> </li> <li><a href="#heading-dont-compare-raw-output-blindly">Don't Compare Raw Output Blindly</a> </li> <li><a href="#heading-normalize-values-before-comparing-them">Normalize Values Before Comparing Them</a> </li> <li><a href="#heading-handle-timestamps-and-other-nondeterministic-values">Handle Timestamps and Other Nondeterministic Values</a> </li> <li><a href="#heading-compare-business-meaning-not-just-json">Compare Business Meaning, Not Just JSON</a> </li> <li><a href="#heading-compare-errors-as-part-of-the-contract">Compare Errors as Part of the Contract</a> </li> <li><a href="#heading-compare-side-effects-too">Compare Side Effects, Too</a> </li> <li><a href="#heading-use-tolerances-when-exact-equality-is-wrong">Use Tolerances When Exact Equality Is Wrong</a> </li> <li><a href="#heading-build-a-reusable-differential-test-harness">Build a Reusable Differential Test Harness</a> </li> <li><a href="#heading-generate-test-cases-from-real-behavior">Generate Test Cases from Real Behavior</a> </li> <li><a href="#heading-classify-every-difference">Classify Every Difference</a> </li> <li><a href="#heading-how-to-use-ai-to-investigate-differential-failures">How to Use AI to Investigate Differential Failures</a> </li> <li><a href="#heading-how-to-use-shadow-traffic-safely">How to Use Shadow Traffic Safely</a> </li> <li><a href="#heading-measure-divergence-instead-of-waiting-for-perfection">Measure Divergence Instead of Waiting for Perfection</a> </li> <li><a href="#heading-how-to-know-when-youre-ready-for-cutover">How to Know When You're Ready for Cutover</a> </li> <li><a href="#heading-a-practical-differential-testing-workflow">A Practical Differential Testing Workflow</a> </li> <li><a href="#heading-what-differential-testing-cant-prove">What Differential Testing Can't Prove</a> </li> <li><a href="#heading-differential-testing-turns-migration-risk-into-evidence">Differential Testing Turns Migration Risk into Evidence</a> </li> <li><a href="#heading-conclusion">Conclusion</a> </li> </ul> <h2 id="heading-what-differential-testing-actually-tells-you">What Differential Testing Actually Tells You</h2> Imagine that your legacy application calculates the final price of an order. The legacy implementation looks like this: <pre><code class="language-typescript">type Order = { subtotal: number; customerType: "STANDARD" | "PREMIUM"; country: string; }; function legacyCalculateTotal(order: Order): number { let total = order.subtotal; if (order.customerType === "PREMIUM") { total *= 0.9; } if (order.country === "AR") { total -= 500; } return Math.max(total, 0); } </code></pre> During the migration, you create a new implementation: <pre><code class="language-typescript">function newCalculateTotal(order: Order): number { const premiumDiscount = order.customerType === "PREMIUM" ? order.subtotal * 0.1 : 0; const countryAdjustment = order.country === "AR" ? 500 : 0; return Math.max( order.subtotal - premiumDiscount - countryAdjustment, 0 ); } </code></pre> The implementations look different. And that's fine. What matters is whether they produce equivalent behavior. A simple differential test can run both: <pre><code class="language-typescript">import { describe, expect, it } from "vitest"; describe("order total migration", () => { it("matches the legacy implementation", () => { const order: Order = { subtotal: 10000, customerType: "PREMIUM", country: "AR", }; const legacy = legacyCalculateTotal(order); const migrated = newCalculateTotal(order); expect(migrated).toBe(legacy); }); }); </code></pre> For this input: <pre><code class="language-text">legacy → 8500 new → 8500 </code></pre> Good. But one matching example proves very little. The value comes from systematically asking: <pre><code class="language-text">same input ↓ legacy implementation ──→ result A same input ↓ new implementation ─────→ result B compare A and B </code></pre> Every mismatch gives you something to investigate. <h2 id="heading-start-with-one-observable-boundary">Start with One Observable Boundary</h2> Don't begin by comparing entire applications. To start, choose one capability. For example: <pre><code class="language-text">Calculate Order Total Generate Invoice Approve Customer Renew Subscription Calculate Commission Create Shipment </code></pre> Suppose the migration boundary is: <pre><code class="language-typescript">interface OrderProcessor { process(order: Order): Promise<ProcessedOrder>; } </code></pre> Now you have two implementations: <pre><code class="language-text">LegacyOrderProcessor NewOrderProcessor </code></pre> That is a useful differential boundary, because both receive the same conceptual input, and both produce the same conceptual output. You can compare them without requiring their internal architecture to match. That matters because migrations often change structure intentionally. The legacy implementation might be: <pre><code class="language-text">controller → service → SQL → provider SDK </code></pre> while the new implementation might be: <pre><code class="language-text">use case → repository → gateway → events </code></pre> Differential testing shouldn't care. It should care about observable behavior. <h2 id="heading-run-the-legacy-and-new-implementations-with-the-same-input">Run the Legacy and New Implementations with the Same Input</h2> Suppose both implementations expose: <pre><code class="language-typescript">interface OrderProcessor { process(order: Order): Promise<ProcessedOrder>; } </code></pre> You can create: <pre><code class="language-typescript">const legacyProcessor = new LegacyOrderProcessor(); const newProcessor = new NewOrderProcessor(); </code></pre> Then: <pre><code class="language-typescript">it("produces the same processed order", async () => { const input: Order = { id: "order-1", subtotal: 10000, customerType: "PREMIUM", country: "US", }; const legacy = await legacyProcessor.process( structuredClone(input) ); const migrated = await newProcessor.process( structuredClone(input) ); expect(migrated).toEqual(legacy); }); </code></pre> Notice the use of: <pre><code class="language-typescript">structuredClone(input) </code></pre> That matters if either implementation mutates its input. Without separate copies, the first execution could influence the second. You want: <pre><code class="language-text">same initial state </code></pre> not: <pre><code class="language-text">new implementation receives state modified by legacy implementation </code></pre> That kind of contamination can create misleading results. <h2 id="heading-dont-compare-raw-output-blindly">Don't Compare Raw Output Blindly</h2> The first version of a differential test is often: <pre><code class="language-typescript">expect(newResult).toEqual(legacyResult); </code></pre> Sometimes that's exactly right. But other times it's wrong. Imagine the legacy system returns: <pre><code class="language-json">{ "id": "order-1", "total": 9000, "status": "PROCESSED", "generatedAt": "2026-09-09T10:00:01.231Z", "requestId": "legacy-f93a" } </code></pre> The new system returns: <pre><code class="language-json">{ "requestId": "new-b517", "status": "PROCESSED", "generatedAt": "2026-09-09T10:00:01.416Z", "total": 9000, "id": "order-1" } </code></pre> A raw object comparison may fail because: <pre><code class="language-text">requestId differs timestamp differs </code></pre> But the business behavior might be equivalent. You need to decide which fields are part of the meaningful contract. Maybe: <pre><code class="language-text">id total status </code></pre> matter. While: <pre><code class="language-text">generatedAt requestId </code></pre> don't need exact equivalence. That leads to normalization. <h2 id="heading-normalize-values-before-comparing-them">Normalize Values Before Comparing Them</h2> Normalization means transforming outputs into a common representation before comparing them. The goal isn't to change the business meaning of the data. It's to remove differences that are expected and irrelevant to the comparison, such as generated request IDs or timestamps, so the test can focus on the fields that actually define the behavior you care about. In practice, that often means creating a canonical representation: a smaller, stable shape that contains only the meaningful fields you want to compare. For example: <pre><code class="language-typescript">type ProcessedOrder = { id: string; total: number; status: string; generatedAt: string; requestId: string; }; function normalizeOrder( order: ProcessedOrder ) { return { id: order.id, total: order.total, status: order.status, }; } </code></pre> Here, <code>ProcessedOrder</code> contains both business-relevant fields and values that may legitimately differ between executions. The <code>normalizeOrder()</code> function keeps <code>id</code>, <code>total</code>, and <code>status</code>, while leaving out <code>generatedAt</code> and <code>requestId</code>. That means two results can still be considered equivalent even if they were generated at slightly different times or used different request identifiers. Now compare: <pre><code class="language-typescript">expect( normalizeOrder(migrated) ).toEqual( normalizeOrder(legacy) ); </code></pre> This makes your equivalence rule explicit. You're saying: <blockquote> These fields define relevant behavior for this comparison. </blockquote> Normalization can also handle: <ul> <li>ordering </li> <li>casing </li> <li>optional fields </li> <li>timestamps </li> <li>generated identifiers </li> <li>numeric formatting </li> <li>provider-specific metadata </li> </ul> But normalization must be deliberate. If you remove too much, you can hide real migration bugs. <h2 id="heading-handle-timestamps-and-other-nondeterministic-values">Handle Timestamps and Other Nondeterministic Values</h2> Legacy systems contain many nondeterministic values. For example: <pre><code class="language-text">timestamps UUIDs random tokens request IDs trace IDs database-generated IDs unordered collections provider-generated references </code></pre> If you compare those values exactly, your differential suite may fail constantly. One option is dependency control. Dependency control means moving a nondeterministic source, such as the current time or an ID generator, behind an interface that you can replace during tests. Instead of letting each implementation read the real clock independently, you inject the same controlled clock into both. That gives them the same value and removes time itself as a source of meaningless divergence. Suppose the code uses: <pre><code class="language-typescript">new Date() </code></pre> You can replace that dependency with a clock: <pre><code class="language-typescript">interface Clock { now(): Date; } </code></pre> Then both implementations receive: <pre><code class="language-typescript">const clock = { now: () => new Date( "2026-09-09T10:00:00.000Z" ), }; </code></pre> Now time becomes deterministic. The same technique can work for ID generation: <pre><code class="language-typescript">interface IdGenerator { next(): string; } </code></pre> Then tests can provide: <pre><code class="language-typescript">const ids = { next: () => "fixed-id", }; </code></pre> If controlling nondeterminism is impractical, normalize it out only when it's not part of the behavior you need to protect. <h2 id="heading-compare-business-meaning-not-just-json">Compare Business Meaning, Not Just JSON</h2> Two systems can return different representations while expressing the same business state. Imagine you have this in your legacy system: <pre><code class="language-json">{ "status": 2 } </code></pre> And this in your new one: <pre><code class="language-json">{ "status": "APPROVED" } </code></pre> Raw comparison says: <pre><code class="language-text">different </code></pre> Business comparison may say: <pre><code class="language-text">equivalent </code></pre> You can create a semantic normalizer: <pre><code class="language-typescript">function normalizeStatus( status: number | string ) { if (status === 2) { return "APPROVED"; } return status; } </code></pre> Here, the normalizer translates the legacy numeric value <code>2</code> into the business meaning used by the new implementation: <code>"APPROVED"</code>. It doesn't claim that every number and string are interchangeable. It encodes one explicit equivalence rule that you've already decided is valid for this migration. Then: <pre><code class="language-typescript">expect( normalizeStatus(newResult.status) ).toBe( normalizeStatus(legacyResult.status) ); </code></pre> This is especially useful when migration intentionally changes: <pre><code class="language-text">database schema API representation enumerations provider-specific formats internal identifiers </code></pre> The important question becomes: <blockquote> Does the observable business meaning remain equivalent? </blockquote> Not: <blockquote> Are the bytes identical? </blockquote> <h2 id="heading-compare-errors-as-part-of-the-contract">Compare Errors as Part of the Contract</h2> Success responses aren't the whole behavior. Errors matter too. Suppose the legacy implementation rejects a missing customer: <pre><code class="language-typescript">throw new Error("Customer not found"); </code></pre> The new implementation accidentally returns: <pre><code class="language-typescript">return null; </code></pre> These two implementations behave very differently for the same invalid input. The legacy version fails explicitly, while the new version silently returns a value that a caller may interpret as a successful result. If your differential tests only exercise cases where a valid customer exists, both implementations may appear equivalent and this contract change will remain invisible. That's why failure behavior has to be compared too. Create cases that capture errors: <pre><code class="language-typescript">async function captureResult<T>( operation: () => Promise<T> ) { try { return { type: "success" as const, value: await operation(), }; } catch (error) { return { type: "error" as const, error: error instanceof Error ? error.message : String(error), }; } } </code></pre> The helper wraps an asynchronous operation and converts both possible outcomes into data. If the operation succeeds, it returns an object with <code>type: "success"</code> and the returned value. If the operation throws, the <code>catch</code> block converts that exception into an object with <code>type: "error"</code> and a readable error message. This gives both implementations the same comparison shape, so the test can compare success versus failure explicitly instead of letting an exception stop the test before the two behaviors can be evaluated. Now: <pre><code class="language-typescript">const legacy = await captureResult(() => legacyProcessor.process(input) ); const migrated = await captureResult(() => newProcessor.process(input) ); expect(migrated.type).toBe(legacy.type); </code></pre> If errors are contractually important, compare: <pre><code class="language-text">error category HTTP status error code retryability validation details </code></pre> Don't necessarily compare exact wording unless clients depend on it. <h2 id="heading-compare-side-effects-too">Compare Side Effects, Too</h2> One of the easiest migration mistakes is preserving the return value while losing a side effect. Suppose both implementations return: <pre><code class="language-json">{ "status": "PROCESSED" } </code></pre> But the legacy version also: <pre><code class="language-text">persists the order publishes an event creates a payment writes an audit entry </code></pre> and the new version forgets the audit entry. Response-level differential testing won't catch that. So you'll want to capture side effects. For example: <pre><code class="language-typescript">type Effect = | { type: "payment"; orderId: string; amount: number; } | { type: "event"; name: string; orderId: string; }; </code></pre> A test adapter can record them: <pre><code class="language-typescript">class RecordingPaymentGateway { effects: Effect[] = []; async charge( orderId: string, amount: number ) { this.effects.push({ type: "payment", orderId, amount, }); } } </code></pre> Instead of sending a real payment request, this adapter records what the application attempted to do in the <code>effects</code> array. You can apply the same idea to event publication: <pre><code class="language-typescript">class RecordingEvents { effects: Effect[] = []; async publish( name: string, orderId: string ) { this.effects.push({ type: "event", name, orderId, }); } } </code></pre> The application still calls its payment and event dependencies as usual. The test doubles simply capture those calls as structured data instead of performing the real external actions. After running the legacy and migrated implementations with their own recording adapters, you can compare the two recorded effect lists and verify that both systems attempted the same observable side effects. Now the differential test can compare: <pre><code class="language-typescript">expect(newEffects).toEqual(legacyEffects); </code></pre> Again, exact ordering should only be required if ordering matters. <h2 id="heading-use-tolerances-when-exact-equality-is-wrong">Use Tolerances When Exact Equality Is Wrong</h2> Some domains shouldn't use exact equality. Imagine a migrated calculation produces: <pre><code class="language-text">legacy → 34.333333333 new → 34.333333334 </code></pre> Is that a migration bug? Maybe not. Floating-point calculations may justify a tolerance. For example: <pre><code class="language-typescript">expect(newResult).toBeCloseTo( legacyResult, 6 ); </code></pre> Or define an explicit comparator: <pre><code class="language-typescript">function withinTolerance( a: number, b: number, tolerance: number ) { return Math.abs(a - b) <= tolerance; } </code></pre> Then: <pre><code class="language-typescript">expect( withinTolerance( migrated.total, legacy.total, 0.01 ) ).toBe(true); </code></pre> But tolerances should come from domain requirements. Don't use them just to make failing tests disappear. For financial systems, one cent can matter. For scientific calculations, a much smaller numerical difference may matter. Equivalence is a business and engineering decision. <h2 id="heading-build-a-reusable-differential-test-harness">Build a Reusable Differential Test Harness</h2> Once you compare more than a few cases, you can create a reusable harness. For example: <pre><code class="language-typescript">type DifferentialResult<T> = { input: T; equivalent: boolean; legacy: unknown; migrated: unknown; }; async function compareImplementations< TInput, TOutput >( input: TInput, legacy: ( input: TInput ) => Promise<TOutput>, migrated: ( input: TInput ) => Promise<TOutput>, normalize: ( output: TOutput ) => unknown ): Promise< DifferentialResult<TInput> > { const legacyResult = await legacy( structuredClone(input) ); const migratedResult = await migrated( structuredClone(input) ); const normalizedLegacy = normalize(legacyResult); const normalizedMigrated = normalize(migratedResult); return { input, equivalent: JSON.stringify( normalizedLegacy ) === JSON.stringify( normalizedMigrated ), legacy: normalizedLegacy, migrated: normalizedMigrated, }; } </code></pre> The harness does four things. First, it runs the legacy and migrated implementations with separate clones of the same input, so one execution can't mutate the data seen by the other. Second, it passes both outputs through the same <code>normalize()</code> function. That applies the equivalence rules in one place instead of repeating them in every test. Third, it compares the normalized results and records whether they're equivalent. Finally, it returns the input and both normalized outputs together. That makes a failed comparison easier to inspect because the test report can show exactly which case diverged and what each implementation produced. Then: <pre><code class="language-typescript">const result = await compareImplementations( input, legacyProcessor.process.bind( legacyProcessor ), newProcessor.process.bind( newProcessor ), normalizeOrder ); expect(result.equivalent).toBe(true); </code></pre> For real systems, I would usually avoid relying on <code>JSON.stringify()</code> as the final equality mechanism. The example keeps the harness readable. In production-quality tooling, use a proper structural or domain-specific comparator. The important part is that comparison logic becomes centralized. <h2 id="heading-generate-test-cases-from-real-behavior">Generate Test Cases from Real Behavior</h2> Hand-written examples are useful. But migrations often fail on cases nobody thought to write manually. Useful sources of inputs include: <pre><code class="language-text">existing test fixtures historical incidents production-safe request samples database records boundary values previous bug reports known customer scenarios </code></pre> Suppose production shows these order shapes: <pre><code class="language-typescript">const cases: Order[] = [ { subtotal: 0, customerType: "STANDARD", country: "US", }, { subtotal: 500, customerType: "PREMIUM", country: "AR", }, { subtotal: 10000, customerType: "STANDARD", country: "AR", }, ]; </code></pre> The first block is the test data. It captures a small set of representative input shapes that you've observed in real usage or reconstructed safely from production behavior. The next block is the test itself. <code>it.each(cases)</code> tells Vitest to run the same differential comparison once for every input in that array. That separates two concerns: defining realistic cases and defining how every case should be evaluated. Now: <pre><code class="language-typescript">it.each(cases)( "matches legacy behavior", async (input) => { const legacy = await legacyProcessor.process( structuredClone(input) ); const migrated = await newProcessor.process( structuredClone(input) ); expect( normalizeOrder(migrated) ).toEqual( normalizeOrder(legacy) ); } ); </code></pre> Real examples help expose assumptions that synthetic test data often misses. But production data must be handled carefully. Remove or anonymize: <pre><code class="language-text">personal data credentials tokens financial identifiers confidential business data </code></pre> The objective is to preserve useful behavioral shapes, not copy sensitive production information into test fixtures. <h2 id="heading-classify-every-difference">Classify Every Difference</h2> A differential failure doesn't automatically mean that the new implementation is wrong. Suppose you find 200 mismatches. Classify them. I like categories such as: <pre><code class="language-text">migration defect legacy defect intentionally preserved intentional behavior change representation difference nondeterministic difference test/comparator defect unknown </code></pre> For example: <pre><code class="language-text">Input: subtotal = 5000 Legacy: discount = 0 New: discount = 500 Classification: unknown </code></pre> Investigation reveals that the new implementation changed: <pre><code class="language-typescript">amount > 5000 </code></pre> to: <pre><code class="language-typescript">amount >= 5000 </code></pre> Now you need a decision. Was that: <pre><code class="language-text">accidental migration change </code></pre> or: <pre><code class="language-text">intentional bug fix </code></pre> Differential testing exposes the decision. It doesn't make the decision for you. That's one of its greatest benefits. <h2 id="heading-how-to-use-ai-to-investigate-differential-failures">How to Use AI to Investigate Differential Failures</h2> Large migrations can produce hundreds or thousands of differences. And AI can help triage them. Suppose you have: <pre><code class="language-json">{ "input": { "subtotal": 5000, "country": "AR" }, "legacy": { "total": 4500 }, "new": { "total": 4000 } } </code></pre> You can give the model: <ul> <li>the input </li> <li>both outputs </li> <li>relevant legacy code </li> <li>relevant migrated code </li> <li>the comparator rules </li> </ul> Then ask: <pre><code class="language-text">Analyze this differential test failure. Identify the smallest behavioral difference that could explain the mismatch. Compare the legacy and migrated implementations. Return: 1. observed difference, 2. relevant legacy branch, 3. relevant migrated branch, 4. likely cause, 5. evidence supporting the cause, 6. additional test cases that could confirm it. Do not decide which behavior is correct. Do not modify the code yet. </code></pre> That last instruction matters. AI can be very useful for locating why two implementations diverge. It shouldn't silently turn that diagnosis into a business decision. <h3 id="heading-dont-let-ai-decide-which-behavior-is-correct">Don't Let AI Decide Which Behavior Is Correct</h3> Imagine the legacy system does this: <pre><code class="language-text">Customer age 65 → no discount Customer age 66 → discount </code></pre> The new system does: <pre><code class="language-text">Customer age 65 → discount Customer age 66 → discount </code></pre> AI may look at the code and say: <blockquote> The new implementation appears more logical because senior discounts typically begin at age 65. </blockquote> That's irrelevant. The business rule might be: <pre><code class="language-text">age > 65 </code></pre> for a reason. Or the legacy behavior might contain a bug. You need evidence. Use: <pre><code class="language-text">requirements existing tests production behavior business owners historical tickets commit history contracts </code></pre> AI can help gather and summarize that evidence. It shouldn't invent the rule. Differential testing is valuable because it tells you that there's a difference before you accidentally turn that difference into production behavior. <h2 id="heading-how-to-use-shadow-traffic-safely">How to Use Shadow Traffic Safely</h2> Once offline differential tests look good, you can sometimes compare behavior with real traffic. This is often called shadowing or traffic mirroring. The pattern looks like: <pre><code class="language-text">real request │ ├────────────→ legacy system │ │ │ ↓ │ real response │ └────────────→ new system │ ↓ shadow result </code></pre> The user still receives: <pre><code class="language-text">legacy response </code></pre> while the new system processes a copy of the request. Then you compare: <pre><code class="language-text">legacy output vs. shadow output </code></pre> This can reveal cases that your test suite never captured. For example: <pre><code class="language-text">unexpected null combinations rare customer states unusual international data old records large values unusual sequence patterns </code></pre> But shadow execution requires careful design, especially when the operation has side effects. <h3 id="heading-how-to-prevent-shadow-execution-from-duplicating-side-effects">How to Prevent Shadow Execution from Duplicating Side Effects</h3> Imagine shadowing: <pre><code class="language-text">POST /payments </code></pre> If both systems really execute the payment, you have a serious problem. The same applies to: <pre><code class="language-text">send email create shipment charge card modify inventory publish event write external record </code></pre> The shadow implementation shouldn't perform destructive or externally visible effects unless they're safely isolated. One approach is to replace real gateways with recording adapters: <pre><code class="language-typescript">class ShadowPaymentGateway implements PaymentGateway { calls: PaymentRequest[] = []; async charge( request: PaymentRequest ) { this.calls.push(request); return { paymentId: "shadow", }; } } </code></pre> The new implementation still tries to execute: <pre><code class="language-text">payment </code></pre> but instead of charging a real card, the shadow adapter records: <pre><code class="language-text">what would have been sent </code></pre> You can then compare that intent with the legacy side effect. This distinction is important: <pre><code class="language-text">compare behavior </code></pre> does not mean: <pre><code class="language-text">duplicate production effects </code></pre> <h2 id="heading-measure-divergence-instead-of-waiting-for-perfection">Measure Divergence Instead of Waiting for Perfection</h2> When running thousands of comparisons, a binary: <pre><code class="language-text">pass / fail </code></pre> may not tell the whole story. You can measure divergence. For example: <pre><code class="language-text">Requests compared: 100,000 Equivalent: 99,620 Different: 380 Divergence rate: 0.38% </code></pre> Then classify those 380: <pre><code class="language-text">250 timestamp differences 80 known intentional changes 30 comparator problems 15 migration defects fixed 5 still unexplained </code></pre> After normalization: <pre><code class="language-text">meaningful unresolved divergence: 5 / 100,000 = 0.005% </code></pre> Now the conversation becomes much more concrete. Instead of: <blockquote> I think the migration is ready. </blockquote> you can say: <blockquote> We compared 100,000 representative executions and have five unresolved behavioral differences. </blockquote> Whether that's acceptable depends on what those five cases are. One incorrect financial transaction can matter more than 100 harmless formatting differences. So don't evaluate only the percentage. Evaluate the severity. <h2 id="heading-how-to-know-when-youre-ready-for-cutover">How to Know When You're Ready for Cutover</h2> Differential testing doesn't give you a universal threshold. But it can give you evidence. Before cutover, I would want to answer questions such as: <h3 id="heading-have-important-input-classes-been-compared">Have Important Input Classes Been Compared?</h3> Not only happy paths. Include: <pre><code class="language-text">boundaries errors historical bugs large values missing values rare states </code></pre> <h3 id="heading-are-meaningful-differences-classified">Are Meaningful Differences Classified?</h3> Avoid: <pre><code class="language-text">we have 47 unexplained mismatches </code></pre> <h3 id="heading-are-critical-differences-resolved">Are Critical Differences Resolved?</h3> Especially: <pre><code class="language-text">money authorization state transitions data integrity external contracts idempotency </code></pre> <h3 id="heading-are-intentional-differences-documented">Are Intentional Differences Documented?</h3> If the new behavior intentionally differs, that should be explicit. <h3 id="heading-are-side-effects-equivalent">Are Side Effects Equivalent?</h3> Not only responses. <h3 id="heading-have-production-like-cases-been-tested">Have Production-like Cases Been Tested?</h3> Synthetic fixtures alone may not be enough. <h3 id="heading-can-the-migration-be-rolled-back">Can the Migration Be Rolled Back?</h3> Differential confidence reduces risk. It doesn't eliminate the need for rollback. If you can answer these questions, you're much closer to a controlled cutover. <h2 id="heading-a-practical-differential-testing-workflow">A Practical Differential Testing Workflow</h2> Here's the workflow I would use. <h3 id="heading-1-pick-one-capability">1. Pick One Capability</h3> For example: <pre><code class="language-text">Process Order Calculate Invoice Approve Customer </code></pre> Don't compare the whole platform at once. <h3 id="heading-2-define-the-observable-contract">2. Define the Observable Contract</h3> List what matters: <pre><code class="language-text">return value status error database state events external calls </code></pre> <h3 id="heading-3-create-legacy-and-new-adapters">3. Create Legacy and New Adapters</h3> Expose both implementations through the same conceptual interface. <h3 id="heading-4-define-normalization-rules">4. Define Normalization Rules</h3> Decide how to handle: <pre><code class="language-text">timestamps generated IDs ordering representation changes optional values </code></pre> Do this before looking at lots of failures. Otherwise you may weaken the comparator simply to make results pass. <h3 id="heading-5-compare-known-cases">5. Compare Known Cases</h3> Begin with: <pre><code class="language-text">existing tests characterization cases edge cases historical bugs </code></pre> <h3 id="heading-6-capture-side-effects">6. Capture Side Effects</h3> Use recording or fake adapters where necessary. <h3 id="heading-7-automate-the-harness">7. Automate the Harness</h3> Produce structured output for every mismatch. For example: <pre><code class="language-json">{ "caseId": "case-493", "equivalent": false, "legacy": {}, "migrated": {}, "difference": {} } </code></pre> <h3 id="heading-8-classify-differences">8. Classify Differences</h3> Use categories: <pre><code class="language-text">defect intentional change normalization issue nondeterminism unknown </code></pre> <h3 id="heading-9-add-representative-real-world-cases">9. Add Representative Real-World Cases</h3> Use anonymized or safely reconstructed production patterns. <h3 id="heading-10-shadow-real-traffic-when-appropriate">10. Shadow Real Traffic When Appropriate</h3> Only after controlling side effects and privacy risk. <h3 id="heading-11-measure-divergence">11. Measure Divergence</h3> Track both: <pre><code class="language-text">frequency severity </code></pre> <h3 id="heading-12-resolve-unknowns-before-cutover">12. Resolve Unknowns Before Cutover</h3> The most dangerous category is often not: <pre><code class="language-text">different </code></pre> It is: <pre><code class="language-text">different and nobody knows why </code></pre> <h2 id="heading-what-differential-testing-cant-prove">What Differential Testing Can't Prove</h2> Differential testing has an important limitation: it compares the new system against the old one. That means the legacy system becomes a behavioral reference. But the legacy system may already be wrong. Suppose: <pre><code class="language-text">legacy output = wrong new output = same wrong result </code></pre> The differential test passes, but that doesn't make the behavior correct. This is why differential testing should complement: <pre><code class="language-text">specification tests characterization tests business requirements security testing performance testing contract testing domain review </code></pre> It answers: <blockquote> Did behavior change? </blockquote> It doesn't automatically answer: <blockquote> Is this the right behavior? </blockquote> That distinction matters. The legacy application is evidence, it's not absolute truth. <h2 id="heading-differential-testing-turns-migration-risk-into-evidence">Differential Testing Turns Migration Risk into Evidence</h2> There's another reason I like this technique. Without differential testing, migration discussions can become subjective. One person says: <blockquote> The new implementation looks ready. </blockquote> Another says: <blockquote> I do not trust it yet. </blockquote> Both may have reasonable instincts, but neither statement is very measurable. Differential testing changes the conversation. Now you can say: <pre><code class="language-text">12,000 cases compared 47 differences found 31 representation differences 9 intentional behavior changes 6 migration defects fixed 1 unresolved </code></pre> That is a much better engineering discussion. You're converting uncertainty into observable differences. Then you can decide what to do with them. <h2 id="heading-conclusion">Conclusion</h2> A legacy migration is not complete because the new implementation passes its own tests. The harder question is whether it preserves the behavior that matters from the system it is replacing. Differential testing gives you another way to answer that question. Run both implementations with the same inputs. Compare outputs. Compare errors. Compare side effects. Normalize only the differences that truly do not matter. Investigate everything else. And when possible, use representative production behavior to discover cases your test suite did not anticipate. The migration sequence now becomes: <pre><code class="language-text">Understand ↓ Characterize ↓ Refactor ↓ Migrate ↓ Compare ↓ Cut over </code></pre> AI can accelerate this process too. It can help build comparators, analyze failures, group similar divergences, inspect code paths, and suggest additional test cases. But it should not decide which implementation is correct. That still requires evidence, domain knowledge, and engineering judgment. The purpose of differential testing is not to eliminate uncertainty completely. It is to make uncertainty visible before you switch production traffic. Because during a migration, discovering that the new system behaves differently is useful. Discovering it after the old system has been turned off is much more expensive.

How to apply

1 Read the full details above and confirm you meet the eligibility criteria.
2 Prepare your documents — an updated CV, and any cover letter, proposal or certificates required.
3 Click Apply on official site to complete your application on Hugo Teijiz’s official page.
4 Submit as early as possible — many close once filled.

Apply on official site

Sourced from freecodecamp. Always verify details on the official website. Opportunihub never charges you to apply.

Frequently asked questions

How do I apply for How to Use Differential Testing During a Legacy Migration?

Review the full details and eligibility on this page, prepare your documents, then use the “Apply on official site” button to complete your application on Hugo Teijiz’s official page.

Is this opportunity remote or location-based?

This opportunity is remote-friendly and open to applicants who can work from anywhere.

Is How to Use Differential Testing During a Legacy Migration free to apply for?

Opportunihub lists this Course for free. Legitimate Courses do not ask for payment to apply — never pay a fee to submit an application.