Last week’s post covered why AWS Transform lost the tooling decision for porting a serviceability calculator from .NET Framework 4.8 to .NET 10. This one covers what won: Anthropic’s Claude models, used across the whole lifecycle rather than as a fancy autocomplete. Evaluation, transformation, and deployment each used them differently, and the differences are the interesting part.

A note on framing before the detail. The constant through this work is the models (Opus for the architecture-bearing reasoning, Sonnet for the mechanical bulk) and the working method built around them; the front-end driving those models is a choice each team makes for itself. I’ll come back to the driver options after describing the harness, because the harness was deliberately built to survive that choice.

Phase one: evaluation as a writing exercise

The migration didn’t start with code; it started with a _plans/ directory that grew to sixteen documents. Claude wrote most of the words, but the value wasn’t the typing. It was that producing a plan of record forces every vague intention into a falsifiable statement, and an LLM that has actually read the codebase is ruthless at finding the vague ones.

The plan of record went through seven versions. v6 to v7 alone removed a VPC Link (single AWS account, private API Gateway can reach an internal ALB directly, saving about AUD$25 a month per environment and a network hop), and then a v7 revision moved LIXI XML translation out of the calculator entirely into a separate Lambda, so the .NET service stays JSON-only forever. Alongside the plan sat a pre-transformation assessment that produced the manual remediation backlog: 15 numbered items for the calculator, sized XS to L, from “delete the App_Start folder” to “replace LINQ-to-SQL with an HTTP client”. That backlog became the unit of work for everything that followed; every later commit traces to an item number.

The habit worth stealing: keep the plans in the repo, next to the code, and let the agent read them. Every subsequent session started from documents the model could load rather than context I had to retype, and the documents kept getting corrected as sessions found errors in them.

Phase two: the transformation, run by a generated agent team

For the port itself I didn’t hand-write the agent setup. I used the harness plugin, which is best described as a team-architecture factory: you give it a domain description and it generates a full multi-agent team under .claude/ – agent definitions, a per-agent skill each, and an orchestrator that runs them as a coordinated team.

The prompt I fed it specified a six-agent pipeline with a producer-reviewer gate overlaid:

  1. regression-baseline runs first, on the .NET Framework engine, and captures golden-file fixtures: input JSON paired with the exact NSRResult/DTIResponse output the legacy engine produces. The fixtures must include multi-income cases, because PAYE is computed per income rather than averaged, and a single-income suite would not protect that ordering logic.
  2. data-access-architect owns the hard seam: define IReferenceDataProvider, delete the .dbml and every LINQ-to-SQL usage, replace static caches with IMemoryCache on a five-minute TTL.
  3. web-host-migrator ports the host: Program.cs replacing Global.asax, ApiController to ControllerBase, the Razor popup render, the en-NZ globalisation fix.
  4. project-cleanup does the mechanical sweep: SDK-style csprojs, dead references, solution-config simplification.
  5. test-porter moves the test project to MSTest v3 on .NET 10.
  6. regression-gate reviews everyone. After every backlog item it rebuilds and re-runs the golden-file suite, and the pipeline does not advance past any field-level diff.

Two details made this work rather than merely demo well.

First, model routing. The harness defaults every generated agent to Opus, which would be slow and expensive for work that is 70 per cent mechanical. The prompt overrides that with a per-agent split: Opus for the architecture-bearing agents and the gate, Sonnet for cleanup and test porting. Rather than trusting frontmatter to stay put, the prompt makes the routing self-verifying; it emits MODEL-ROUTING.md as the source of truth and a check-model-routing.sh guard that exits non-zero if any agent’s frontmatter or the orchestrator’s team definition drifts from the manifest. The orchestrator runs the guard before the team forms. A silent reversion to all-Opus becomes a loud failure instead of a quiet invoice.

Second, the gate is non-skippable and byte-level. An LLM can shift a rounding or an ordering without noticing, and a green compile proves nothing about a serviceability calculation. Byte-identical golden-file output against the Framework baseline was the only definition of done, one backlog item per commit, and the constraint list baked into every agent included the boundaries no refactor may cross: the public JSON contract is immutable, no direct database connection, and one hard-coded flag that must never grow a branch again.

Later I extended the team by asking for it: two more agents, one owning a Postgres schema plus a small reference-data sidecar service seeded from the committed CSV extracts, one owning Dockerfiles and a three-service docker-compose.yml. The pay-off was proving the seam: the golden-file suite passes with the calculator reading CSVs in-process, and passes again with it reading the sidecar over HTTP. Identical bytes both ways means the provider swap is safe, which is exactly what Wave 2 of the plan needed demonstrated.

Pick your driver

The harness lives in the repo as plain markdown and shell scripts, and that’s what makes the driver a choice rather than a commitment. Three configurations were on the table in this environment, and the runbook documents two of them.

Claude Code runs the harness natively: the orchestrator skill triggers off a one-line request, each agent definition becomes a real teammate under the experimental agent-teams mode, the producer-reviewer gate runs itself through the shared task list, and the Opus/Sonnet routing is applied automatically from each agent’s frontmatter. This is the path of least translation, since the artefacts are in Claude Code’s own format.

VS Code with GitHub Copilot drives the same models where that’s the sanctioned enterprise tooling, with two real differences: Copilot runs as a single agent, so you adopt each persona in turn and enforce the gate yourself by rebuilding and re-running the golden files after every item, and the model routing maps onto Copilot’s model picker by hand rather than a frontmatter field. There’s also an adoption gotcha worth knowing about before you plan around this path: agent mode can be switched off at the org level, and the failure reads “You are not authorized to use this Copilot feature, it requires an enterprise or organization policy to be enabled”. Getting that policy flipped is an admin ticket, not a local setting.

And for organisations that need inference to stay inside AWS, either driver can point at Claude on Bedrock instead of Anthropic’s API; the same Opus and Sonnet models are available there under Bedrock model IDs, IAM carries the auth, and nothing about the harness changes. The method (generated team, difficulty-based routing, a non-skippable byte-level gate) is identical across all three; only the invocation plumbing moves.

Phase three: deployment, where the plan met an SCP

Terraform for the AWS side was ordinary model-assisted work – ECR, the ECS task, the Aurora cluster and its Lambda – until it very much wasn’t.

The design had the calculator fetch reference data from the Lambda via a Function URL with IAM auth. The original code shipped without request signing, so every call in dev returned 403 and the calculator responded with a 500. Fine, that’s a known shape: Claude implemented a SigV4 signing handler (signer, delegating handler, a unit test against the canonical get-vanilla test vector), and the signatures were provably correct. Still 403.

This is where an agent that can run commands earns its keep. The diagnosis step that settled it: assume the ECS task role and sign the same request with botocore, taking our .NET code out of the equation entirely. Still AccessDeniedException. At that point the fault can’t be the signing or the role policy, and the answer turned out to be an organisation-level Service Control Policy that denies lambda:InvokeFunctionUrl for workload roles while exempting interactive ones. Which explains the cruellest symptom: everything works when a human tries it from a shell, and fails only for the running service. The Function URL pattern was dead in this org no matter what IAM we attached.

The fix was a transport swap: call the Lambda directly through lambda:InvokeFunction with the AWS SDK, which signs as the task role without any hand-rolled SigV4. The Lambda handler now accepts a direct payload alongside the HTTP-shaped event it keeps for local dev; Terraform drops the Function URL and grants plain invoke; the compose stack still uses HTTP locally. The hand-built SigV4 code, correct and useless, survives only for local mode, and its removal is an open decision on the TODO list.

There’s a second, quieter deployment-phase win worth recording. Asking Claude to triage the build warnings, rather than suppress them, surfaced two real engine defects hiding in the noise: a throw ex; that resets the stack trace on the student-loan error path, and two catch blocks that swallow an ArgumentException and return 0M from a percentage calculation, so an arithmetic edge case reads as perfect serviceability instead of a failure. Both predate the migration. Neither is the kind of thing you find by reading 8,200 lines with human eyes on a deadline.

What I’d actually claim for this way of working

Not that the models did it alone; the constraint list, the backlog, the gate, and every merge decision were human. The honest division of labour: Claude wrote the plans and kept them consistent, executed a 15-item porting backlog through the generated team without batching unrelated changes, and drove the deployment debugging down to an org-policy root cause I would have circled for days. The golden-file gate carried the correctness burden the whole way, which is precisely what let the AI move fast everywhere else.

The lifecycle framing is the point. Evaluation, transformation, and deployment usually get three different toolchains and three different mental models. Running one model family across all three meant the deployment sessions could read the evaluation documents, and the SCP discovery flowed back into the plan of record the same afternoon. The documents stayed alive because the thing editing them was also the thing doing the work; whichever driver you pick, that property is the one to preserve.