A small validation change in a customer onboarding application looked like routine work for an AI coding agent. The requirement was clear. The acceptance tests were known. The agent read the relevant files, proposed the change and updated the tests.

The first attempt failed. The agent read more files, explored another path and checked additional dependencies. The final code passed.

The change was accepted. The consumption was much higher than a similar change.

Our first design treated token usage mainly as a reporting issue. We expected teams to review model spend after the work, much like cloud usage. The complete delivery flow showed why that was too late.

The real problem was not the token count. It was whether the consumption could be connected to an authorised feature, the right model class and an accepted engineering result.

This field note describes the internal architecture experiment that changed our view. It shows what we moved into the delivery path, what we measure against a baseline, and what we are still refining.

Our current view: Token cost is part of the engineering control system, not only a finance report.

At a glance

1. Token consumption is useful only when it is tied to an authorised task and accepted output.
2. Context selection, retries and restarts can consume as much attention as code generation itself.
3. Different engineering tasks need different model capacity, limits and approval paths.
4. The useful metric is not tokens per engineer. It is cost per accepted engineering outcome.

What we tried

We built a controlled path for a coding agent to work on bounded application changes. The path combined code generation, architecture checks, testing and traceability.

For each run, we captured the model used, the context supplied and the consumption created. The work used a controlled repository and simulated delivery conditions. It was an internal architecture experiment, not a client production deployment.

We began with a simple operating model: allow the agent to work, record usage and review unusual spend later. Four observations changed that design.

What changed our view

1. Usage without purpose was only an activity measure

Our first dashboard could show model usage by team and period. It could not show why the usage happened. A rise in tokens could mean productive adoption. It could also mean a large context window, repeated failures or exploration that never contributed to the accepted change.

We attached every run to an authorised feature, a stated purpose and an expected output. Consumption then became attributable to a piece of work rather than to a team in the abstract.

2. The cost often started before generation

We initially focused on the code the model generated. The experiment showed that the input context could be just as important. Too many files made every later step heavier. Failed attempts also carried test output, alternative code and prior reasoning into the next cycle.

We narrowed the starting context by task. The agent could request more information, but each expansion was visible. That made unnecessary context growth easier to spot.

3. One model and one ceiling were too blunt

A unit-test change and a cross-system identity change do not need the same reasoning capacity or risk control. A single model and a single token cap treated unlike work as if it were the same.

We moved to task tiers. Bounded work could use a smaller, faster model. Ambiguous or cross-system work could move to a stronger one. Higher-risk changes still required human approval. Each tier had an expected consumption range and an exception path.

We are still comparing alternative models by task type. The comparison includes accepted output, review effort, delivery speed, failures and total cost. Token price by itself does not identify the more economical model.

4. Cost became useful only beside evidence and a baseline

The final code file did not prove value. We also needed passed tests, architecture compliance, review effort, rework and acceptance status.

We grouped completed changes into comparable categories such as defect fixes, minor enhancements, refactoring and cross-system changes. Recent work from the same codebase gave us a baseline for engineering effort, time to acceptance, review effort and defects.

This changed the metric. Tokens per engineer were easy to report. Cost per accepted engineering outcome was more useful.

A model may save 90 minutes of coding but add two hours of review and correction. The code appeared faster. The accepted change did not.

What we first assumedWhat the work showedWhat we changedMonthly usage would explain costUsage did not identify the feature or activity that caused itAttach each run to an authorised feature and expected resultGeneration was the main cost driverBroad context, retries and restarts also created substantial useLimit initial context and make expansion visibleOne model and one cap simplified controlDifferent tasks required different capability, risk control and rangesTier models by activity and add rerouting or approval pathsMore AI activity implied more outputHigh usage did not prove speed, quality or completionCompare full cost with accepted output and a delivery baseline

Exhibit 1: The assumptions that changed

The pattern we currently use has four parts: define, bound, observe and verify. The important change is where the control sits. It moved from a central cost dashboard into the engineering flow.

DefineBoundObserveVerifyApproved featureClear purposeRequired resultModel tierContext limitExpected rangeContext growthRetriesRestarts + costTestsReview effortAcceptance + baseline

Exhibit 2. The control moves into the delivery path, where cost and risk are created.

Define the work. Every run starts with an approved feature and a required result.

Bound the run. The task receives a model tier, a context limit and an expected consumption range.

Observe the activity. The path records context growth, retries, restarts and cost while work is in progress.

Verify the result. Consumption is reviewed beside tests, controls, review effort, acceptance status and the pre-AI baseline.

What technology leaders should test next

Establish one baseline before scaling

Choose one class of software change and measure recent comparable work. Use engineering effort, time to acceptance, review effort, rework and defects. Without a baseline, usage growth is hard to interpret.

Tier work by activity and risk

Separate bounded work, complex reasoning and higher-risk changes. Give each category a default model, a consumption range and an escalation path.

Measure the accepted result, not the generated code

Compare the full cost with accepted output, time to acceptance, review effort and quality. Faster code generation is useful only when it improves the delivery outcome.

Where our thinking has landed - for now

In this experiment, token consumption stopped looking like a finance metric. It became part of the engineering control system. Some difficult changes will justify more model work. The reason, limit and result should remain visible.

We are still refining task tiers, expected consumption ranges and model-comparison thresholds. We are also testing when a run should stop, ask for approval or change its method.

Do not reward the tokens consumed. Control for the engineering result produced.

Are coding-agent costs rising faster than accepted delivery?

Bring us two or three recurring classes of software change where AI coding agents are being used or evaluated.

Ampersand can help determine where model tiers, context limits, runtime controls and acceptance measures should sit in the delivery flow.

Practical output: a bounded coding-agent control model linked to accepted engineering outcomes and a measurable baseline.