Before I decide whether a team's token burn is high or low, I think it is important to first understand what that number represents. An agent may read a large part of the repository, try a change, run the tests, and repeat that cycle several times before it produces something an engineer wants to keep. The token total captures all of that activity, including the work that led to the answer and the work the agent threw away along the way.
That context matters because two tasks can consume the same number of tokens for different reasons. A migration may require the agent to understand several packages and the boundaries between them. A small bug may consume the same amount because the agent read the wrong files and took a few attempts to find the problem. Both runs produce a bill, although I would make a different decision about each one.
I would start by breaking token usage down by model, task type, and service. This gives me enough information to see whether a workflow keeps loading the same large files, whether a smaller model can handle a common task, or whether one area of the codebase requires more context than we expected. I can use that information to improve how we provide context and route work without treating every expensive run as waste.
I would also keep token usage away from individual performance. A low total may come from an engineer who gave the agent clear context and kept the task narrow. It may also mean the engineer stopped using the tool. A high total may represent an agent completing a difficult upgrade that saved the team time, or it may represent an agent getting stuck on a test it wrote. The token count does not include enough information to make that distinction for me.
This is where I think DORA metrics become useful. Token usage gives me the cost and volume of the AI activity, but the code still has to move through review, testing, deployment, and whatever work comes back afterward. I wrote more about that handoff in The human in the loop owns the loop, because generating the change is only one part of the delivery process.
DORA currently uses five software delivery performance metrics:
- Change lead time
- Deployment frequency
- Failed deployment recovery time
- Change fail rate
- Deployment rework rate
The first three help me understand throughput, while change fail rate and deployment rework rate help me understand instability. I would look at both groups as token usage increases because faster implementation can put more work into review and production before the rest of the delivery process has changed.
For example, a team may start using an agent and produce pull requests in less time. If review, testing, and deployment can absorb that work, I would expect change lead time to improve and deployment frequency to increase. If those pull requests become larger or harder to review, the implementation time may go down while the change waits longer somewhere else. Looking at the full lead time helps me see whether the team saved time or moved it into another queue.
I would apply this to one service first and compare that service with itself over time. DORA recommends measuring one application or service at a time, which helps account for differences in architecture, deployment processes, and test coverage. I would establish a baseline, mark when the team introduced the AI workflow, and track token spend alongside lead time, deployment frequency, failures, and rework.
I would also annotate the larger changes happening around the same time. A release-process improvement can reduce lead time. A migration can increase failures or rework for a period of time. A quieter roadmap can make the delivery numbers look better even if the AI workflow had little effect. DORA metrics can show me that performance changed, but I still need the team and the surrounding context to understand why.
The 2025 DORA report on AI-assisted software development found a positive relationship between AI adoption and software delivery throughput. The report also found a negative relationship with delivery stability. I think that combination is useful when looking at token burn because it gives me a reason to check both the additional output and the work created after deployment.
I would add a few local measures where they help explain the DORA results. Review wait time can show whether faster implementation created a new bottleneck. Pull-request size can tell me whether the team started sending reviewers larger batches. I would also ask the team which tasks they gave to the agent and where they had to redo or heavily edit the output. Those measures are more specific to the workflow, so I would use them to explain the delivery metrics instead of turning them into another score.
I would be careful with any ratio that divides token burn by deployments and labels the result efficiency. AI usage can support design work, incident investigation, or a task that ends with the team deciding to leave the code alone. Those outcomes may still have value even though they do not produce a deployment. A ratio built around deployments would also give the team a reason to create smaller deployable changes whether those changes improved the product or not.
Once we have enough data, I would look for a pattern between the spend and the delivery results. If token usage grows while lead time improves and rework stays stable, I would have more confidence that the workflow is helping. If implementation gets faster while review wait time and rework increase, I would look at batch size, test quality, and the context engineers provide before expanding the workflow.
The DORA report describes AI as an amplifier, which fits how I think about the rollout. A team with fast feedback, useful tests, and a deployment process people trust has a place to use the additional output. A team with a slow review queue will give the agent more work to put into that queue. Token burn helps me understand what we spent to create the work, and the delivery metrics help me decide what to change next.