Devint

Performance evaluation

AI coding ROI: how to calculate the return on copilots and agents

Eduardo Missaka
CEO and Founder of Devint
Published · Updated · 12 min read
In this article

The AI budget renewal has arrived, and the board needs to know what the tools returned. The math is short: the return in money, minus the total cost, divided by the total cost. The hard part is what goes on each side and what to measure before calculating. This guide walks through three questions, in the right order: is the team really using it, is it working, and is it worth the spend?

In short

  • AI coding ROI is the return in money minus the total cost, divided by the total cost. The hard part is what goes on each side of the equation.
  • Before calculating, define the baseline and the measurement window for each squad. Without a “before”, there is no return to measure.
  • Follow three layers, in this order: is the team using it, is it working, and is it worth the spend?
  • The cost goes beyond licenses, and hours saved only become return when they are reallocated to something that creates value.
  • The team’s perception is not enough: the METR studies show how hard it is to measure this effect.
  • Bring the board a range of scenarios and the trend over the cycles. The number opens the conversation; it does not decide promotions or dismissals.

Why proving the return on AI became a board question (and why easy metrics mislead)

With every license renewal or increase in agent consumption, the question moves up from the CTO to the Founder/COO and the board: what did this return? “The team liked it” cannot be the answer.

The temptation is to use what is at hand:

  • active licenses;
  • suggestion acceptance rate;
  • lines of code generated;
  • commits and PRs per person.

All of them can be collected and are easy to inflate. None of them says whether the work delivered improved.

Perception is not a measurement either. In 2025, a randomized study by METR followed 16 experienced developers across 246 real tasks in mature repositories. With AI allowed, tasks took 19% longer, yet participants estimated they had become about 20% faster.

In February 2026, METR published an update with a second study, started in August 2025, with 57 developers and more than 800 tasks. The raw numbers suggest some speedup, but with wide intervals that include zero, and METR itself considers this data a weak signal of AI’s current effect. The reason is selection bias: many developers no longer accept working without AI, and between 30% and 50% said they stopped submitting tasks they did not want to do without the tool. This tends to underestimate the gain. METR believes the 2026 gain is probably larger than in 2025, but with weak evidence.

The lesson for the CTO is not that AI slows teams down or speeds them up. It is that measuring this effect is hard, even for researchers, and that the team’s feeling does not support a calculation presented to the board.

Anyone who has managed thousands of developers knows the pattern: a number without a ceiling becomes a target, and a target becomes theater. If AI return is measured by volume of use or of code generated, the volume goes up and the return does not always follow. That is why the calculation below is done in layers, and each one protects the next.

Before any calculation: define the baseline and the measurement window

Without a “before”, there is no return to calculate. If the tool has not been expanded yet, record it now. If it has, rebuild the past from Git and delivery data, which are usually available. The minimum to keep, per squad:

  • delivery pace: PRs merged per week and time to merge;
  • code review time;
  • rework: reopened items and fixes right after delivery;
  • monthly squad cost (payroll and tools), which later allows converting gains into money.

The measurement window must cover a few complete delivery cycles, to absorb seasonality such as vacations and quarter close, and the first weeks of the learning curve, when performance may even drop before it rises.

The cleanest comparison is the squad against itself before and after, or against a similar squad that has not adopted yet. Avoid overlapping the measurement with other big changes, such as hiring, reorganization or a stack change: when everything changes at once, no one can say what caused the result.

Layer 1, is the team using it? Measure adoption in the real workflow

An active license is not usage. The first layer separates who has access from who uses it, and who uses it from who changes the way they work with the tool. Three signals help to see this:

  • frequency: in how many weeks of the period the person actually used it;
  • depth: autocompleting snippets of code or delegating whole tasks to agents;
  • diversity: which AI tools the squad combines day to day.

Native dashboards already help to get started. The Copilot impact dashboard, launched in July 2026, groups users by adoption phase, from those who mostly use it for code to those who work with agents, and isolates the passive segment of people who have a license and do not use it. It is a good starting point for one tool.

The reading becomes more reliable and easier to accept when it follows three rules. First: read by squad and by person, in reference ranges, and not as a generic ranking. Second: transparency with the team, where each developer sees their own overall result and per dimension, sees the squad result as a whole, and never the results of individual colleagues. Third: look at the trend each cycle, with no real-time monitoring. Measuring the work does not have to feel like surveillance of people.

Finally, using more AI cannot become a goal in itself. Beyond a certain point, more usage does not bring more return. What matters is usage tied to results, and that is what the next layer checks.

Layer 2, is it working? Observe the effect on delivered work

With usage mapped, compare the same baseline signals between those who use it and those who do not, or between before and after:

  • delivery pace and time to merge;
  • review time;
  • rework and post-delivery fixes;
  • code that stays in the product, instead of being rewritten shortly after.

Be wary of the gain from an isolated task. A GitHub experiment with 95 developers, in which the task was to write an HTTP server in JavaScript, recorded 55% faster completion in the group using Copilot. It is a relevant data point, but it comes from the tool’s vendor, covers a single task and does not include review, integration or maintenance. Read alongside the METR studies, it shows that the result changes with context, and that your context only shows up when you measure your own.

Speed can also turn into cost. If code comes out faster, but review takes longer and rework rises, the gain dissolves along the way. That is why the signals must be read together, each one acting as a counterweight to the others.

The DORA 2025 report helps explain why squads with the same license perform differently: according to the report, AI acts mainly as an amplifier, and the biggest returns come from the organization’s system of work, more than from the tool itself.

Layer 3, is it worth the spend? The calculation step by step

Only with the two previous layers does the final number have a foundation. The formula:

ROI = (Return − Total cost) ÷ Total cost

1. Add up the total cost (denominator). Start with licenses, agent and API consumption, amortized onboarding and governance, and the extra review time. The list varies with the stack, the contract and the degree of agent autonomy, so add whatever is specific to your operation, such as dedicated infrastructure, security review or other model tools in use.

2. Convert the return into money (numerator). Use what layer 2 measured: freed capacity, faster deliveries and avoided rework, valued at the baseline monthly cost of the squad. Note: an hour saved only becomes money when it is reallocated to something that creates value. Capacity left over without a destination does not enter the calculation.

3. Calculate and read it as a range. Present the result with the range of scenarios (conservative and optimistic), and not as a single number.

An illustrative example, with fictional values only to show the mechanism, for a squad of 10 people with a monthly cost of R$ 200,000 (Brazilian reais):

Item Monthly value (R$)
Licenses (10 × R$ 250) 2,500
Agent and API consumption 3,500
Onboarding and governance (amortized) 1,500
Extra review time 2,500
Total cost 10,000
Freed and reallocated capacity (8% of squad cost) 16,000
Avoided rework 4,000
Return 20,000

In this fictional scenario, the ROI is (20,000 minus 10,000) divided by 10,000, that is, 1.0, which equals 100%. In a real calculation, each value in this table must come from a measurement in the previous layers, and not from a back-of-the-envelope estimate. That is what makes the number defensible before the board.

Where the calculation usually fails and how to take the number to the board

Four failures appear frequently:

  • the average that hides: part of the team uses it a lot, part does not use it, and the average number represents no one;
  • several tools in parallel: the dashboard of one of them shows only part of the usage;
  • wrong attribution: the gain from another change, such as a new review process, is credited to AI;
  • inflated number: when usage becomes a target, usage goes up, even without results.

Two objections usually come up when the calculation reaches the table. The first: “every dev metric can be gamed”. The answer is in the design: ceilings per dimension and structured human evaluation alongside the automatic signals, so that inflating one isolated number does not improve the final result. The second: “I already have dashboards”. Native dashboards are useful and, in the case of Copilot, come included, but, according to GitHub’s own description, they cover one tool and organize usage by adoption phase and PR throughput. Git and Jira dashboards cover delivery, but do not show AI usage. Linking usage to each person’s results, across several tools, requires joining these sources.

There are three paths for this: your own spreadsheet fed by Git data, which has a low cost but depends on manual work and tends to produce divergent versions across areas; the native dashboards of each tool; or a dedicated platform. Devint is one of the options on this third path: it treats AI and agent adoption as one of the six DevScore dimensions, links usage to each person’s results, with a ceiling so that using more does not become a goal, and the score composition is open. Consolidation happens in biweekly closings, and the developer sees their own overall and per-dimension score, plus the squad score, without access to colleagues’ scores.

When taking it to the board, present the range of scenarios, the trend over the cycles and what changed in the operation. The number opens the right conversation: continue, expand or adjust the AI investment. It does not decide anyone’s promotion or dismissal.

Next step

In practice, ask first whether the team uses it, then whether it works, and only then whether it is worth it. With a baseline, complete cost and measured return, ROI stops being opinion and becomes a calculation the board can follow.

Want to see how this looks in Devint? Talk to the Devint team.

Frequently asked questions

How do you calculate the ROI of AI copilots and agents in software development?

Subtract the total cost from the return in money and divide the result by the total cost. Before the calculation, define a baseline per squad and answer three questions, in order: is the team using it, is it working, and is it worth the spend? Each value of the return should come from a measurement, and not from a back-of-the-envelope estimate.

Which costs go into the AI ROI calculation?

Licenses, agent and API consumption, onboarding, governance and the extra code review time. The list varies with the stack, the contract and the degree of agent autonomy, so add whatever is specific to your operation, such as dedicated infrastructure or security review. Forgetting an item overstates the return.

Do hours saved with AI count as return?

Only when they are reallocated to something that creates value. Capacity left over without a destination does not turn into money and stays out of the calculation. That is why the return considers the freed and reallocated capacity, the avoided rework and the accelerated deliveries that the squad measurement showed, valued at the team’s monthly cost.

How do you keep AI usage metrics from being gamed?

Do not treat usage as a goal. Combine signals that keep each other in check, such as delivery pace, review, rework and code that stays, and set ceilings so that using more does not raise the result on its own. A structured human evaluation alongside the automatic signals reduces the room to inflate an isolated number.

Is measuring AI usage per person surveillance?

It does not have to be. What changes is the design: trend reading each cycle, no real-time monitoring, an open result composition and transparency with the team. In Devint, each developer sees their own overall and per-dimension score and the squad score as a whole, but never the scores of individual colleagues.

Sources

See the DevScore applied to your team.

Book a demo