Beyond Coding Speed: Measuring Software Engineering Productivity in the Age of AI

November 14, 2026 | Artificial Intelligence

A developer finishes an API change before lunch with the help of an AI coding agent. By the end of the day, the pull request is still waiting for review. The next morning, an integration test fails. Two days later, the change finally reaches production.

Was the coding faster? Almost certainly. Was the software delivered faster? Not necessarily. The difference matters for every engineering organization considering AI tools, especially teams working on client projects or long-lived SaaS products.

Why doesn't faster AI coding always lead to faster software delivery?

AI can reduce the time spent drafting, modifying, and explaining code. But delivery also depends on requirements, review capacity, integration, testing, security checks, approvals, deployment, and recovery. If those stages do not improve together, saved coding time becomes waiting time, rework, or additional verification. Teams should measure the complete path from task readiness to production outcomes, not only lines of code, prompt counts, or how quickly an agent finishes.

This is the fourth article in Centizen's AI engineering series. Our earlier article on next-generation SaaS engineering teams addressed ownership and organization. We then explored reliable AI-assisted workflows and verification of AI-generated code. Here we examine the question leaders eventually ask: is the whole delivery system improving?

1. Engineering teams operate on three different clocks

The first clock measures how long a developer spends implementing a change. The second measures how long it takes a completed change to travel through review, testing, integration, and deployment. The third measures how quickly the customer receives a reliable improvement that solves the intended problem.

AI can make the first clock move faster without changing the other two. A feature that takes two hours to code but sits in a review queue for three days is not a three-day improvement over a feature that took six hours to code. The customer still waits for a production release.

MeasurementQuestion it answersWhat it can miss
Implementation effortHow much developer time did this task require?Review, QA, integration, deployment, and rework
Delivery lead timeHow long did the change spend moving toward production?Whether the shipped feature solves the customer problem
Customer outcomeWas usable, reliable value delivered?Developer experience and internal workload if used alone

For planning, track all three. Use the first to understand task productivity, the second to expose workflow constraints, and the third to keep engineering focused on outcomes rather than activity.

Three separate clocks for AI coding productivity: implementation effort, end-to-end delivery lead time, and reliable customer outcome.
Faster code generation is a local gain; delivery is a whole-system outcome.

2. What current research actually tells us

The evidence is more useful than either extreme claim that AI will transform every engineering organization overnight or that it never helps.

Google's 2025 DORA research, informed by nearly 5,000 technology professionals, describes AI as an amplifier of existing organizational capabilities. Better underlying practices help teams turn faster local work into delivery improvements; weak delivery processes can become more visible under the extra load. DORA also reported an association between AI adoption and higher throughput, alongside instability concerns. These are research associations, not proof that any individual tool caused a particular outcome.

In DORA's March 2026 analysis, researchers describe a common tension: time saved generating code is often redirected into auditing and verification. That work is not necessarily waste. A proper security review or negative test may be what prevents a fast change from becoming a production incident.

The independent research organization METR found that experienced contributors working on familiar open-source projects with early-2025 tools took 19% longer on tasks when AI was available. Its February 2026 update said later experiments were affected by participant selection and measurement limitations. The researchers considered improving results plausible but could not reliably estimate the size of any improvement. These findings should not be generalized to all developers, tasks, or newer tools.

The practical lesson: measure your own workflow. Neither positive self-reports nor one controlled study is a substitute for observing actual engineering outcomes in your product and team.

3. Where saved coding time goes

Pull-request queues

When developers or agents produce more changes, reviewers may become the constraint. Reviewers still need to understand the requirement, inspect behavioral changes, identify security risks, and check tests. Bigger AI-generated diffs can consume more time than several small, well-scoped changes.

A better response is not simply to add an AI reviewer. Require manageable pull requests, clear acceptance criteria, explicit risk labels, and concise explanations of what changed. Automated checks should remove routine noise before human review begins.

Integration and regression testing

Code that works in isolation may break existing APIs, database assumptions, or user workflows. For a SaaS system, a change to order creation can affect payment events, inventory, notifications, refunds, and analytics. Faster code generation increases the importance of contract and end-to-end tests for these connected behaviors.

Generate test candidates with AI if useful, but let engineers define critical invariants independently. The team should not rely on the code-generating agent's own tests as the only evidence of correctness.

Build and environment constraints

Slow CI jobs, shared test databases, flaky test suites, and inconsistent staging environments create queues that AI coding cannot remove. A developer may finish early and still spend hours waiting for the pipeline or fixing a failure unrelated to the feature.

Track median and upper-percentile pipeline duration, flaky test frequency, and the time lost waiting for usable environments. Fix the longest recurring delays instead of merely increasing code output.

Approval and release coordination

Some changes require product confirmation, security review, data migration planning, customer scheduling, or production access approval. Those steps exist for reasons. The goal is to make approval timely and risk-based, not to remove it from sensitive work.

Illustrative delivery timeline in which faster AI coding saves implementation hours but review queues, testing, and release coordination still determine elapsed delivery time.
Illustrative bottleneck map. The specific constraint varies by team and application.

4. Worked SaaS example: a two-hour change that still takes days

Consider a team adding a new discount eligibility rule to a SaaS checkout service. A coding agent proposes the implementation and unit tests in two hours rather than the six hours the developer initially estimated. This is a hypothetical comparison, not Centizen performance data.

The pull request then waits a day because the reviewer is assigned to several other changes. QA finds that the rule behaves incorrectly when a coupon and a gift card are used together. The team fixes the interaction, adds an integration test, and waits for the next controlled release. The code was created faster, but the time to deliver a safe customer-facing feature changed very little.

StageBefore AI (illustrative)With AI (illustrative)
Initial implementation6 hours2 hours
Review queue1 day1 day
Integration check and fixes1 day1 day
Release windowNext scheduled releaseNext scheduled release
OutcomeFeature shipped after safeguardsSimilar elapsed delivery time despite 4 hours less initial coding

Now change the process, not just the coding tool. Make the acceptance criteria include coupon-and-gift-card interactions before implementation. Assign a reviewer when the task starts. Run the affected integration tests automatically on the pull request. Release smaller, independently deployable changes where the architecture and risk allow it. Only then is the shorter coding time more likely to translate into a shorter end-to-end lead time.

For critical commerce workflows, keep the checks that matter: duplicate requests, concurrent inventory changes, payment failures, and order consistency. Removing verification to improve a speed chart is not a delivery improvement.

5. Build a measurement system that reflects delivery, not activity

DORA's current framework uses five software delivery metrics, split into throughput and instability. This includes change lead time, deployment frequency, failed deployment recovery time, change fail rate, and deployment rework rate. These are more meaningful for operational performance than counting generated lines of code.

MetricWhat to monitorWhy it matters with AI
Change lead timeTime from commit to productionShows whether faster work actually flows to release
Deployment frequencyFrequency of successful production changesShows whether the system can ship increments
Failed deployment recovery timeTime to restore service after a deployment failureShows operational resilience
Change fail rateDeployments that require interventionSurfaces quality and release risk
Deployment rework rateUnplanned deployments to repair production incidentsCaptures costs hidden by faster initial coding

Add a small number of local diagnostic indicators: pull-request time to first review, review rework cycles, CI queue time, flaky-test rate, escaped defects, and developer-reported friction. These are not additional official DORA metrics; they help explain why your DORA outcomes change.

Measure at the team or service level, not as a leaderboard for individual developers. A developer who writes fewer lines but prevents a dangerous migration may contribute more than someone who generates many pull requests. Avoid performance targets that encourage splitting changes artificially or bypassing checks to improve the metric.

Proposed engineering metrics dashboard separating DORA throughput, delivery instability, and diagnostic review and pipeline measures without fabricated data.
Measure output, stability, and the sources of delay together; do not use a single productivity score.

6. How to test whether AI improves your team's delivery

Start with one application or delivery stream rather than announcing a company-wide productivity target. Select a representative sample of changes: routine UI tasks, API updates, bug fixes, and a few integration-heavy changes. Avoid mixing very different tasks into one average.

  1. Define the baseline. Record recent task cycle times, review wait, CI duration, rework, production failures, and release cadence. Note holidays, outages, and unusual projects.
  2. Define the intervention. Document which AI tools and workflows teams may use, including context, permissions, required tests, and review gates.
  3. Compare similar work. Prefer comparable task categories and teams; a before-and-after comparison alone cannot isolate AI from staffing or process changes.
  4. Measure total effort. Include prompt preparation, code verification, test fixes, reviewer time, and rework, not just initial implementation hours.
  5. Watch for quality regressions. Inspect escaped defects, rollback frequency, and customer-impacting incidents while delivery volume changes.
  6. Decide what to keep. Continue practices with measured benefit, refine bottlenecks, and stop workflows that create more risk or effort than value.

For an initial internal review, a four-to-six-week observation window can help identify directional signals, but it will not automatically provide statistically reliable causal evidence. The right evaluation duration depends on task volume, variability, and release cadence. If results are inconclusive, say so.

7. Improve the bottleneck before increasing AI output

An engineering lead can use a weekly flow review to make this actionable. Look at the work that was started, the work that reached customers, and the work still waiting. Identify the oldest or riskiest items, then ask which waiting stage they share.

Observed problemPractical adjustmentGuardrail
PRs pile upSmaller changes, review ownership, scheduled review capacityNever auto-approve sensitive changes for speed
Repeated test failuresStabilize tests, improve fixtures, add contract checksDo not hide flaky or failed checks
Long CI queuesParallelize independent checks, cache dependencies, right-size runnersPreserve required release evidence
Integration surprisesEarlier cross-module review, representative integration testsKeep product and data invariants explicit
Releases wait on coordinationSmaller deployable batches and clearer release ownershipHonor contractual, security, and production approvals

DORA recommends using a team discussion to find the biggest constraint, selecting one improvement, measuring the result, and repeating the cycle. This is a stronger use of engineering attention than simply encouraging more AI tool usage.

Continuous improvement loop for AI-assisted software delivery: map work, find the constraint, improve it, retain safeguards, measure outcomes, and repeat.
A recurring improvement loop makes local coding speed relevant to end-to-end delivery.

8. Client projects and SaaS products need different measures of success

In a client project, delivery may include contractual acceptance, customer demonstrations, security requirements, change-control procedures, and handover documentation. AI coding speed matters only if it reduces total effort or improves outcomes without undermining those commitments. Track the time from approved scope to client acceptance, not just developer task completion.

For a SaaS product, delivery continues after deployment. A change that ships quickly but causes subscription billing errors, regressions, or increased support tickets is not necessarily a net gain. Track production stability, support impact, and maintainability alongside release cadence.

The measurement approach should reflect those differences while keeping one principle consistent: delivery is a team and system outcome, not a count of AI-generated artifacts.

Frequently asked questions

Does AI make software development faster?

It can make certain tasks faster, such as drafting code, tests, and documentation. Actual gains depend on the task, developer, tool, repository, and effort needed to review and integrate the output. Measure total task effort rather than assuming every AI-assisted change is faster.

Why can faster coding make reviews and testing slower?

AI may increase the volume of changes entering queues. Reviewers and test systems still have finite capacity, and generated code may require additional verification. Smaller changes and dependable automated checks can help prevent the bottleneck from moving downstream.

Which metrics should engineering leaders track after adopting AI?

Use the five current DORA delivery metrics for throughput and instability. Add local measures such as review wait, CI duration, rework, escaped defects, and developer effort to explain changes. Do not rely on lines of code or pull-request count alone.

Should teams reduce testing to get more benefit from AI?

No. Reduce wasteful waiting and redundant checks where justified, but retain tests and approvals that protect business rules, security boundaries, and production reliability. Fast code that creates incidents can increase total delivery cost.

How can we demonstrate that AI improves delivery?

Establish a baseline, evaluate comparable task categories with a clearly documented AI workflow, include review and rework effort, and observe deployment and quality outcomes. Report uncertainty and avoid presenting correlation as proof of causation.

Conclusion: optimize the path to production, not the typing

AI can be an important engineering productivity tool. The stronger opportunity is to improve the system around it: better task definition, smaller changes, earlier feedback, reliable tests, review capacity, sensible approval boundaries, and controlled releases. When those parts work together, teams have a better chance of converting local speed into customer value.

The useful question for a CTO is not, “How much code did AI write this month?” It is, “Are we delivering valuable changes more reliably, with less waiting and less rework?”

Continue the Centizen AI Engineering series

Read the companion guides for deeper treatment of related engineering decisions.

Looking at AI-assisted delivery across your engineering organization?

Centizen works with organizations on custom software development and SaaS engineering. For practical quality controls, read our guide to GitLab CI/CD quality gates for AI-generated code. A useful starting point is one workflow map and a baseline of where changes currently wait.

Research and further reading

Editorial note: The checkout example, time comparisons, measurement plan, and recommendations are illustrative. They are not reported Centizen internal measurements or client outcomes. Evidence on AI coding productivity varies across studies and contexts.

Our services:

  • Staffing: Contract, contract-to-hire, direct hire, remote global hiring, SOW projects, and managed services.
  • Remote hiring: Hire full-time IT professionals from our India-based talent network.
  • Custom software development: Web/Mobile Development, UI/UX Design, QA & Automation, API Integration, DevOps, and Product Development.

Our products:

  • ZenBasket: A customizable ecommerce platform.
  • Zenyo payroll: Automated payroll processing for India.
  • Zenyo workforce: Streamlined HR and productivity tools.