An AI coding assistant fixes a failing test. The build turns green, and the change looks small. During review, an engineer notices why the test now passes: the assistant removed the assertion that checked the original business rule.
This is the problem with treating a passing build as proof that AI-generated code is safe. The code can compile, the tests can pass, and the change can still be wrong. AI makes it easier to produce software changes. It does not remove the responsibility to establish what those changes do.
How should teams test and secure AI-generated code?
Teams should treat AI-generated code like any other untrusted proposed change: define the expected behavior before implementation, inspect the complete diff, run independent functional and security checks, test failure paths and access boundaries, review new dependencies and infrastructure changes, and require an accountable engineer to approve the result. The level of review should increase with the consequence of a defect. Neither a successful build nor an AI review is sufficient evidence on its own.
This is the third article in Centizen's AI engineering series. The first examined how SaaS engineering teams are changing. The second established a reliable AI-assisted development workflow. Here we focus on a narrower question: what evidence does a team need before trusting AI-assisted code in production?
1. More code is not the same as more verified software
AI tools can generate a service method, its unit tests, documentation, and a pull-request summary in one session. That is useful. It also creates a verification problem: several artifacts may come from the same interpretation of the requirement. If that interpretation is wrong, the generated tests can reinforce the mistake rather than catch it.
For example, a requirement says that a tenant administrator may view only orders in their own tenant. An agent adds a filter to the standard order-list query and writes a test using a single tenant. The test passes. But another endpoint, an export job, or an order-detail lookup may still accept an order ID from a different tenant. The original feature appears complete while a critical access boundary remains untested.
The independent check is not to ask the same agent whether it wrote secure code. It is to test the intended invariant from outside the implementation: a user from tenant A must not retrieve or modify tenant B's order, regardless of the endpoint or guessed identifier.
2. Write the acceptance rules before the agent writes the code
Quality begins before generation. When requirements are ambiguous, the agent fills the gaps with assumptions. For routine UI styling, that may be inexpensive to correct. In payments, authentication, inventory, or customer data, the same assumption can become a production incident.
For each task, record the behavior that must remain true. A short task contract is enough: intended change, out-of-scope modules, critical invariants, failure cases, and verification owner. Keep these expectations in the issue or pull request so the reviewer can compare them with the implementation.
| Task contract field | Example: multi-tenant order API |
|---|---|
| Expected change | Add an admin endpoint to download a selected order invoice. |
| Invariant | The authenticated user can access invoices only for the active tenant and authorized role. |
| Negative case | A valid user requesting another tenant’s invoice receives an authorization failure. |
| Out of scope | No changes to payment state, tenant assignment, or invoice calculations. |
| Evidence | Authorization tests, audit of data retrieval path, full diff review, and API integration test. |
| Reviewer | A developer familiar with tenancy and the authorization model. |
Do not ask the agent to decide the business rule and also judge whether the result fulfills it. Engineers should provide the rule, then check the finished change against it.
3. Test behavior at several levels, not just the function the agent changed
AI-generated unit tests help speed up coverage, but passing unit tests do not establish end-to-end correctness. A useful test strategy separates fast local feedback from independent checks of the surrounding system.
| Test level | What it should establish | Typical failure missed by a narrow test |
|---|---|---|
| Static and type checks | Source can be built and follows baseline rules. | Incorrect business behavior that still compiles. |
| Unit tests | A bounded function handles expected and invalid inputs. | Wrong integration assumptions or missing tenant context. |
| Integration tests | Database, queues, APIs, and third parties interact correctly. | Transaction boundary or retry errors. |
| Contract tests | Public request/response and event formats remain compatible. | Breaking change to a consumer not present in the agent context. |
| Security and negative tests | Unauthorized or malformed requests fail safely. | IDOR, privilege bypass, injection, information disclosure. |
| Concurrency and recovery tests | Concurrent requests, duplicate events, timeouts, and retries preserve invariants. | Double processing or inconsistent state. |
| Release smoke tests | The deployed feature behaves under real configuration. | Environment-specific failures. |
Beware of tests that merely confirm the generated behavior
Reviewers should inspect new or modified tests as carefully as production code. A deleted regression test, a weakened assertion, or a mocked dependency can produce a green pipeline while reducing confidence. This risk is explicitly discussed in the OWASP Secure Coding with AI Cheat Sheet.
For important business and security rules, obtain independent evidence. That could be acceptance tests prepared from the original requirement, tests written by a second reviewer, or integration checks that exercise public interfaces instead of generated internals. A second AI reviewer may find useful issues, but it is not a substitute for a trustworthy test oracle.
4. Security review must cover the agent's changes and its operating environment
There are two separate concerns. First, the generated application code may contain vulnerabilities. Second, the coding agent itself may access files, execute shell commands, install packages, invoke external tools, or modify CI configuration while performing the task. Both need controls.
Review the entire change surface
Inspect authentication, authorization, input validation, data handling, logging, and error behavior. Also review less obvious files: dependency manifests, lockfiles, package scripts, Dockerfiles, build workflows, infrastructure configuration, and agent-instruction files. A seemingly harmless feature change should not silently add a package, weaken a pipeline gate, or expand runtime privileges.
Apply least privilege to coding agents
Provide only the code, tools, and credentials needed for the task. Use isolated workspaces, restricted network egress where practical, scoped tokens, and approval checkpoints for sensitive operations. Do not expose production secrets to an agent working on routine feature code. Anthropic describes filesystem and network sandboxing as ways to limit the actions its coding agent can perform without approving every low-risk command.
Do not treat repository text as trusted instructions
Issue descriptions, documentation, comments, external websites, and tool results are data that the agent reads. They may contain malicious instructions. OWASP recommends treating them as untrusted inputs and monitoring unexpected edits, tools, and destinations. Project instruction files such as CLAUDE.md or AGENTS.md deserve code review because they influence future agent behavior.
5. Human review should be risk-based and evidence-driven
Human accountability does not mean requiring a long manual meeting for every typo fix. It means a person understands the intended effect, has sufficient evidence to accept it, and can be identified as the owner of the change. A risk-based approach helps teams spend review time where a defect would have the greatest consequence.
| Change risk | Examples | Suggested review approach |
|---|---|---|
| Low | Copy change, contained formatting, non-executable docs. | Automated checks plus lightweight owner review. |
| Moderate | Internal feature logic, non-sensitive APIs, limited refactor. | Full diff, unit and integration tests, knowledgeable peer review. |
| High | Authentication, tenant isolation, payment logic, financial calculations, data migration. | Domain reviewer, negative and concurrency tests, security review, controlled rollout. |
| Critical | Production credentials, permissions, deployment gates, destructive operations. | Separate approval authority, explicit change management, restricted execution and rollback plan. |
These are suggested tiers, not a universal policy. A regulated application or sensitive client project may require stricter approvals. The key is to classify change risk explicitly instead of assuming every generated pull request is equally safe.
A five-question reviewer checklist
- Scope: Did the agent change only the authorized behavior and files?
- Correctness: Can the engineer explain the implementation and trace it to acceptance criteria?
- Security: Are trust boundaries, input handling, authorization, dependencies, and secrets protected?
- Evidence: Do independent tests cover failure paths, regressions, and critical invariants?
- Release: Are monitoring, ownership, rollback, and approval ready?
GitHub supports AI-assisted code review with configurable review effort. That can help triage findings, but a reviewer should evaluate the output against repository policies and actual business behavior. A polished AI-generated summary is not an audit of every altered line.
6. Practical SaaS example: a tenant-scoped invoice download
Consider a SaaS application with many customer organizations. A product team asks an AI agent to add an endpoint that allows an administrator to download an invoice. This is an illustrative example, not a reported Centizen implementation.
What the agent can help with
The agent can inspect established API patterns, draft a route and service method, propose tests, and explain which modules are affected. It might also identify an existing invoice renderer and avoid rebuilding that functionality. Those are useful implementation tasks when the instructions and file scope are clear.
What the engineer must verify
The engineer must confirm the active tenant is derived from trusted authentication context, not accepted from a caller-controlled parameter; that the invoice lookup includes tenant and role authorization; that responses do not leak invoice metadata through errors; and that export logs do not expose customer payment information.
Before approval, tests should attempt a cross-tenant invoice ID, an expired session, an insufficient role, a deleted invoice, malformed input, and repeated downloads. For a high-volume export endpoint, concurrency and rate-limit behavior may also matter. A green test that exercises only a valid invoice request is not enough.

7. Turn verification into repeatable CI/CD evidence
Manual reviews become inconsistent when teams produce more changes. CI/CD should preserve baseline checks regardless of who or what wrote the code. A practical pull-request pipeline can run formatting and type checks, unit and integration suites, dependency and secret scanning, static security analysis, and tests for key business invariants. A protected branch and required reviewer approvals prevent bypassing these checks to get a release out quickly.
Build systems should also flag high-risk files, including lockfiles, infrastructure definitions, agent rules, deployment configuration, and modified tests. Teams can use our related Centizen article on GitLab CI/CD quality gates as a complementary discussion of enforcing these checks in a pipeline.
Do not optimize only for the percentage of checks passing. Track why tests fail, how often developers override checks, and whether new incidents appear after deployment. Security scans can miss business-logic flaws, and a passing coverage threshold does not prove that assertions are useful.
8. Accountability continues after the code is merged
A named engineer or team must own the change after release. They should know what was modified, why it was accepted, where evidence is recorded, and what to do when it fails. For client delivery, the same principle extends to confidentiality requirements, approved AI tooling, auditability, intellectual-property terms, and the client's change-management process.
Release monitoring should look for signals related to the change: authorization failures, request error rates, unexpected database access, duplicate operations, performance regressions, and support tickets. A staged rollout or feature flag can reduce exposure when the change carries material risk. Teams should retain a practical rollback path instead of relying on AI to fix a production incident under pressure.
9. Measure verification cost and outcomes, not just AI coding speed
A faster first draft can still be more expensive if it requires repeated correction or causes a regression. Measure total lead time from requirement to accepted release, review effort, failed checks, rework, escaped defects, change failure rate, and time to recover. For high-risk modules, also examine whether new negative and integration tests discover problems before production.
A useful pilot compares similar tasks using the same quality expectations. Avoid publishing percentage improvements based on tool-reported productivity or isolated anecdotes. Results depend on developer experience, repository maturity, task type, test infrastructure, and review discipline.
10. A practical starting point for engineering leaders
Begin with one repository and a small set of bounded tasks. Write a one-page acceptance template. Classify sensitive modules. Verify that agent permissions exclude production credentials. Require full-diff inspection and review of modified tests. Add a few independent security and regression checks to CI/CD, then review results after several completed changes.
If the process creates excessive review overhead, adjust task scope or automation before expanding agent access. If defects appear despite passing tests, improve the test oracle and integration coverage instead of simply asking the model for more tests. This is an engineering improvement cycle, not a one-time AI rollout.
Frequently asked questions
Is AI-generated code less secure than human-written code?
Not necessarily. Security depends on the code, development environment, dependencies, and verification process. AI coding agents also introduce additional tool-access and prompt-injection risks that require controls.
Are AI-generated tests enough to verify AI-generated code?
No. They can be useful, but an independent check of the requirement is essential, especially for authorization, payments, concurrency, and security-sensitive behavior. Review test deletions, mocks, and weakened assertions.
Who is responsible when AI-generated code causes a production incident?
The engineering organization remains accountable through its established ownership, approval, and incident processes. AI does not replace the responsibility of the engineers and teams that accept and release software changes.
Should every AI-generated change receive the same level of review?
No. Apply review proportional to risk. Routine documentation edits may need lightweight checks, while authentication, tenant isolation, financial workflows, and deployment permissions need stronger independent verification and approval.
What are the most important quality gates for AI-assisted development?
Start with scope validation, deterministic build and static checks, independent behavioral and security tests, accountable human review, and controlled release with monitoring and rollback readiness.
Conclusion: the standard is evidence, not authorship
The useful question is not whether an AI model or a person typed the code. It is whether the team can demonstrate that the change meets the requirement, respects security boundaries, survives meaningful tests, and has a responsible owner. AI can accelerate implementation. Good engineering determines whether the result is ready to run.
Building quality controls into AI-assisted software delivery?
Centizen supports organizations with custom software development, DevOps and release reliability, and AI quality and testing. A sensible first step is to review one high-impact development workflow and identify where independent evidence or approval controls are missing.
Sources and further reading
- OWASP, Secure Coding with AI Cheat Sheet
- NIST, Secure Software Development Framework (SP 800-218)
- Anthropic, Beyond Permission Prompts: Making Claude Code More Secure and Autonomous
- GitHub Docs, About Copilot Code Review
Editorial note: The tenant-invoice example and review tiers are illustrative recommendations, not reported Centizen customer outcomes or productivity measurements.
Our services:
- Staffing: Contract, contract-to-hire, direct hire, remote global hiring, SOW projects, and managed services.
- Remote hiring: Hire full-time IT professionals from our India-based talent network.
- Custom software development: Web/Mobile Development, UI/UX Design, QA & Automation, API Integration, DevOps, and Product Development.
Our products:
- ZenBasket: A customizable ecommerce platform.
- Zenyo payroll: Automated payroll processing for India.
- Zenyo workforce: Streamlined HR and productivity tools.