We read 2,675 code-review threads. 65% were opened by AI bots.

§ 01The numbers

• The split is not even. In 6 of the 18 apps, bots opened 90% or more of all threads. In 4, almost none.
• About a dozen different AI review bots appeared across the 18 apps.
• We picked the 79 reviews with a real risk, a visible fix and a lesson. In 30 the bots alone drove the fix; in 34 bots and humans both did; in 15 only humans did.

¹ A later commit touched the line the thread was on. That's a sign the comment was acted on, not proof.

§ 02What the bots caught: mechanics

The bot findings that changed code were mostly about how the code runs:

• a status read in one query and the update made in the next, so two requests could both pass the check;
• a “processing” flag that was never cleared when the next call failed, so the retry skipped the work;
• a job queue that de-duplicated waiting jobs but not the one already running;
• a validation schema that silently dropped a new field before the handler saw it;
• a server action that allowed a change the UI had disabled;
• a regex on inbound email that slowed to a crawl on unclosed tags;
• a “fix flaky tests” change that mocked the database everywhere, overriding the working fake that other tests relied on;
• a client store shared by every request during server rendering, so one user's data could reach the next;
• “never remove the last owner” checked without locking, so two admins could each remove the other.

§ 03What the humans caught: rules, order and scope

The human findings were about what the product allows and what happens when things fail halfway:

• Business rules. A new “apply to another group” path also let banned users back in. The code was correct; the rule wasn't.
• Crash order. A file was deleted before the database row that pointed at it was committed. Stage, commit, then delete.
• Side effects. A new “dry run” endpoint still ran the hooks that call the payment provider.
• Fail-safe defaults. An AI check that ran out of quota answered “not flagged”, and the editor cached it as a pass.
• Scope. A bot's valid security note was reverted because it didn't belong in that PR.
• Tests that test themselves. A test passed the very setting it claimed to check.
• Unsafe defaults. A Dockerfile made the scripts the container runs at start-up writable by every user.
• Meaning of old data. A field being reused already held empty values that old code wrote by accident; the new code would have read them as a user's choice.

§ 04What happens when AI is on both sides

• Coding agents wrote a share of these PRs, and AI bots reviewed them. In one PR the agent applied two bots' contradictory advice.
• Bot findings chain: a leak in the rollback, then a leak in the fix, then in that fix. In another PR, one bot found a bug and a second bot found the bug in its fix.
• Authors often push back and the bot withdraws. Many major bot findings are closed with no reply at all.
• Some PRs had no human reviewer at all: an AI-assisted change, reviewed only by a bot, then merged.
• Some human reviewers paste findings they label as AI-written. And one maintainer kept asking for AI-generated docs and rules files to be taken out of PRs.

§ 05What this means if you build with AI

1. Keep the bot. It catches races, missing guards and leaks faster than a person, on every PR.
2. Give a human the rules. Permissions, status changes and money flows need someone who knows what the product should allow.
3. Ask “what if it fails halfway?” on every PR that writes to two places (database + storage, database + email, database + payment provider).
4. Decide scope. An out-of-scope finding becomes its own ticket, not a silent revert.
5. Watch the silent closes. A bot comment closed with no reply and no change was never reviewed.
6. Feed findings back. Put repeated findings into the rules file your agent and your bot read.

§ 06The 10 questions only a human asked

Run these on your own app:

1. For each new “who may do X” path, which current statuses may use it, and did you test banned users?
2. If your server crashes between the file upload and the database save, which one points at nothing?
3. Does your “preview / dry run” mode skip every side effect?
4. When an AI or fraud check can't run, does the caller see “skipped” or “pass”?
5. If two saves each pass your limit check, is it still a limit?
6. When a file-processing step throws, does your code close the stream it opened?
7. Does each “not found” come from one specific lookup?
8. Does your test assert the app's behaviour, or its own setup?
9. Does every CI step in a sub-folder job actually run in that folder?
10. Is this finding in scope for this PR, or a ticket of its own?