Ask one AI agent to review a pull request and you usually get a reasonable list: a possible null, a missing test. What you rarely get is a bug in code the diff didn’t touch, like a second call site that needed the same change. A single reviewer reads only what it was shown, through one lens.
This article describes an alternative: a small council of narrowly focused AI reviewers on the same change, plus a step that tries to knock down their findings before you see them. It’s based on a personal skill I use daily, which does most of the setup on its own, and the idea works in any harness that can launch subagents.
1. What Is an AI Review Council?
A review council is a handful of agents reviewing one code change in parallel. Each owns one failure mode, such as correctness, security, tests, or concurrency. They don’t see each other’s output. A coordinator then merges their findings into one ranked review.
Adding agents without splitting up their responsibilities mostly adds noise. The benefit comes from each reviewer looking for a different kind of failure. A general reviewer asked to “check everything” spreads its attention thin and tends to stop at the first few plausible issues. A reviewer told “you only care about swallowed errors and fallbacks that hide faults” is much more likely to open every catch block. That focus depends on how precisely you define each reviewer’s job.
2. Give Each Agent a Job, Not Just a Personality
“Act like a senior security engineer” describes a personality. It doesn’t tell the agent what to look for or when to stop, so give each persona a job instead: a charter (what it hunts), the access it needs (just the diff, or the whole repo), and when to include it.
Three roles are worth including on almost every run:
- Correctness Reviewer. Logic errors, inverted conditions, unhandled branches, broken builds. Without it, the council is commentary.
- Untouched-Areas Sweeper. Every other reviewer looks at what changed. This one looks for what should have changed and didn’t: other call sites, sibling implementations, the matching test, the config for the other environment. In practice it’s the highest-yield lens.
- Steelman. It argues the change is right as written and points to code that already answers likely objections. Without it, a sound change can get buried under nits, and nobody argues for the author’s choices.
Everything else is picked per change. Seventeen personas cover most changes, and a typical run needs five or six. A few examples are below, and the full catalogue is in the skill’s repository:
| Persona | Hunts for | Include when |
| Silent-Failure Hunter | Empty catch blocks, fallbacks that mask a fault | Error handling, retries, or defaults |
| Test Reviewer | Would any test fail if this change were reverted? | New logic needs coverage |
| API / Consumer Advocate | Removed fields still read elsewhere, new required parameters | A shared contract has more than one consumer |
| Scope / YAGNI Skeptic | Speculative parameters, flags nobody asked for | The diff is bigger than the intent implies |
Two rules keep the panel useful. First, personas must not overlap, or two reviewers become one lens run twice and their agreement looks like consensus when it isn’t. Second, never merge an adversarial persona with the Steelman. Their jobs contradict each other.
Every persona also gets the same contract:
Rule 6 matters more than it looks. Without explicit permission to return nothing, agents tend to produce something, and that something is usually noise. A good contract still depends on its inputs, though, and every reviewer must see exactly the same change.
3. Run the Council in Your Own Harness
The workflow has four steps, none tied to a vendor. Build them into your skill so they run on their own when it starts on a branch, and keep everything it writes out of the repository. If you’d rather have review built into an agent’s coding loop, VirtusLab’s open-source Orca runs parallel read-only reviewers and feeds their findings to a fixer until they come back clean.
Review the branch as it is. Reviewers work read-only on the local branch, or on a separate worktree if you want to keep coding while the review runs. They start from the diff against the git merge-base and read as much of the surrounding code as they need, because many bugs sit between the changed lines and the code around them.
Establish intent. Take what the change was supposed to do from the ticket, the PR body, or one sentence from the author. If nobody can say, write “intent: not stated” and skip the spec-compliance lens. Inferring intent from the diff makes that lens circular: it will always confirm that the code does what the code does.
Write one shared briefing. Have your skill write a single markdown file to a temporary directory outside the repo. It holds the base SHA, intent, test commands, pointers to conventions the project already documents (lint config, CLAUDE.md, sibling code). Keep your own suspicions out of it. If every reviewer reads “I think the timer is wrong”, they all chase the timer and stop covering their own lens. Put suspicions in a separate file that only one persona reads, a Suspicion Investigator that settles each one as confirmed, refuted or unclear. Recommend it whenever you’ve named a suspicion. A file, rather than the diff pasted into every prompt, keeps reviewers on identical facts and your context small.
Launch reviewers in parallel. One subagent per persona, each with fresh context and nothing but the briefing and its contract. Asking a single model to “be” six reviewers in turn brings back the shared blind spot you were trying to avoid.
Here is a meta config, just to visualize the setup:
You don’t need a different model per role, since separate contexts and narrow charters do most of the work. Spend the stronger model on the adjudicator, where most of the judgment happens.
4. Turn Multiple Opinions into One Useful Review
Raw reviewer output is a pile of reports nobody wants to read. Two stages turn it into a review.
Stage one: try to refute every Major. For each Major finding, a skeptic tries to prove it wrong: the guard exists elsewhere, the path is unreachable, the test does cover it. The skeptic defaults to REFUTED when it can’t show the problem is real, and “could in principle” counts as a refutation. Security is the exception: a secret that could end up in a log is the finding itself, so that skeptic should keep plausible paths. For a “no test covers this” finding, a skeptic that runs code beats one that reads it. Break the line, run the suite, and if a test fails, the gap isn’t real. Minors skip this step. Without it, a council mostly generates noise.
Stage two: adjudicate. One coordinator gets every report, the skeptics’ notes, the diff, and the briefing. It makes decisions instead of concatenating lists:
- Merge duplicates, noting when several lenses converged.
- Resolve splits by re-reading the cited code. Counting votes doesn’t work, because two reviewers can agree and both be wrong.
- Rank Majors by blast radius.
- Collapse similar Minors into one line with a count.
- When a finding conflicts with the stated intent, show both and leave the call to the author.
The final report has a fixed shape: a verdict, the confirmed Majors, missed sites (in their own section, or readers assume anything outside the diff is noise), the Steelman’s case for the change, resolved disagreements, a “checked and fine” list of refuted findings, aggregated Minors, and what wasn’t reviewed. The “checked and fine” list shows the author what was examined and why each point was dismissed. If one of them comes up again in a later round, it’s easy to check whether that reason still holds.
Fixing the Majors raises the question of when to stop, since each extra round finds less. Run another round on the fixes while a round still confirms Majors. Once a round turns up only Minors, stop and give the remaining fixes a targeted review of just the changed lines. The process is easier to judge on a concrete change.
5. Walk Through a Realistic Code Change
To test the council, I picked a change whose outcome I already knew: strapi/strapi#26165. It adds a timer that logs an idle admin out when their session expires. A maintainer approved it, and within a month users reported two bugs it introduced: active users logged out mid-edit (#26571), and an idle tab logging out all the others (#26677).
The council got only what a reviewer had on merge day: the diff, the PR description, and the repository at that commit, with no git history. Five reviewers ran, picked for an authentication change: Correctness, Untouched-Areas Sweeper, Steelman, Security and Concurrency/State. Agents vary between runs, so I ran the council three times.
One caveat: the PR and its bug reports are public, so they may be in the model’s training data. The single-agent baseline below had the same advantage and still missed things, but a private repo would make a cleaner test.
What they found, in all runs:
- The main bug (all five reviewers). The timer resets only when the token is refreshed, and the token is refreshed only after a request fails. So an active user gets logged out every 30 minutes, which is #26571. The PR description claimed the opposite.
- The cross-tab bug (Concurrency, Security and the Sweeper, plus Correctness in one run). Tabs don’t share a refreshed token, but logout is shared, so an idle tab can log out the one you’re working in. That’s #26677.
- A timing mismatch (the Sweeper, with Security or Correctness). The timer follows the access token’s lifespan, not the server’s idle-session setting.
- A problem nobody reported (Correctness, joined by the Sweeper in one run). Reloading the admin panel with an expired stored token now logs the user out instead of quietly refreshing the session.
- The Steelman defended the PR’s other half, a redirect after a failed request, and called the timer unsound.
The skeptics confirmed the same four Majors in every run, including both bugs users later reported, and downgraded the rest. The clearest cut was Security’s complaint that the timer never ends the session on the server. The PR never promised that and nothing got worse, so the skeptic made it a Minor.
Strapi later fixed the two reported bugs in #26680. Each council run took eleven or twelve agent runs and 420,000 to 470,000 tokens.
For comparison, I gave a single agent the same inputs and one prompt with every check the council ran on this PR: all five charters, the same contract, and an instruction to try to refute each Major before reporting it. I ran it three times. Each run used 58,000 to 64,000 tokens. All three flagged the main bug and the timing mismatch as blocking. Two ranked the cross-tab bug as a Major, and the third only mentioned it among missed sites. One found the reload problem, and another looked at the same behavior and called it correct. Two runs also raised a Major the council never listed: a client clock running ahead of the server can log users out right after login.
With the full checklist, a single agent got close to the council for about a seventh of the tokens. Its three runs didn’t agree on what was blocking, though, and the council’s did. One PR and three runs per setup don’t prove much, but on this change that agreement is what the extra tokens bought.
6. Decide When the Council Is Worth It
A council is the expensive option, and it shouldn’t be the default.
Costs. Five reviewers plus skeptics plus a coordinator means about seven times the tokens of a single agent running the same checks, and it takes longer. Personas overlap at the edges however carefully you define them. Agents on the same model also share some blind spots, which different charters reduce but don’t remove.
Filtering has a cost too. A strict skeptic can downgrade a real bug, and then it never makes the list of Majors.
When it pays off. Risky or wide changes: auth and tenancy, data migrations, shared contracts with several consumers, hot paths, cross-repo work. Also when a previous review missed something and you want independent lenses. I’d expect consistency to matter more on large diffs, where a single pass has more to drop, though that’s worth measuring on your own PRs. The council’s report also puts several perspectives on each issue into one entry. Correctness traces the path that breaks, the Sweeper lists the other sites that need the same fix, the skeptic’s notes show what was ruled out, and the Steelman marks what’s worth keeping. The author gets context and a clearer idea of how to fix the issue, without reading five reports. For a 40-line config tweak, a single reviewer is cheaper and usually enough.
How to compare it fairly. Run the council and a single-agent review on the same recent PRs and, instead of counting comments, measure:
- confirmed useful findings (ones the author acted on),
- false positives the author had to dismiss,
- minutes the author spent reading and checking the review.
A council that produces three times the comments and the same number of real fixes is a regression, even if it looks more thorough. The Strapi test can’t give these numbers, because no author ever acted on the council’s review. The closest I have is that both bugs users later hit were among the four confirmed Majors.
The same measurements work for a managed reviewer like Claude Code Review from Anthropic, generally available since October 2026. It follows a similar pattern (several agents, a verification step, findings ranked by severity), but it runs as a service on GitHub pull requests and focuses on correctness by default. A council of your own lets you choose the lenses, and review a branch before any PR exists.
7. From One Branch to a Whole Team
Most of the value comes after the reviewers finish. The skeptic step is what cuts the noise down to a few confirmed Majors, and it’s also the easiest to get wrong, since a skeptic that’s too strict throws out real bugs. Keeping it tuned for one developer’s branches is manageable. Doing it on every pull request, for a team that ships a lot of AI-generated code, is a different job.
That’s the problem we’re working on with Visdom Code Review. Our experience designing code review agents, this council included, is going into it.
The difference we’re aiming for is in what a verdict means. A council like this one is best effort: it reviews as well as it can, and a clean report only means nobody found anything. Visdom is meant to make a PASS mean that the quality bars your team requires were met. Some of the code review will always be a matter of taste, so this covers the bars a team can state and check. Prompts alone can’t give that guarantee. It takes benchmarking the reviewer against known outcomes, and that’s the hard part.
It’s still a work in progress, and we’d rather shape it together with teams who face this problem than guess what they need. If that’s you, let’s talk: we’re happy to compare notes. You can see where it stands today in the docs.
Whether you build your own council or use a tool, judge the review by how many of its findings the developer actually acts on.
Special thanks to Jakub Bocheński, Adam Warski, Piotr Kukiełka, Artur Skowroński, Przemysław Wojnowski for their feedback and peer review of this blog post.