PandaCode's Council Feature: Multi-Agent Review That Catches Real Bugs
PandaCode's Council feature uses fresh AI reviewers with separate lenses to score agent work and catch bugs self-grading misses.

What is PandaCode’s Council feature?
Council is a review system built into PandaCode that replaces an AI agent’s self-assessment with judgment from multiple independent reviewer agents. Instead of an agent declaring its own work “done,” a panel of fresh AI reviewers, each focused on a different concern (called a lens), scores the output. The lowest score sets the overall grade, and the agent keeps revising until it clears a target score or hits a round limit. It was built to solve a specific problem: agents that grade their own homework tend to give themselves high marks, even when the work has real defects.
TL;DR
- Self-grading is unreliable because the agent that builds something only sees its intention, not what’s actually there, the same reason human writers use editors and courts don’t let defendants judge their own cases.
- The gap is measurable: in one test, seven agents scored their own work 9 out of 10 across the board, while separate agents with no context scored the same work between 5.5 and 8.
- Council replaces a 10-day habit of manual prompting where the creator typed a review instruction by hand in 61 separate tasks before it became a built-in toggle.
- Four rules make the review honest: the lowest reviewer score counts (not the average), reviewers never see the target score or prior rounds, every round uses fresh reviewers with no memory of past fixes, and lenses are locked after round one so the agent can’t swap in easier judges.
- Real councils caught real bugs, including a project switcher that silently failed, phantom “installed” skill statuses, a misaligned UI element, a security hole letting view-only phone access read terminal output, and a tagged release that never actually passed CI.
- It’s expensive and slow: one three-day stretch burned roughly 180 hours of agent time across 250 rounds and 837 reviewers, and no council run with a 9.5 target has reached it, typically landing between 8 and 9.
- Vague prompts produce vague reviews, so results improve sharply when you name specific, checkable states and behaviors instead of asking for something “top notch.”
- Self-hosted
- SOC 2
- HIPAA
- Audit log
How does Council actually work?
When Council is turned on for a task, PandaCode assembles a team: a lead agent that plans the work and assigns it to worker agents, same as normal. The difference shows up at the end. The lead agent is no longer allowed to just declare the task complete. It has to convene a council of reviewer agents first.
The lead picks the reviewers and assigns each one a lens, a single concern that reviewer is responsible for judging. One lens might focus on visual layout, another on security, another on whether a diagnosis or explanation the agent gave is actually true. You can leave the number of reviewers on auto, letting the lead decide how many the task needs, or set it manually.
Three configurable settings control the process:
- Target score: defaults to 9 out of 10, adjustable from 7 to 10.
- Max rounds: defaults to 10, meaning the agent gets up to that many rounds of review and revision.
- Number of reviewers: auto by default.
If the lead tries to finish below the target score, PandaCode blocks it and sends it back to fix the flagged problems and call another round. This enforcement happens at the application level, not through a prompt the agent could ignore.
Why does averaging scores fail?
Council uses the lowest reviewer score as the final score, not an average. The reasoning: an average lets strong work in one area mask a serious problem in another. A product with excellent visual design and a critical security hole shouldn’t score well just because the design reviewer loved it. If a security lens scores a 6 while two other reviewers score 9.5, the task scores a 6. This single rule is what forces agents to fix weak spots instead of padding scores with easy wins elsewhere.
Why do reviewers need to be fresh each round?
Every round of Council uses new reviewer agents with no memory of previous rounds. This matters because a reviewer that remembers the prior version stops evaluating the work itself and starts evaluating whether the agent did what it was told. That’s a different (and less useful) question. A fresh reviewer sees the output the way a new user would, with no context about what changed or what was promised. That’s the view that actually catches problems a returning reviewer would overlook.
Reviewers also never see the target score or the scores from earlier rounds. If a reviewer knew the target was a 9, it would likely hand out a 9. If it knew the last round scored a 7, it might inflate its own score to be polite. Keeping reviewers blind to both numbers keeps the scoring honest. Lenses are also locked after the first round, so the lead agent can’t quietly replace a strict reviewer with an easier one partway through.
What kinds of bugs has Council actually caught?
Nothing gets lost.
In one documented run, PandaCode was asked to build a skill manager with instructions that the UX needed to be “top notch.” The first council round returned a lowest score of 5, flagging that switching projects did nothing for tasks running in a separate work tree, that certain skills showed as “installed” across roughly 14 tools regardless of whether those tools existed on the machine, and that a scope switcher was visually misaligned across every screenshot.
The score climbed slowly: 5 to 6 to 7, then stalled between 6 and 7 for about 18 rounds. By round 22 it hit 8.5, when a reviewer found that a failed uninstall could leave a skill half-deleted with no way to recover it. That got fixed, round 23 hit 9, and the run ended at the 25-round cap still at 9, short of the 9.5 target. At that point PandaCode handed the decision back: keep going, lower the target, or accept the current result.
Other examples from real use: a security lens found that phone access with view-only permissions could read the output of every terminal on the connected machine. A redesign of how tool calls appeared in chat was tested against roughly 4,200 old records, which the reviewer flagged as rendering as junk under the new design. A release-review lens caught that a tagged commit had never actually passed CI. Non-code work gets reviewed too: a factual-accuracy lens caught a script’s voiceover stating a dollar figure of $203 when the actual source value was $23.04, and a thumbnail reviewer flagged that the image showed a different product than the video covered.
Is Council worth the cost?
The honest answer, based on the creator’s own numbers, is: sometimes. Across three days of use, councils ran about 250 rounds with 837 total reviewer calls, consuming an estimated 180 hours of agent time. PandaCode’s own interface warns that Council consumes more of a plan’s usage, and that warning is accurate. It’s not something to switch on for small fixes like a typo.
No run targeting a 9.5 score has actually reached it. Scores typically start between 5 and 7 and settle between 8 and 9. Progress is also uneven: one lens scored a task a 6 in round one and a 6 again in round 24 with no improvement. In the skill manager example, going from 5 to 7 took three rounds, but going from 7 to 9 took twenty more, with most of those rounds producing little movement until one round surfaced a serious bug.
Reviewer agents also do real work on the machine they’re reviewing, including compiling code to check it, which in one case filled up disk space. And mixing models matters: three reviewers running on the same underlying model tend to produce one opinion repeated three times rather than three independent judgments.
Frequently Asked Questions
What makes Council different from an agent just checking its own work?
The agent that built something only has access to what it intended to build, which colors how it judges the result. Council uses separate reviewer agents with no stake in the outcome and no visibility into the target score, so their judgment isn’t biased toward a passing grade.
Does a high Council score mean the work is actually good?
Not exactly. A high score means several reviewers, each focused on a different concern, looked closely and didn’t find much to criticize. It reflects the absence of flaws reviewers could detect, not a guarantee of quality against goals the reviewers were never told about.
How should prompts be written to get useful Council reviews?
Reviewers can only check work against goals stated in the prompt, not against unstated taste. Vague instructions like “make it top notch” produce vague reviews. Specific, checkable requirements, naming exact states, behaviors, and interactions, produce reviews that catch real problems.
Can Council be used for non-coding tasks?
Yes. It’s been applied to reviewing scripts for factual accuracy, checking release notes against actual build status, and judging thumbnail appeal and relevance, suggesting the lens-based review approach generalizes beyond code.
What’s the main downside of using Council?
Cost and time. Running multiple rounds with multiple reviewers consumes significantly more compute and plan usage than a single agent finishing a task, and even well-configured runs often fall short of very high target scores within reasonable round limits.
