LLM Council: When a Well-Known AI Researcher Independently Validated the Principle Behind Expert Consilium

What Karpathy Actually Built

In late November 2025, Andrej Karpathy - a co-founder of OpenAI, former director of AI at Tesla, and today one of the most influential voices in AI - posted a small project on X (Twitter) called LLM Council, described as a "weekend hobby."

The project, published on GitHub as karpathy/llm-council, runs as a local web app with three clear stages: first opinions (every model answers independently), peer review (each model receives the others' anonymized answers and ranks them for accuracy and insight), and a final answer (one designated "chairman" model synthesizes everything into a single answer).

The default configuration includes models like GPT-5.1, Gemini 3 Pro, Claude Sonnet 4.5 and Grok 4 via OpenRouter. Karpathy was candid about its purpose: this is code written in an afternoon, that he does not plan to actively maintain, meant above all as inspiration for others to adapt to their own needs.

Why This Got So Much Attention

The post quickly crossed over from the usual AI audience. VentureBeat argued that Karpathy's hack actually reveals something more important than the technique itself: routing a query through multiple models is no longer hard to build. The real missing layer for serious business use is governance - authenticating users, protecting data, an audit trail, and consistent reliability of the answers.

Analytics Vidhya published a detailed technical walkthrough of the architecture, while numerous independent writers covered the broader trend of "AI councils" as a possible future standard for more reliable AI answers. Within weeks, more than ten derivative open-source projects had already appeared, inspired by the same idea.

Same Principle, Different Purpose

It's worth being precise here: Karpathy's llm-council is, in his own words, a project he will not actively maintain - as he put it himself, code doesn't need to last long anymore - and it was never meant to be a product a company would rely on for decisions with real financial consequences.

That is exactly where Expert Consilium's methodology picks up where a hobby project naturally stops. The same underlying mechanism - separate parallel answers, anonymous peer review, synthesis by a chairman - has, from day one, been built with us as a process for real business decisions: with 4, 9 or 15 separate AI perspectives, a structured Decision Brief, and a final automated consistency check before the client ever sees the answer.

A Methodology That Actually Keeps Evolving

The open discussion around Karpathy's project has already surfaced real weaknesses - one of the most widely discussed being the risk that a "chairman" model can quietly discard a minority view from the council without explaining why, letting that disagreement simply vanish into the final synthesis.

That is exactly the risk we addressed in our own architecture this week. When the individual models in our council genuinely disagree on the recommended course of action, our chairman model must now explicitly name, in the report itself, which alternative view was set aside and the concrete reason why - weaker evidence, an assumption that doesn't hold here, or a risk the other option leaves unaddressed.

This, we believe, is the difference between an interesting experiment and a methodology entrusted with a real business decision: one is finished the moment it stops being fun to tinker with; the other keeps being questioned and improved.

Karpathy's DIY Approach vs. Expert Consilium

Karpathy's llm-council: self-hosted and self-maintained, 3-4 models in a fixed default configuration, the author explicitly says he won't maintain the code, a raw text answer with no audit trail - great for personal experimentation and learning.

Expert Consilium: delivered as a service with no technical burden on the client, 4, 9 or 15 separate AI perspectives scaled to the decision, a methodology that is continuously developed and documented, a structured Decision Brief with options and risks, an anonymous comparative review before synthesis - built for decisions with real financial consequences.

Conclusion: A Spark of an Idea vs. a Proven Process

Karpathy's weekend hack deserves every bit of the attention it got - it made the case for multi-perspective AI reasoning about as clearly as it can be made. But he himself never claimed it was more than a spark of an idea.

Expert Consilium is that same spark, turned into a proven business process: more separate AI perspectives, a structured synthesis that's transparent about where the models disagreed, and anonymous cross-checking before the decision ever reaches you.