Councils of agents

Group thinking with LLMs.

Introduction

It’s been two months since I finished a sequence of LLM-based posts which were intended to think of unique ways that you could improve your leadership skills by leaning into AI as a coach, a contrarian thinker, and a way in which to expand and accelerate your decision-making.

If you’re interested in reviewing those posts, then you can find them here, in reverse chronological order:

• Leadership co-processing with LLMs, which introduces a number of prompting and usage ideas that could help you develop your thinking.

• A bag of worries: tackling overwhelm with LLMs, which is a technique I’ve been using to help me manage my own never-ending to-do list by offloading some of the cognitive load of prioritization to an LLM.

• A weekly mind meld, which uses some LLM assistance to communicate weekly with my department.

• LLMs: an operator’s view, the original post in this series, which covers some of the cultural change addressed in the first post in this list, and how code review and hiring are also changing.

Building on the leadership co-processing article, we’re going to go further this week and think about how we can expand our usage of a thinking partner into multiple thinking partners by using LLM agents to create your own councils that you can use to accelerate and supercharge your thinking and also simulate situations where many actors may not have default consensus on issues.

Inspiration

At work, I’ve mostly been using Claude Code as my go-to interface. Generally speaking, I like the Claude models, which have become particularly good since Sonnet 4.5 and Haiku 4.5 were released. However, I also enjoy the terminal interface, which I always keep open at the side of the screen as it is space efficient. I can mix regular prompting along with the generation of code, and it looks cool as well.

One neat feature of Claude Code, which is the backbone of this article, is that it makes it extremely easy to build agents that you can delegate to. For example, you just type /agents like below:

From here, you can create a new agent.

In order to create an agent, all you have to do is give a rough sketch of what the role of the agent is. For example, here is my initial prompt for a QA Engineer agent.

Claude Code is able to expand that into something far more detailed which you can edit and tweak however you wish.

If you’d like to see more examples of how a number of agents could be defined, then there’s a great public GitHub repository where people have been collecting together their examples. Each of the folders has a different type of agent (or agents) combined with a collection of commands that you can use in order to work with them.

This is especially useful when you’re programming since you can delegate specific types of work to these agents, such as refactoring, security testing, initial code reviews, and so on. Now, none of these are meant to be substitutions for the real thing, however, they become great partners when you’re coding because you can call upon specific functionality when you need it, which is especially useful when you know that you have blind spots. For example, a security engineer agent can continually make sure that you’re aware of these kinds of issues as you code.

One particularly interesting observation, highlighted in Simon Willison’s recent post, is that Claude Code could be considered a general purpose agent, rather than just a coding tool.

Following my own usage, I agree with this viewpoint. Claude Code is not just a tool for coding, but is also a tool to think. And agents let you get really creative with your thinking.

In previous posts, we looked at how we could use simple prompts to “pair think” with LLMs, from organising our time, to cross-checking decisions, to enabling mob sessions where architecture or code is designed together, and so on.

The more senior you get as a leader, the more nuanced and complex some of your decisions can be: not necessarily only because of the impact of those decisions, but also because of the challenge of finding consensus within large groups with differing opinions, biases, and experiences.

Sometimes the act of maintaining fast, synchronous connections with groups of people in order to debate, discuss, and forward your thinking can be blocked by others’ busyness or timezone. That’s where agents come in.

Council of agents

In the same way that there were examples above of how to use agents in Claude Code for specific technical functions (such as refactoring and testing), you can use agents to form specific thinking councils that you can use to accelerate your thinking faster than you could by working either on your own or by requiring synchronous time with others.

This is most easily shown by example, so we’ll go through two of them in this section. The first will be a technical council, and the second will be an executive council.

Creating a technical council

Now that we’ve seen how to build one agent, we can now think about building multiple agents that can make up a council that you can work with and think with. For this first example, let’s consider a technical council that can help you.

Firstly, you’ll need to think about which kind of roles you’d like to have in your technical council of agents. For example, you might want to have the following:

• Principal Engineer, covering broad technical vision, architectural patterns, and cross-cutting concerns.

• Platform Engineer, covering infrastructure, deployment pipelines, developer experience, and tooling.

• Security Engineer, with an interest in threat modeling, secure architecture, compliance, and vulnerability management.

• QA Lead, covering testing strategies, quality metrics, automation, and release confidence.

• AI/ML Engineer, responsible for machine learning systems, model deployment, and intelligent feature design.

Your own mileage may vary on the above depending on what kind of role and company you’re in.

With a decision made about what kind of roles you want to have in your council of agents, you then create them using the same methodology as we did in the example above.

For each of those roles, you create an agent in Claude Code, and you use your initial prompt to outline what they’re responsible for. Then, Claude Code will expand those into more detailed role definitions for the agents, which you can edit if you wish. And then, you continue doing this until you’ve defined your whole council.

Once this is done, it means that you can then ask questions to your whole council of agents. Here’s an example of what that could look like.

Here’s the output that was achieved from that particular query.

For this example, I specified in the query that I wanted just one paragraph of output per agent. However, if you wanted it to generate a detailed report and use one of the more expensive thinking models to do so, and then output that to a file, then the choice is yours.

And that’s it: with a small upfront effort, you now have a model of a technical council at your fingertips that can help you quickly interrogate ideas and decisions taking into account the diverse strengths and biases of each of those particular roles. It’s an incredibly useful tool to have.

Creating an executive council

Here’s one that helps me a lot.

As a CTO, my team is probably the most multi-disciplinary of them all, given that everybody in my executive team runs one of the other functions of the business. I am the only engineer!

Therefore, in order to help with my own thinking, especially about company-wide issues, having an executive agent council is highly valuable, and improves the kinds of proposals and decisions that I can bring to the group. Essentially, I almost get one or two meetings of iteration on my ideas completely for free (token costs aside) by doing it locally with my agent council.

In order to create this executive council, I go through the exact same steps but change the role definitions. For example:

• CEO, owning the overall company vision, board relations, strategic direction, and having final decision authority.

• Chief Product Officer, covering product strategy, roadmap prioritization, user experience, and feature decisions.

• Chief Operating Officer, responsible for operational execution, cross-functional coordination, process efficiency, and delivery.

• Chief Revenue Officer, owning sales strategy, pipeline, revenue targets, and go-to-market execution.

• Chief Marketing Officer, covering brand, demand generation, market positioning, and customer acquisition

• Chief Customer Officer, responsible for customer success, retention, NPS, implementation, and the voice of customer.

Again, depending on what kind of company you work for and who you’d like to have in your executive council, you may want to switch these roles out for whatever is relevant for you.

Here’s an example of the kind of query that you could ask. Let’s assume that the chat functionality that we were previously thinking about with the technical council is now ready to launch: how do we bring it to market?

You can see in the query that I accidentally wrote “rollout” twice, which I guess proves to you that these articles are still written by a human.

What’s fascinating about our query this time is that Claude Code even asked for input from the user to model the company that we’re working at. I did not explicitly ask to be given these choices in the setup.

After thinking for a while, I received output that looks like this. There was far more output than could fit in the screenshot, but for each of the executive council members, there are at least three to five interesting points that I know that we need to think through when we discuss it as a real human team.

Try it yourself

I’ve been finding this council of agents approach really useful for thinking through larger, more strategic plans and also in helping me develop my ideas before I bring them to the human version of the groups they represent. They allow me to do much more asynchronously. And then, when I do connect with my team synchronously, I find that the discussion and ideas that I am bring are more nuanced, considered, and researched.

So why not try it yourself?

Experiment with creating your own councils of agents using Claude Code or other tools and frameworks that support them. Get creative and think about ways in which you could improve your programming, your thinking, and your strategic decision-making.

It’s also a whole lot of fun as well. What councils of agents will you create?