How I Manage AI Agents at Scale (Prompting Is No Longer the Problem)
DevOps & AI Toolkit · 2026-11-09
💡 Quick Take
1. The host created an open-source tool called Agent Deck for running and managing multiple coding agents at the same time.
2. According to the host, stuffing every debugging step and compiler log into a single conversation is a wasteful way to use context.
3. The host recommends separating roles by context, so coders, testers, and reviewers do not share the exact same reasoning trajectory.
4. According to the host, running a team of agents is slower and more expensive than a single pass, but it produces much better software quality.
5. The host notes that local practices and rules should always include the reason behind them, helping agents adapt without falling into the "unless" trap.
📊 Detailed Explanation
Managing a team of autonomous agents changes the core challenge from crafting the right prompt to handling permissions, planning, and context limits. According to the host, when working with dozens of agents, prompts stop being the bottleneck. The real problem is deciding what each agent is allowed to know, who holds the plan, and how to maintain clean slates. To delegate effectively without micromanaging, the host allows agents to decide why a test failed or how to make it green, while strictly reserving non-negotiable boundaries—like required checks and pull requests—for automated guardrails that agents cannot bypass.
To avoid stuffing every detail into one bloated conversation, the host built an open-source application called Agent Deck. This tool helps run and manage many coding agents at once by enforcing a strict hierarchy of context. The orchestrator holds the main goal, plan, and current state, while disposable worker contexts handle specific, bounded operations. The coder receives implementation context, the tester focuses on behavior, and the reviewer checks the diff without seeing the long, messy conversation that led there. This separation prevents agents from simply rubber-stamping their own mistakes.
To challenge assumptions, the host prefers using different model families for roles that need to cross-examine each other. Running a coder and a reviewer on the exact same model creates a clone rather than a colleague, leading to shared blind spots. While research shows that agent teams are slower and cost more inference budget than a single agent, the host argues that the extra expense is worth it for complex work because independent verification uncovers deeper logic errors and vulnerabilities.
As agent organizations scale, they cannot rely on human tacit knowledge or memory; they need concrete artifacts like Git branches, test results, and pull requests to maintain organizational memory. A top-level dispatcher agent manages the portfolio by checking the backlog, claiming issues, and deciding whether a task needs a single agent or an entire team based on divisibility. The host warns against overloading agents with bad rules or endless exceptions. Instead of listing every permutation, sharing the underlying reason for a practice allows agents to figure out when a rule actually applies.
🎯 Tech Expert Opinion
The speaker’s shift away from prompt engineering toward structural agent management mirrors the maturation of software engineering workflows. Treating AI agents less like magical oracles and more like junior developers with strict boundaries is the correct mental model for production-grade systems. By isolating contexts—preventing the reviewer from inheriting the coder's rationalizations—the host addresses one of the most insidious failure modes of LLMs: confirmation bias within a single, lengthy attention window.
The pragmatic trade-off highlighted in the video—sacrificing speed and token budget for rigorous, multi-model verification—makes sense for teams building complex, mission-critical software. While casual users looking for quick script generation will find agent swarms over-engineered and expensive, engineers tackling large repositories benefit immensely from this separation of concerns. The emphasis on artifact-driven memory (relying on Git branches, pull requests, and CI checks rather than session states) ensures that the system remains resilient even if processes crash.
However, adopting this architecture introduces new overhead that teams must account for. Managing orchestrators, dispatchers, and multi-model review loops requires robust infrastructure and careful definition of project skills. Organizations must continuously audit their rules and best practices to prevent agents from faithfully executing outdated or contradictory guidelines at scale. Ultimately, this approach suits advanced engineering teams with high-complexity codebases, while smaller projects or rapid prototyping will likely find single-agent setups more than sufficient.
Kanal: DevOps & AI Toolkit