What 90 Docker Runs Taught Me About Simplicity

The Setup

At EkLine, we built a complex multi-agent AI system for documentation review. Multiple specialized personas. Orchestration logic. Routing between agents. It was architecturally interesting. It felt sophisticated. We were proud of it. I need to be honest about that pride. As an INTP, the multi-agent architecture appealed to my instinct for elegant systems. Many agents with specialized roles; it mirrored how I think about cognition itself. It felt right. Feeling right and being right are different things. We tested it. 90 Docker-isolated evaluation runs across real open source repositories. The complex system versus a simple alternative: a single-agent loop with basic scaffolding. Same model, same prompts, dramatically less architecture. The simple loop won. Not by a little. Consistently. Across different repository types, different documentation styles, different complexity levels. Fewer hallucinations. Less latency. More accurate reviews. This was humbling.

Why Complex Lost

The multi-agent system had a problem we didn't anticipate: compounding coordination costs. Each agent handoff introduced a translation layer. Agent A's output became Agent B's input, and every translation lost fidelity. The orchestrator spent tokens managing state that the simple system never needed to track. Error propagation was multiplicative; one agent's misunderstanding didn't just affect its own output, it cascaded through the pipeline. More moving parts didn't mean more intelligence. It meant more surface area for failure.

The simple loop, by contrast, maintained a single context throughout the evaluation. No handoffs. No translation layers. No coordination overhead. It could focus its full capacity on the actual task: reviewing documentation against the codebase.

The Insight About Smartness

Here's what I learned: smartness in AI systems is compounded, not front-loaded. Front-loading means designing the most sophisticated architecture you can imagine, then hoping the complexity serves the task. It's building a Formula 1 car to go to the grocery store. The engineering is impressive. The outcome isn't better. Compounding means starting with the simplest thing that could work, measuring it rigorously, then adding complexity only where the measurements demand it. Each addition earns its place. Each new component exists because we proved the system needed it, not because we imagined it might. After the simple loop won, we didn't throw away the multi-agent idea. We grew the system incrementally. We added a second agent only when we found a specific task where the single loop consistently underperformed. We added orchestration only when we had concrete evidence that coordinated agents outperformed sequential processing on that task. The final system was more sophisticated than the simple loop. But every piece of that sophistication was earned: validated against the benchmark, justified by data.

The Parallel to Software Engineering

This isn't specific to AI. It's a general principle that software engineering rediscovers in every era. Microservices beat monoliths. Except when they don't, which is most of the time for most teams. Complex architectures beat simple ones. Except when the coordination cost exceeds the capability gain, which happens more often than architects admit. The instinct to add complexity is strong because complexity signals competence. A simple system looks like you didn't try hard enough. A complex system looks like you understood the problem deeply. But understanding a problem deeply often means recognizing that it's simpler than it appears. The deepest expertise isn't knowing how to build complex systems. It's knowing when not to.

The Rule We Adopted

After those 90 runs, we adopted a rule: no new component without evidence. Every proposed addition to the system required a hypothesis and a benchmark. "I think a dedicated citation agent would improve accuracy" became "The simple loop misses source citations in 23% of cases. Let's test whether a dedicated citation step reduces that number." Most proposed additions failed the test. The ones that passed made the system meaningfully better. The system grew slowly and deliberately, and every piece of it was load-bearing.

What Simplicity Actually Means

Simplicity isn't the absence of sophistication. It's the presence of justification. A simple system that works is not unsophisticated. It's disciplined. It means every component earned its place. It means the builder understood the problem well enough to know what wasn't needed. Ninety Docker runs taught me that the hardest engineering decision isn't what to add. It's what to leave out. And the confidence to leave things out only comes from measuring (rigorously, repeatedly, honestly) what actually makes a difference. Start simple. Add complexity that earns its place. Measure everything. Trust the data over your intuition about what should work.

That's it. Ninety runs, one lesson.