Early-stage technologies tend to be inefficient. The first internal combustion engines were grossly wasteful compared to modern ones. Their makers were focused on making the damned things run, not on making them run well.
That’s where we’re at with AI. Companies are throwing stuff at LLMs to see what works. Right now, traction and speed matter more than efficiency. But that won’t always be the case. Eventually, we’ll shift to AI-powered systems that are both more efficient and controllable. What will they look like?
My bet: a combination of more carefully structured inputs (i.e., context engineering) and good, old-fashioned deterministic programming.
I like Simon Willison’s definition of agents: “An LLM agent runs tools in a loop to achieve a goal.” The most common way to do this is to give an agentic system (e.g., Claude Code) the ability to call deterministic tools — Unix CLI utilities, APIs, etc. It’s a powerful and flexible approach, but one that uses a lot of tokens and requires advanced models. It’s also unpredictable in ways that matter when you’re running a business.
But for many use cases, you can achieve similar results by inverting the hierarchy. Instead of giving an LLM the ability to use deterministic programs, you have a deterministic program call an LLM at specific points in its flow, using heavily curated context. It’s an information architecture challenge as much as a software challenge.
This isn’t as flashy as an autonomous agent improvising its way through a problem. But for many business tasks, it doesn’t need to be. The constrained approach produces more predictable outcomes, runs on smaller and cheaper models, and keeps your data under control.
The tradeoff: you must know what you’re building before you build it, since the bulk of the work happens in traditional software. But — and here’s the kicker — coding agents can help design and write that software. A bit of probabilistic work upfront to spin up a mostly-deterministic system.
I’m using such “agents” on my Mac. For example, one monitors a directory for new PDFs and fires a script that calls an LLM to transcribe my handwritten notes. About 80% of the work is classic if-then logic. The AI handles only the part that actually needs AI. Because those asks are tightly constrained, I use small models running locally — free to download, private by default, and using hardware and energy I’ve already accounted for.
This is what more mature AI adoption will look like for most organizations. Instead of open-ended agents improvising at scale, tightly-scoped systems that call on AI for the things AI does best. Heavy thinking upfront, with day-to-day operations using limited AI. Upshot: increased control, efficiency, and predictability.
This post first appeared in my newsletter.