Why Most Agent Demos Stop Before the Hard Part
The gap between doing a task once and operating the same workflow repeatedly.
Almost every agent demo shows the same thing: a task completed once, cleanly, on a happy path the author already walked. It’s genuinely impressive and almost entirely uninformative, because the hard part of an agent system isn’t completing a task — it’s operating a workflow, which means doing the task again tomorrow, on inputs nobody previewed, and being able to say afterward what it did and whether to trust it.
Once is a demo; repeatedly is a system
The difference between the two is where all the engineering lives:
- Idempotence. Running the workflow twice must not double-write, double-charge, or double-count. A demo never runs twice.
- Provenance. When the output is wrong — and eventually it is — you need to trace each number back to the fact it came from. A demo has no yesterday to compare against.
- Failure that stops. A demo skips the case where a source is down, a schema drifted, or a tool returned garbage. A system has to notice and halt loudly, not improvise.
- Auditability. Someone other than the author must be able to see what the agent did and why. A demo is narrated by the person who built it.
The tell
I’ve started reading agent demos by asking one question: what happens on the second run? If the answer is “we haven’t run it twice,” the demo has stopped exactly where the interesting problems begin. Most do.
This isn’t a criticism of demos — they’re the right artifact for showing a capability exists. It’s a caution against mistaking the demo for the product. The demo is the first 20% and it looks like the whole thing. The workflow is the other 80% and it’s invisible until you try to live with it.