What kind of AI automation request actually gets turned down?
Not the ones people usually worry about (content drafting, internal data summarisation, first-pass classification): those are genuinely well-suited to automation because a human reviews the output before it has any external effect. The pattern that gets pushed back on is irreversible external action taken without a review step: an automation that sends an email to a customer list, changes live pricing, or publishes content, entirely on its own judgment, with nothing between the model’s output and the real-world consequence.
The distinction isn’t about how good the model is at the specific task. A model can draft an excellent customer email and still be the wrong thing to let send that email unsupervised. The cost of a rare bad output landing in ten thousand inboxes is high, and the cost of adding a five-minute human check is low. The asymmetry is the whole argument.
Isn’t this just being overly cautious about a mature technology?
The caution isn’t really about the model’s average performance. It’s about tail risk in a specific, narrow category of task. A model that’s correct 99% of the time is a strong result for most work. It’s a bad design for the 1% of cases where the output is wrong, if there’s no checkpoint between generation and an irreversible real-world effect. The rare failure is exactly what a review step exists to catch, and its rarity is precisely why teams stop bothering with the check, right up until the failure happens.
This is a design principle about where automation sits in a workflow, not a verdict on AI capability generally. The same model, generating the same draft, is fine when a human reviews it before it goes out and genuinely risky when it doesn’t. Nothing about the model changed between those two setups; only the checkpoint did.
What do we actually build instead of just saying no?
Almost always the same automation, with a review step inserted at the point of highest consequence rather than removed. A campaign-send automation that drafts and schedules but requires a one-click human approval before the send fires. A pricing-suggestion tool that recommends a change and logs the reasoning, rather than one that updates the live price directly. The automation still does the heavy lifting; a person still owns the moment the action becomes irreversible.
That’s the actual meaning behind “AI-augmented, human-governed” as a design principle rather than a slogan: the AI does more of the work, a specific, deliberately-placed checkpoint still exists at the point where a mistake would be expensive to undo. The rule holds whether the system is a chatbot, a fixed workflow or an agent choosing its own steps; an agent build simply needs the checkpoint drawn more carefully.
What does this actually look like for a specific task, like automated customer review responses?
Take a common request: automatically generate and post responses to customer reviews. The drafting part is a strong fit for automation. Reading a review and producing a relevant, well-worded response is exactly the kind of task a model handles well, at volume, faster than a person reviewing each one individually. The risk sits entirely in the second half: posting that response publicly, under the brand’s name, with no review.
The failure mode isn’t hypothetical caution. It’s a specific, foreseeable one. A model can misread sarcasm as sincerity, respond to a serious complaint with a tone that reads as dismissive, or occasionally generate something that’s subtly wrong about the product. Any one of those, posted automatically and publicly, is now a screenshot-able mistake attached to the brand permanently, while the same error in an unposted draft costs nothing at all once a human catches it before it ships.
The governed version of this automation is straightforward: the model drafts every response. A queue of drafts then waits for a quick human approval (typically seconds per review, not the minutes a fully manual response would take) before anything posts. The automation still removes the vast majority of the manual effort. The checkpoint sits at the one point where removing it entirely would trade a small time saving for a real, public, hard-to-reverse risk.
Does insisting on a checkpoint slow down how much automation a client actually adopts?
Occasionally, for the specific slice of a request that touches irreversible external action, and that’s the correct place for it to slow down, rather than everywhere. Most of what a client wants automated (drafting, summarisation, internal triage, first-pass classification) isn’t affected by this principle at all, because it was never going to run unsupervised on an external-facing outcome in the first place.
Where it does add friction, the trade is usually easy to make once it’s explained plainly: a five-minute-per-day approval step is a small, bounded cost against the specific, if infrequent, cost of an unreviewed error reaching customers publicly. Clients who initially want the checkpoint removed almost always keep it once asked directly whether they’d accept the downside case happening even once, unmoderated, under their brand.
