Most of the operational work at a small company does not need an AI agent — a well-scoped internal tool or a website is usually the right answer, and cheaper to maintain (see the internal-tools case study). Sometimes it does need one: work that has to run on a schedule, make judgement calls against live data, and stay safe doing it without someone watching every step. This page is about that harder case.
Aclymate is five people. That is too few to hire our way out of every new piece of operational work, and enough work that the alternative is everyone slowly drowning in follow-ups. So we built software that watches the parts of the business that usually only get checked when someone remembers to — security settings, subscription spend, financial records — and flags what actually needs a decision.
It has been running in real production, on a real schedule, for about two months now. The underlying AI model is a commodity — anyone can call one. What actually took the work, and what this page is mostly about, is everything built around it on the assumption that it will, occasionally, get something confidently wrong.
What it actually does
Something watches for a trigger — a schedule, a new record, a button someone presses. It gathers whatever context it is allowed to see, writes up what it thinks should happen, and sends that write-up to a person for a plain yes or no. Nothing happens until someone approves it. An idea nobody signed off on is just a draft sitting in a queue — it cannot send an email, move money, or touch anything on its own.
That one rule is the whole safety model. It turns an unanswerable question — “is the AI good enough to trust?” — into one a person can actually answer in four seconds: “is this specific thing good enough to send?” It is also why every mistake below was embarrassing rather than expensive.
None of the rest of this went that cleanly the first time. Here is what actually broke, in a system this size, over a couple of months — because the failures are the only honest evidence that this has run in production and not just in a demo.
▸For the technically curious — how the guardrails actually work
Three layers, with a hard boundary in the middle. The customer-facing product is one system. The internal tooling — agents, automation, the internal admin app — is another. Between them sits a locked door: a permissioned service that internal tools have to ask for customer data, which decides what they are allowed to see.
Internal tooling never reaches into the customer product directly. Every agent follows the same pipeline: something triggers it, it gathers the context it is allowed to see, a model drafts an output, the draft is saved and pushed to its owner for approval, and only on approval does the action actually happen. An unapproved draft is inert.
Six specific guardrails came directly out of incidents like the ones above — none were in the original design.
| Guardrail | What it does | What it prevents |
|---|---|---|
| Approval gates | Nothing outbound sends without a human approving that specific draft | Prevents the failure everyone fears — a wrong message reaching a customer |
| Per-agent allowlists | Each agent can only act on the entities it is scoped to | Contains blast radius when an agent misidentifies its target |
| Rejection-aware scheduling | A rejected proposal is remembered, not re-proposed | Stops the loop where an agent argues with its owner indefinitely |
| Pause that actually pauses | A paused agent does not advance its own schedule | The difference between a stopped agent and a very patient one |
| Evaluation harness | Scored test runs gate changes to agent behaviour | Catches quality regressions that no unit test would notice |
| Isolated evaluation data | Test runs never write to production records | Keeps the measurement system from corrupting what it measures |
None of this runs fully on its own yet, and that is on purpose, not a limitation waiting to be lifted. Every suggestion goes to a person before anything happens — for the higher-stakes work especially, that stays true indefinitely, not just until the technology improves. Reviewing a draft takes someone four seconds. Trusting software to make a security or financial call with nobody looking is a completely different bet, and not one worth making before it has actually earned it.
~$20
Cost of the one real runaway loop, over roughly three hours
~2
Months in continuous production
~90
Scheduled checks per week — most report nothing on a quiet day
What this means if you’re considering something like it
The real lesson here is not technical — it is that the safety mechanism has to exist before you need it, not after. Every mistake above was recoverable specifically because a person still had to sign off before anything actually happened. Adding that in after software is already taking real actions is much harder than building it in from day one.
It is also not the cheap part to skip. Most of the real effort here went into exactly this kind of guardrail, not into making the AI itself smarter. If someone offers to set up “AI agents” for your business without talking about any of this, that is usually the part they are skipping.
And staying on “a person approves everything” indefinitely, for the parts of your business that actually matter, is a perfectly good place to stop — not a smaller version of some fully-autonomous future you are supposed to be racing toward.