Note 004

When automation breaks silently

The usual failure isn’t a crash. Something changes upstream, the flow keeps running, and nobody notices for weeks.

A man at a timber table tracing a finger down a long paper printout that spills over the table edge, in front of a lilac backdrop.

The most common thing we hear in a first conversation about automation is that someone tried it before and it broke. Asked how, the answer is nearly always the same. Nobody noticed for a while.

We tried automation before and it broke. What should be different this time?

The usual failure isn’t a crash. Something changes upstream: a supplier renames a field, a form adds a step, an export moves a column. The flow keeps running and keeps being wrong, and the first anyone knows is when the numbers don’t add up weeks later. What should be different is simple to say: a workflow that stops and asks a person when it isn’t sure, and a record of every run so the failure is loud and early.

Why does it fail quietly rather than loudly?

Most connector tools assume the shape of the data they were set up with. When that shape changes, they don’t know it has; they map the columns they can find and carry on. Nothing checks whether the result made sense, nothing watches the run, and the only alarm is a person downstream noticing that a total looks wrong. By then the wrong data has been posted for weeks, and unpicking it costs more than the automation ever saved.

What’s the difference between an error and an exception?

An error is when the run stops: a login expires, a service is down. Those are annoying but honest; you know about them. An exception is an item the workflow can’t resolve with confidence: an invoice that doesn’t match any purchase order, a name that matches two records, a figure outside the usual range. A workflow built well treats an exception as a decision for a person, not a guess to make. It routes the item, with what it couldn’t resolve, to someone who can, and carries on with the rest.

Where should the checkpoint be?

At the point where a decision changes what happens next, and before anything with a consequence happens. Not at the end, where a review screen becomes a formality, and not everywhere, where approvals become noise that people click through. For a supplier invoice, the checkpoint sits after the match and before the posting. For a reply to a client, it sits before the send. Field note 002 covers where they go.

What does “every run leaves a record” mean in practice?

Each time the workflow runs, it writes down what came in, what it drafted, what it matched to, who approved it, what it flagged and when. The record lives where your team can read it without us, usually inside the systems you already use. It’s how a finance manager can answer “why was this posted?” in a minute rather than an afternoon, and it’s how a change upstream shows up as a run of exceptions on the day it happens, rather than a wrong number at month-end.

What happens when the process changes?

It will. A supplier changes their invoice layout, a new entity is added, a step moves between people. A workflow built to stop and ask will surface the change as exceptions rather than absorb it silently, which is the signal to adjust it. We treat that as maintenance, not a new project: the workflow changes with the process, and the monthly review of the record is where those changes get agreed.

How do we rebuild trust after the last time?

Slowly and visibly. The new workflow runs alongside the old way until it has earned trust, with the team comparing the two. The old way stops when the people who do the work say it can, not when a project plan says so. And nothing we build runs entirely on its own, as a matter of method rather than caution; the checkpoints are in the checkpoint map, and anyone on the team can point at one and ask why it’s there.

Our own inbox runs this way. An agent reads, sorts and files what comes in and drafts the replies; when it isn’t confident about an enquiry it does nothing at all and leaves it for a person, and no reply goes out that a person hasn’t sent. It’s slower than full automation by a few minutes a day. It’s also why we still trust it.

If you’ve got a flow somewhere that broke and was quietly replaced by a spreadsheet, that’s the one worth looking at first.

All field notes

Start with the work that keeps getting in the way.