What happens when an AI worker gets something wrong?
It will, sometimes, as a person does. What matters is that the mistake is caught before it counts, can be traced, is put right, and who answers for it.
·
5 minute read
An AI worker will get something wrong sometimes, as a person does. Four things decide whether that matters: whether the mistake is caught before it counts, whether it can be traced, how it is put right, and who answers for it. The first is the job of approval, the second of the log, the third of whoever manages the worker. The fourth does not move: your firm answers to its own clients and regulators, and your contract says what the provider owes you.
What kind of mistakes are these?
They are ordinary ones. A date misread from a scanned document. A record matched to the wrong person of the same name. A reminder sent to someone who paid yesterday, because the ledger had not caught up. A summary that states something the source did not say. A careful administrator makes each of these now and then. The difference with an AI worker is that it repeats a mistake faithfully until the cause is fixed, and that it leaves a complete record of having made it.
What stops a mistake before it counts
Approval. By default a message or a high-impact action waits for a person's explicit approval: the worker proposes, a named person confirms or cancels, and nothing runs until they do. A mistake in a draft is a correction, not an incident. If your firm has chosen to let routine messages go in wording it approved in advance, nobody checks those one by one: a fault there is caught by the log and by whoever receives it.
A narrow job. The worker does what its specification lists and has access to nothing else. It never moves money, so no error of its own can send any.
Stopping instead of guessing. The specification names the cases the worker hands to a person: a document it cannot read, two records that might be the same, a reply that disputes a fact, anything outside its rules.
Saying it is AI. Every message says so, and anyone who wants a person gets one. A recipient who sees something odd knows to query it.
Regulated decisions are never the worker's; our article on AI in AML and KYC work sets out that line.
How a mistake is traced and put right
Every action is logged: what the worker did, when, on which record, and what it was working from. So the first question after any error, "what else did this touch?", has an answer you can read. From there the steps are those you would take with a member of staff.
Stop it. The affected step is paused, or the worker's access is switched off. You can do the second yourself.
Find the extent. The log shows every record and message the same fault reached.
Correct the records, and tell whoever needs telling. A person decides what is said to a client or a customer.
Fix the cause. The rule, the wording or the specification is changed so the same input cannot produce the same error, and the change is recorded.
Review. The error and its fix go into the next review of the worker's performance.
Step four is what "managed" means: the provider owns the repair. See what AI staff are.
When the mistake involves personal data
Here the law sets the clock. These are duties under UK GDPR, which applies across the UK, as the Information Commission's Office (ICO) states them. Whether a given incident meets the tests is your firm's decision, with your own advisers.
A wrong recipient is a breach. The ICO lists sending personal data to an incorrect recipient among its examples of a personal data breach.
72 hours. A notifiable breach is reported to the ICO without undue delay and, where feasible, not later than 72 hours after you become aware of it. A breach need not be reported where a risk to people is unlikely, but that decision is documented.
Telling the people affected. Where the risk to individuals is high, they are told directly and without undue delay.
Your processor tells you. A processor that has a breach must inform the controller without undue delay once it becomes aware.
Every breach is recorded, reported or not.
Inaccurate data is corrected. A person has the right to have inaccurate personal data put right, a request is answered within one month, which can be extended in some cases, and anyone the data was passed to is told of the correction unless that is impossible or involves disproportionate effort.
Who is responsible?
To your clients, your customers and your regulator, your firm is. The ICO says controllers are primarily responsible for overall compliance with UK GDPR. It also says that a processor which fails in its obligations, or acts outside the controller's instructions, may itself be liable. Beyond data protection, who bears the cost of a mistake is a matter for your contract with the provider and for your own insurance.
What stays with a named person
Deciding whether an incident is a breach to report, and reporting it.
What a client or customer is told, and any apology.
Handling a complaint.
Approving the changed rule before the worker starts again.
Questions to ask any provider
Which actions wait for approval, and which run without it?
Can we pause the worker ourselves?
How soon are we told of an incident, and where does the contract say so?
Can we see the log for one case, start to finish?
Who fixes the cause, and is that included?
By default the worker would propose and a named person of yours would confirm or cancel anything that matters, with every action logged so a fault can be found, paused and put right, and what a client is told would stay with your people. Bring the mistake you are most worried about. Talk to our agent.
Carlos Oliveira
Founder of The AR Dept
Losing billable hours to this process every week?
Check if your team’s routine admin can be handed to an AI worker.
