Your software may soon need liability cover of its own

The first rate card for autonomous agent liability would not read like a warning about the future. It would read like a rate card. Four classes, a base premium, a rate per thousand actions, and one column doing all the work: actions taken without human confirmation.

A bakery owner in a town of eleven thousand would open hers on the counter between the morning bake and the school run. Her ordering agent — flour, butter, yeast, adjusted for weather, school holidays and the local fixture list — would fall into class 3B: unsupervised procurement, low value, high frequency. Roughly forty binding commitments a week, none of which she would see before they were made.

The structure would be familiar to anyone who has ever insured a van. Premium scales with exposure, exposure is measured in decisions, and decisions are risky in proportion to how hard they are to reverse. The sentence underwriters would consider the most important in the document, and customers the least readable, is the one stating that the excess applies per action, not per claim.

“We are not pricing intelligence,” an underwriter would explain. “We are pricing unsupervised commitment. A system that asks first costs less, because asking is a control, and controls are what we give discounts on. Everything beyond that is theology, and theology has no loss history.”

The figure at the bottom of the bakery's page would be higher than the employer's liability cover for her part-time counter assistant. This would not be an error. The assistant cannot commit the bakery to eleven pallets of flour by misreading a decimal point, and in the unlikely event that she did, she would mention it at some point during the afternoon.

The rational response would be obvious, and would also be the joke. Switch on confirmation above a threshold and the premium would fall by a third. She would then read and approve forty orders a week herself, a minute each, for the rest of the agent's working life. She would be paying in attention what she had been paying in premium. The rate card would have done something nobody had done before: put a price on the part she had been supplying for free.

None of this would be mandatory. But once a premium exists it becomes a number, and numbers travel. It would appear in business cases, then in procurement templates, then among the standard questions a bank asks a small firm about its suppliers. Within two renewals the question would no longer be how capable the agent is. It would be what it costs to be wrong on your behalf, per action, per year.