Autonomy needs an audit log

The strongest objection to everything we are building came from an owner in Wahroonga, about four minutes into a survey, and it was this:

“I don’t mind it being wrong. I mind not being able to ask it why.”

That is the entire problem, stated better than we had managed in a year of writing about it. It is also the reason most “AI for the home” demonstrations are unsellable into a serious house. They show a system doing something clever. They never show what happens on the evening it does something stupid, at 2am, to a room with a sleeping child in it.

A house is not a phone. The cost of a bad inference is not a wasted tap — it is a cold nursery, a gate that opened, a freezer that was load-shed for six hours. The bar for acting without being asked is therefore much higher than the bar for suggesting, and it is met by architecture rather than by accuracy.

Three properties. None of them optional.

One: every action carries its reasoning

The unit of record is not “light turned on at 18:42”. It is the full decision:

• the policy that proposed the action
• the proposals it beat, and on what grounds
• the evidence each proposal was standing on — which sensors, which readings, which inferred states, at which timestamps
• the precedence tier that resolved it
• the outcome of execution, including failure

Written append-only, at the moment of the decision, before the telegram goes out. Not reconstructed afterwards from logs, because a reconstruction is a story and what is needed is a record.

This sounds like a lot of writing. It is roughly 1,200 decisions a day in a house of the size we work with, which is nothing — the volume of a modest application log. The expensive part is not storage, it is discipline: the engine must be structured so that it is impossible to act without producing the record, rather than conventional to produce one. In our case the executor takes a resolved decision object as its only input. There is no code path that reaches the bus without one.

Two: precedence is structural, not advisory

Policies disagree constantly. That is not a defect to be tuned away — it is the system working. The vacancy aggregator genuinely believes the west wing is empty. The comfort policy genuinely believes that zone should hold. Both are reasoning correctly from what they can see.

What matters is how the disagreement resolves. In most automation, the answer is “last writer wins”, which means behaviour depends on scheduler timing. That is unacceptable in a building, because it is unpredictable in exactly the conditions where predictability matters.

So resolution runs on explicit tiers, and the tiers are enforced by the arbiter rather than observed by convention:

1. Safety and statutory. Smoke, water, gas, egress, electrical protection.
2. Security. Perimeter, access, alarm state.
3. Health and habitability. Temperature bounds, humidity, air quality, refrigeration.
4. Explicit human instruction. What somebody in the house actually asked for.
5. Comfort. Light levels, pre-heating, the pleasant things.
6. Efficiency. Tariff shifting, solar alignment, load management.

A tier-6 proposal cannot beat a tier-3 proposal. Not “should not” — cannot, because the arbiter compares tiers before it compares anything else. This is the property that makes the system safe to leave alone: no amount of clever optimisation can reach past a habitability constraint, because the optimiser does not have a mechanism for doing so.

It is also the property that makes failure legible. When the house does something surprising, the log names the tier that decided it, and the surprise resolves into one of three things: a policy reasoning from bad evidence, a policy placed at the wrong tier, or — most often — a policy reasoning perfectly well from evidence you did not know it had.

Three: the human always wins, and the loss is the lesson

Every system in this category claims the owner is in control. Most implement it as a manual override mode — a switch that suspends automation, which you then have to remember to switch back.

That is control the way a fire axe is control. It works, and using it means the system has already failed.

The behaviour we want is narrower and much more useful. When somebody touches a wall switch, a panel, or says something out loud, three things happen:

The action executes immediately, ahead of any pending proposal, because a person in a room outranks a policy about that room.

The contradicted policy yields for a scoped, bounded period — this room, this capability, the rest of the evening — rather than globally and rather than for ever. The house does not sulk, and it does not immediately re-argue.

The override is recorded as evidence, weighted far above ambient signal. A person walking to a switch to undo something is the clearest statement of preference the system will ever receive, and it is worth more than a month of passive observation.

This is why we publish the overrule rate on the evidence page. It is the honest measure of how often the house was wrong about what somebody wanted. Any vendor in this category could compute it. That approximately none of them do should tell you something about what it usually looks like.

The failure we had to design out

One example, because abstraction is cheap and specifics are not.

An early version of our own safety policy did what fire-safety intuition suggests: on smoke detection, illuminate the full egress path and release the door locks.

It is defensible on paper. Every building code in the world wants egress unimpeded. Fire services want the doors to open.

In a sold product it is a burglary vector triggered by burnt toast.

A single smoke sensor, a grill left unattended, and the house unlocks its own doors at a moment when everybody inside is distracted and looking the other way. The policy is not wrong about fire. It is wrong about evidence — it treats one noisy sensor as sufficient grounds for an irreversible physical action with a severe failure mode.

What replaced it: multi-sensor confirmation before any lock actuation, heat rise corroborating smoke, the action separated into stages so that lighting the egress path — which is harmless and useful — happens immediately on low confidence, while unlocking requires either corroboration or a human.

We describe this publicly for two reasons. It is the clearest example we have of why autonomy is a governance problem rather than a modelling problem — the model was fine, the authority was wrong. And any vendor asking to act on your house without asking should be willing to show you the decisions they got wrong on the way, because the alternative is asking you to believe there were none.

What this buys

Nothing here makes the house cleverer. Precedence tiers, append-only logs and scoped yields are, if anything, constraints on cleverness.

What they buy is the only thing that matters for a product that lives in a building for fifteen years: when it is wrong, you can find out why, and you can be confident about the size of the mistake before you go to sleep.

That is a lower promise than the category usually makes. We think it is the only one worth making.