Agent stack build plans · № 6 · 2 of 6

Product spec

A judgment layer on top of Home Assistant — for the rules you could never quite write as YAML.

Who it’s for

The HA tinkerer at the automation ceiling. You have forty automations and the next ten you want are all conditional and vague: “warm the house before whoever’s home first, most days.” Deterministic triggers can’t say “most days”. An agent with memory can.
The household of two-plus where the automations keep annoying somebody. The fix isn’t more YAML, it’s a place to state house rules in plain language — “never turn off the reading lamp while someone’s in the chair” — and an agent that respects them.
The person who wants nudges, not dashboards. The garage is open, the heating’s been fighting an open window for an hour — you want a Telegram message at the right moment, not a panel you forget to check.

A day in the life, three weeks in

07:10 — you’re up before your alarm; the bedroom motion sensor fires. Hermes says nothing and does nothing: memory says mornings are sacred, and the kettle isn’t smart anyway.

08:02 — last person leaves. Hermes notices person.* all away, checks ha_get_state on the climate and the upstairs window sensor, drops the heating, and — because the window is open — adds one line to tonight’s review: windows get left open on school-run mornings.

17:40 — sunset. Lamps on, no message. This stopped being worth announcing in week one; the rule that actions matching an established pattern happen silently is one you wrote into memory.

20:47 — the garage has been open for ninety minutes. This crosses the nudge threshold the pattern distiller proposed last week and you approved. One Telegram message, one “yes”, one ha_call_service with cover.close_cover.

23:30 — the nightly cron reviews the day’s events, proposes one new pattern, trims one stale one. Memory stays under its 2,200-character cap because it has to.

What it deliberately won’t do

Replace your existing HA automations. Keep deterministic, safety-relevant logic (smoke alarm → lights on, leak sensor → water off) in Home Assistant, where it runs even when a model API is down. The agent layers on top; it is not the foundation.
Voice. This plan is text-in, text-out over your messaging platform. HA’s own voice stack is a different project.
Lock and alarm autonomy. By policy, anything that secures the house only ever moves after an explicit yes — setup wires this with approvals.mode and by keeping those entities out of the watch lists.
Per-person memory. Hermes keeps one household memory per profile. Per-member memory inside a group chat isn’t a documented capability, so this plan doesn’t pretend otherwise — separate family profiles are the capstone plan, the family OS.
Run offline. Every decision is a model call. Hermes is model-agnostic, so you choose the provider and the privacy trade — but “my motion sensors never leave the LAN” is only true if you point it at a local model, and event-stream reasoning on small local models is a rougher ride than this plan budgets for.

The scope cut worth restating: this build’s product is maybe fifteen good rules and two good nudges a day. If you measure it by actions per hour it will look lazy. That’s the design.