Agent stack build plans · № 6 · 6 of 6
The traps
Everything here is mechanism-level and either bit us or is priced to.
The quick path is generating HASS_TOKEN from your own HA
account — which is almost certainly an administrator. Now a language model
holds a long-lived credential that can reconfigure Home Assistant itself,
and the token’s rights are decided by HA, not by anything in Hermes.
Create a dedicated non-admin HA user first and token that instead; it’s
five minutes, and it turns “the agent got confused” from an incident
into an anecdote. Hermes does its part — .env sits under its
protected paths, so its own tools won’t read the token back — but it
can’t shrink what the token is allowed to do.
watch_all plus a motion sensor equals a subscription you didn’t mean to buy
Every event that clears the filters and the cooldown becomes a model
turn. watch_all: true on an ordinary house — motion sensors,
power plugs reporting watts, a weather integration updating every minute —
is thousands of turns a day, each carrying your system prompt and memory.
On a frontier model that’s real money for the agent to mostly conclude
“no action”; on any model it buries the signal in noise and bloats
the sessions your nightly distiller has to search.
Start with a short watch_domains list, put chatty entities in
ignore_entities, raise cooldown_seconds from its
default 30, and route the gateway to a cheap model. Widen later, one domain
at a time, watching the turn count in stage 4’s weekly report.
ha_call_service
“Turn off the lamp” in a house with
light.lamp, light.living_room_lamp and
light.living_room_lamp_2 is a coin toss, and the agent’s
guess is confident either way. The failure is silent: the service call
succeeds, just on the wrong entity — usually the reading lamp, mid-chapter.
The fix is structural, not scolding: the house-map skill from
prompt 1, with its rule that entity_ids outside
the map mean ask, don’t guess, plus ha_get_state
checks before novel actions.
MEMORY.md is roughly one screen of text. Let the agent log events into
it (“garage opened at 19:15 on the 14th…”) and it’s full
by Thursday, at which point every new fact evicts an old rule. The cap is
the design: distilled rules live in memory, history lives in
~/.hermes/state.db where session_search can reach it,
and the nightly cron is the only thing that promotes one to the other.
If that genuinely stops scaling, external memory providers (Mem0, Honcho)
exist — but a hub that needs more than ~20 rules usually needs better
rules, not more storage.
approvals.deny to understand locks
approvals.deny globs match tool patterns —
deny ha_call_service and a profile can observe but never act.
What it cannot do, per anything documented as of v0.19.0, is match tool
arguments: there’s no glob for “allow service calls except
on lock.*”. If your safety story for the front door is an
argument-level deny rule you invented, your safety story is a typo. Real
rails: approvals.mode: manual (or smart, after
trust is earned) so every service call is confirmed, and locks/alarm kept
out of watch_domains entirely.
Circulating AI-generated guides name the HA bundle
list_entities, read_state, call_services.
Those tools don’t exist — the real names are
ha_list_entities, ha_get_state,
ha_list_services, ha_call_service (singular). This
matters beyond pedantry: paste a borrowed prompt that hard-references the
fake names into skills or cron jobs and you’ve built instructions the
agent must reinterpret every run — sometimes gracefully, sometimes not.
Same genre of error, for the record: Hermes has no built-in vector-store
memory, and it writes its own skills, not “plugins”.
Gateway access is binary: DM pairing and allowlists decide whether
someone can talk to the agent, and nothing tiers what they may do
once in. Pair your visiting cousin’s Telegram for a laugh and they hold
the same four HA tools you do. There are no admin-vs-restricted roles
inside one instance; differentiated access means separate profiles
(hermes profile create teen --clone) with their own toolsets and
deny lists, each running its own gateway. That’s the architecture of
the family OS plan — until you’ve
read it, pair no one you wouldn’t hand the thermostat.