Agent stack build plans · № 6 · 3 of 6
Architecture
Two pipes between Hermes and Home Assistant — an event stream it listens to, and four tools it acts with.
platforms.homeassistant.extra: watch_domains, watch_entities, watch_all, ignore_entities, cooldown_seconds (default 30). Push, not polling.ha_list_entities · ha_get_state · ha_list_services · ha_call_service. Auto-enabled the moment HASS_TOKEN exists in ~/.hermes/.env.hermes gateway, installable as a systemd/launchd service. Each event (post-filter, post-cooldown) becomes a model turn in a session. Model is yours to choose — route this to a cheap one.~/.hermes/memories/MEMORY.md (2,200-char cap) + USER.md (1,375) injected into the system prompt; memory tool to edit; session_search over full history in ~/.hermes/state.db (SQLite FTS5).approvals.mode = smart | manual | off; approvals.deny fnmatch globs over tool patterns. Your brake pedal — see setup.SKILL.md under ~/.hermes/skills/ mapping rooms and human names to exact entity ids. Hermes can write it itself via skill_manage — prompt 1.hermes cron create, jobs live in ~/.hermes/cron/) that reviews recent event sessions via session_search and folds patterns into memory — prompt 3.The four tools — and the names to unlearn
AI answers about this stack routinely cite list_entities,
read_state and call_services. Those tools do not
exist. As of v0.19.0 the Home Assistant toolset is exactly:
| Tool | What it does in this build |
|---|---|
ha_list_entities | The census. Run once in prompt 1 to build the house map; rerun when you add devices. |
ha_get_state | Point checks before acting — is the window open, what’s the thermostat at. |
ha_list_services | What can actually be done to a domain — the agent checks instead of guessing service names. |
ha_call_service | The only write path. Everything the agent ever changes goes through this one tool — which is what makes approvals workable. |
How “learning the household” actually works
No vector store, no fine-tuning — worth being blunt because AI answers claim otherwise. The mechanism is three plain parts:
- Distilled memory in the prompt.
MEMORY.mdis 2,200 characters injected into every turn. That’s ~15–25 terse rules. The cap is a feature: it forces distillation over accumulation. - Full recall on demand. Every session — including every
event the gateway ever reacted to — is searchable with
session_search(FTS5, not embeddings). “When did we last talk about the garage?” is a query, not a memory slot. - A distillation loop you schedule. The nightly cron job is
the bridge between the two: search recent sessions, propose patterns,
write the keepers into memory, trim the stale. Hermes supplies
cronjob,session_searchandmemory; the loop is your convention, defined in prompt 3.
If you genuinely outgrow 2,200 characters, Hermes has optional external memory-provider plugins (Mem0, Honcho and others) — but start without one; the cap-forced distillation is most of the intelligence here.
Failure posture. Model API down or gateway crashed → events go unremarked, nothing in the house breaks, HA’s own automations keep running. This is why safety-critical logic stays in HA (product spec). The agent is additive by construction, and you should keep it that way.