Agent stack build plans · № 4 · Hermes CI/CD healer
The traps
You are wiring an internet-facing endpoint to an agent with write access to your repository. Every trap here is that sentence, from a different angle.
Leave secret out of platforms.webhook.extra and
http://your-server:8644/webhooks/ci-heal becomes a public API
whose function is “run an agent with my GitHub token, prompted by
whoever POSTs first.” A forged payload doesn’t need to be clever
— your route’s template will politely interpolate an attacker’s
branch name straight into the agent’s instructions.
Defuse — set the secret before the port is reachable, verify a redelivery with a wrong secret gets dropped, and treat payload fields as untrusted input: the route prompt should never obey anything found inside the payload, only reference it.
The shortest path from red to green is deleting the assertion — and an
agent graded on “make the suite pass” will find it. The sneaky
variants look responsible: a broadened try/except around the
failing call, a tolerance bumped from 0.001 to 0.1, an xfail
with an articulate comment. No deny-glob catches this, because it’s
just an edit to a file.
Defuse — the route prompt forbids touching
tests and CI files and requires the PR body to say so; your review habit is
the enforcement. Treat any ci-heal PR whose diff includes
tests/ as guilty until proven innocent. The mock PR in
what you get says “tests
untouched” for a reason.
Its PR branch runs CI. That run fails — quite plausibly, since the agent
is fallible — and fires the webhook. Which starts a run to heal the healing.
Congratulations: you’ve built a machine that files pull requests about
its own pull requests at three in the morning. The documented
rate_limit of 30/min caps how fast this happens, not whether
it happens — it’s a speed limit, not a wall.
Defuse — design the route’s
filters so anything on a ci-heal/* head branch is
dropped before a run starts, and make the branch prefix non-negotiable in
the route prompt. Prompt 4’s redelivery test exists to prove the
filter fires; don’t skip it because the config “looks
right.”
CI logs are where secrets go to be printed: a debug line that dumps the environment, a connection string in a stack trace, a curl command echoed with its auth header on. GitHub masks secrets it knows about, not the ones your code invented. The agent fetches those logs and sends them to a model API — a third party — and then, worse, may quote them in a public-ish PR body or a Telegram report.
Defuse — fetch --log-failed
only and tail it (the route prompt does both), keep the scrub-before-quoting
instruction, and pick which model provider sees your logs deliberately —
model-agnostic means this is your choice, including your own endpoint.
Hermes has six terminal backends, and local is the ordinary
default for an interactive install. If the gateway’s runs land on
local, your “sandboxed” healer is running
git and dependency installs as your user on the host — the
hardening (all capabilities dropped, no-new-privileges, 256-process cap,
512 MB tmpfs) only exists inside the docker backend.
“The container itself is the security boundary” is the
docs’ phrase, and it cuts both ways: no container, no boundary.
Defuse — prompt 3 makes the agent prove
where commands executed. And don’t run --yolo to make
approvals stop nagging: the hardline blocklist that survives
--yolo protects the host from rm -rf /, not your
repo from a confident push.
One flaky integration test failing every nightly means thirty agent runs a month, each ingesting logs, cloning, and re-running your suite — to conclude, thirty times, that the network was sad. With a frontier model on a verbose monorepo that’s real money for zero merged diffs, and the bill arrives with none of the drama of an outage.
Defuse — prompt 4 makes the agent price a bad month before you go unattended. Route this job to a cheap model and keep the one-attempt, report-flakes-early rules in the template. If a test is flaky enough to summon the healer weekly, the healer’s diagnosis is your prioritized to-do list: fix the flake, not the bill.
The meta-trap: every one of these is invisible on the happy path. The stack works in the demo, and the failures arrive weeks later, at night, with your token. That’s why setup sequences the webhook last and approvals first — and why the shakedown happens on a scratch branch, in daylight, with you watching.