Agent stack build plans · № 4 · Hermes CI/CD healer
Product spec
A junior engineer who only works nights, only touches one repo, and is contractually forbidden from clicking merge.
Who this is for
- The two-to-eight-person team whose main branch protection is “Priya notices.” Nightly and cron workflows fail into the void; this gives the void a work ethic.
- The solo maintainer with a scheduled build (docs, data refresh, release smoke test) that breaks while they sleep and greets them as a red X over breakfast.
- The OSS maintainer drowning in “CI is red on main” — with the caveat that you run it on your branches, not on arbitrary fork PRs (see traps; running an agent against untrusted contributors’ code is a different threat model and not this plan).
A day in the life
Monday, 18:40. Dana pushes a refactor to
feat/date-ranges and logs off. The nightly suite runs at 02:00.
02:14. Two tests fail. GitHub fires the webhook. Hermes
validates the signature, the ci-heal route renders its prompt
from the payload, and the agent goes to work in its container: fetch the
failed-job logs, clone, checkout, reproduce, patch, re-run.
02:24. Suite’s green in the sandbox. PR #413 opens against Dana’s branch with the diagnosis and test output in the body. A one-line summary lands as a comment on the failed run — or in the team Telegram, if you prefer waking up to it there.
09:03. Dana reads a seven-line diff instead of an hour-old stack trace, requests one rename, merges. The failure existed for seven hours; it occupied a human for four minutes.
The other Monday. The failure is a flaky S3 timeout. The agent can’t fix the weather, says so, and delivers a diagnosis instead of a PR. That degraded mode — failure → explanation in chat — is the documented Hermes webhook use case, and it’s the floor this product stands on: even when healing fails, triage succeeded.
What it deliberately does not do
- Merge. Ever. The agent opens PRs; humans merge them.
This is a product decision, not a technical limit — enforce it in the
route prompt and in
approvals.deny(setup, stage 6). - Push to the broken branch. Fixes arrive on a fresh
ci-heal/*branch as a PR. Nobody wants an agent rewriting the branch they’re working on overnight. - Modify tests to make them pass. The prompt forbids it; your review habit is the backstop. A trap in its own right — trap 2.
- Retry infra flakes forever. One attempt per failure.
If the diagnosis is “flaky,” it reports and stops. Retries are
what
retrybuttons are for. - Handle more than one repo — in this plan. Routes generalize (one route per repo), but earn trust on one repo first.
The honest success rate
Expect the agent to genuinely heal the mechanical majority: breakage with a clear stack trace, a recent diff to blame, and a test that reproduces locally. Expect it to fail on flakes, environment drift between CI and the sandbox, and anything whose real cause is a design disagreement. Budget for maybe half of failures ending in a PR and the rest ending in a good diagnosis — and notice that the second half still beats reading raw CI logs at 9am. If your failures are mostly the unfixable kind, build only the notify-and-diagnose half and keep your evening.