The traps
You’re wiring an internet-facing endpoint to an agent that can write to your repo. Every trap here comes back to that.
Without a secret in platforms.webhook.extra,
anyone who POSTs to your URL runs an agent with your GitHub token, and the
template pastes their branch name straight into the prompt.
Fix: Set the secret before the port is open, confirm a redelivery with a wrong secret gets dropped, and have the route prompt treat payload fields as data, never as instructions.
Deleting the assertion is the shortest way to green. The sneaky versions
look responsible: a wider try/except, a tolerance bumped from
0.001 to 0.1, an xfail with a thoughtful comment. No deny glob
catches a file edit.
Fix: The prompt forbids touching tests, and your review enforces it. Treat any
ci-heal PR that touches tests/ as guilty until
proven innocent.
Its PR branch runs CI, that run fails, and the webhook starts a run to heal
the healing. At 3am you get PRs about PRs. The rate_limit only
slows this down.
Fix: Make the filters drop ci-heal/* branches, and
run prompt 4’s redelivery test even if the config looks right.
CI logs leak secrets GitHub doesn’t know to mask, like a connection string in a stack trace or a curl echoed with its auth header. The agent sends those logs to a model API and may quote them in a PR.
Fix: Fetch only --log-failed and tail it, keep the scrub
instruction, and pick the model provider that sees them on purpose. Your
own endpoint is an option.
local is the default backend. If the gateway’s runs land
there, the healer installs dependencies as you, on your host, and none of
the container hardening applies.
Fix: Prompt 3 makes the agent prove where commands ran.
One flaky test failing nightly means thirty runs a month, each concluding the network had a bad night. On a frontier model, that’s a steady bill for zero merged diffs.
Fix: Prompt 4 prices a bad month up front. Use a cheap model, keep the one-attempt rule, and fix the flake.
All of this is invisible in the demo and shows up weeks later, at night, with your token. That’s why setup locks approvals before opening the webhook, and why you run the shakedown in daylight while you watch.