A failed CI run is evidence, not a replay recipe

heyimusaReading

A failed GitHub Actions run is not a reproduction recipe. It is evidence from a particular revision, runner image, dependency state, and execution context. The useful first move is to preserve that evidence without pretending the environment can be recreated from one log line.

I released ci-capsule v0.1.0 after building it around that constraint. It is a small, read-only CLI for failed GitHub Actions runs: collect a local evidence bundle, sanitize bounded credential-shaped values, recover a static command only when the workflow source supports it, and say unavailable when it does not.

The debugging gap is usually evidence, not another runner

A failed workflow already has useful material scattered across GitHub: the run identity, commit SHA, job and step outcomes, the workflow source at that SHA, a failed-job log, and metadata for any artifacts that existed.

The usual handoff loses some of that context. Someone copies a failing command into chat, another person checks out the current branch rather than the failed SHA, and a local success becomes a vague answer to a different question.

ci-capsule keeps the boundary deliberately narrow:

  • it makes read-only GitHub API requests;
  • it writes a local bundle rather than uploading evidence to another service;
  • it inventories artifact metadata but does not download artifact or cache contents;
  • it does not rerun workflows or execute a recovered command.

That last point is important. A command that appeared in a workflow is not automatically safe or meaningful to replay. Matrix values, expressions, inherited environment, action setup, shell selection, working directories, and runner behavior can all change what a step actually means.

A candidate is earned, not guessed

The CLI only emits a replay candidate for a uniquely matched, literal one-line run: step with a sufficiently simple execution context. Complex or inherited semantics turn into evidence-only or unavailable output instead.

For example, the usable outcome is intentionally modest:

REPLAY_CANDIDATE go test ./...

That says: this command was recovered from the exact static workflow source, at a cited source line. It does not say it recreates a GitHub-hosted runner.

If the tool cannot establish the same thing, it should not fill the gap with a confident shell command. An unavailable result is often the more operationally honest output because it tells the reviewer where manual context is still required.

The bundle is designed for a bounded review

A capsule includes sanitized workflow and failed-log evidence, source identity, static analysis, and artifact metadata. The sanitization model is best-effort, not a promise that an arbitrary log is secret-free. The release notes and README say that directly because credential-shaped matching cannot recognize every proprietary token or sensitive string.

The implementation also treats ambiguous parsing as a stop condition. Duplicate YAML keys, anchors, aliases, merge keys, and multiple YAML documents are rejected rather than interpreted creatively. ZIP job logs are bounded; oversized or unsafe entries fail instead of silently producing partial evidence.

These are small constraints, but they matter when a tool is placed between an incident and a decision. A compact local bundle is useful only if its omissions are visible.

What I verified for v0.1.0

The release process was intentionally kept separate from the claim that the tool can analyze a failed run.

Before publishing, I verified:

  • unit tests and go vet locally;
  • a read-only collection against a public failed GitHub Actions run;
  • an independent review of the evidence, filesystem, redirect, YAML, and replay boundaries;
  • a hardened container acceptance path with network disabled, a read-only root filesystem, a non-root user, dropped capabilities, and no-new-privileges;
  • a draft release archive, checksums, CycloneDX SBOM, and embedded binary version;
  • the same checks again from the publicly downloaded release assets.

The release asset is Linux amd64 only for now. Its checksums and SBOM are useful provenance inputs, not a claim that an artifact is benign or that the entire supply chain is solved.

Try it, but keep the claim small

The project and release assets are available at:

The intended question is simple: what can I preserve and state honestly about this failed run before I ask someone to reproduce it?

That is a better starting point than turning a log excerpt into a promise.

Sources