work — heyimusa

I built a security-impact diff for GitOps

Tue, 21 Jul 2026 00:00:00 +0000

Original diagram for this build note. It shows the tool’s review model, not a production cluster topology.

A Kubernetes pull request can look harmless when the visible change is short: a new RoleBinding, one securityContext field, a route rule, or a service type. The difficult part is that these changes rarely read like the security event they create.

A Role can gain access to Secrets. A workload can begin sharing a host network namespace. A route can make a new hostname reachable. None of that has to look dramatic in a line-by-line diff.

I wanted a small tool that starts from the artifact GitOps teams already review: rendered manifests before and after a change. That became kube-blast-radius, an offline CLI that explains the security-relevant capability or surface added by that change.

The input is the delivery artifact, not another policy file

The command compares two already-rendered states. It can also render Kustomize directories or Helm charts first.

kube-blast-radius diff \
  --before rendered/main.yaml \
  --after rendered/pr.yaml

For a simple fixture, a Role gained Secret access while a Deployment enabled hostNetwork and added a privileged container. The CLI reported:

HIGH  Role/payments/checkout: RBAC rule newly grants access to secrets
      (RBAC_SECRET_ACCESS_ADDED)

HIGH  Deployment/payments/checkout: workload newly enables hostNetwork
      (WORKLOAD_HOST_NETWORK_ADDED)

HIGH  Deployment/payments/checkout: workload newly adds a privileged container
      (WORKLOAD_PRIVILEGED_ADDED)

The intent is not to decide whether every finding is wrong. A privileged DaemonSet or a public route can be intentional. The point is to turn a subtle manifest change into a question a reviewer can answer deliberately.

I treated false assurance as the main bug

The first version found the obvious cases quickly. The more important work was finding ways it could say too little.

An independent review caught a few examples. A ClusterRole referenced by a namespace-scoped RoleBinding was initially easy to miss because ClusterRoles do not have a namespace. A kind: List wrapper could hide objects from a naive document parser. And an omitted ServiceAccount and an explicit default ServiceAccount should mean the same thing, not produce a scary but meaningless finding.

Those became regression tests, not release notes:

ClusterRole + RoleBinding -> effective Secret access
kind: List              -> expanded and analyzed
missing identity        -> analysis error, not silent skip
implicit default        -> normalized before comparison

The tool also reports unsupported resource kinds as information instead of quietly declaring the change clean. Its text output always carries the same caveat: enabled checks are not a complete security assessment.

Render first, then compare

Raw YAML is useful, but many GitOps changes live in overlays and values files. The tool supports three modes:

raw        YAML manifest stream
kustomize  kubectl kustomize 
helm       helm template  with before/after values

The render commands use argument vectors rather than a shell, have a fixed timeout, and do not load kubeconfig or talk to a Kubernetes cluster. That is a deliberate boundary. Rendering still means trusting the chart or overlay source, so untrusted inputs belong in an isolated runner.

I tested the renderers inside a Docker container with a read-only filesystem, a non-root user, all Linux capabilities dropped, no-new-privileges, and read-only mounted fixtures. Kustomize caught a newly enabled hostNetwork; Helm caught a values change that made a container privileged.

A real GitOps artifact changed the scope

The useful test was a read-only render of a detached Traefik route-review artifact from a GitOps repository. I exported only that tracked review directory at the commit that introduced it, rendered it with Kustomize, and compared it with an empty prior state. No cluster, DNS provider, Secret values, or live route was accessed.

The first run did not understand Traefik IngressRoute resources. That was the right result to take seriously: an “unsupported kind” message is better than pretending a route addition has no security meaning.

I added focused support for Traefik route matches. The next run marked the added desktop and mobile route rules as high-severity exposure changes. It did not call them malicious. It made their external-surface effect visible in the review output.

What the first release covers

v0.1.0 looks for changes including:

bound RBAC access to Secrets, wildcards, escalation verbs, and broader resource-name scope;
new bindings to roles that already grant Secret access;
privileged containers, host namespaces, hostPath volumes, added Linux capabilities, UID 0, and weakened container hardening;
Service external exposure, Kubernetes Ingress hosts and paths, removed NetworkPolicies, and Traefik IngressRoute rules.

It returns text for a reviewer and JSON for CI. High findings exit with code 1; malformed manifests and renderer failures exit with 2.

What it does not claim

This is not a cluster security platform. It does not calculate full NetworkPolicy reachability, inspect cloud IAM, query a live cluster, read Secret values, or certify compliance.

That limitation is part of the product. A GitOps diff tool should be trusted for the specific questions it can answer, not for imaginary coverage.

The source, release binary, and checksum are public:

Test notes: I built and tested the CLI locally and in disposable Docker containers. The hardened renderer test used a read-only filesystem, non-root UID 65532, dropped Linux capabilities, no-new-privileges, and read-only mounted synthetic fixtures. The GitOps validation was a read-only render of a detached review artifact; no cluster, Secret value, DNS provider, or live route was accessed. The examples are sanitized.

A rollout needs a return address

Sun, 19 Jul 2026 17:31:09 +0000

Original diagram for this note. It describes a Docker-only simulated release flow, not a production deployment or benchmark.

I do not trust a deployment plan until I can explain what happens when the new version is the problem.

That sounds obvious. It is still easy to build a delivery process around the happy path: create an image, update a manifest, watch the rollout, call it done. The awkward part starts when a release looks healthy enough to leave the pipeline but is not healthy enough to keep.

To keep this small, I ran a toy release flow in a disposable Docker container. There was no host change, cluster, registry, or real service. The container held a desired-state file for a fictional checkout service, plus a saved copy of the known-good release.

apply: checkout:1.4.3
healthcheck: /readyz -> FAIL (simulated)
rollback: checkout:1.4.2
healthcheck: /readyz -> OK (simulated)
final_desired_state: image=checkout:1.4.2 replicas=3

The experiment is intentionally boring. That is the point. A rollback should not require somebody to reconstruct the previous state from memory while a production graph turns red.

The release needs a return address

A deployment is a state transition. The candidate release is only one half of that transition; the other half is the state you can return to when the candidate fails.

In the tiny experiment, that state was just a saved file:

image=checkout:1.4.2
replicas=3

Real systems are less neat. There may be configuration changes, schema compatibility, feature flags, asynchronous workers, or traffic shifts. But the basic question does not change: what exact state are we restoring, and can the deployment system express it?

If the answer is “we will figure it out,” the rollback plan is not really a plan.

Rollback is not an apology button

People often talk about rollback as if it is the opposite of deployment. It is not. It is another deployment, with the same need for identity, evidence, and verification.

A useful rollback path has at least three properties:

The previous artifact or desired state is identifiable.
The path to apply it is known before the incident.
The system has a signal that says the restored version is actually healthy.

The third item gets skipped surprisingly often. Reverting an image tag is not proof that the service recovered. It only proves that the deployment controller accepted another instruction.

Why declarative state helps

This is where GitOps and other declarative delivery patterns earn their keep. They make the intended state visible. They also make a reversal more concrete: restore a reviewed revision, reconcile it, and watch the same health signals that justified the rollout.

That does not make every rollback safe. Database migrations can make a simple reversal impossible. A downstream dependency may have changed underneath you. A feature flag may be the safer first lever. Declarative state is not magic; it just removes one common source of panic: having to guess which version and configuration were running before the change.

The part I would test next

The toy flow did not cover the hard cases. It did not include a database, traffic management, or an actual Kubernetes controller. It only checked the shape of the idea: a failed health check should lead to a named previous state, then to a second health check.

The next useful step is to run the same exercise against a non-critical service in an isolated environment:

deploy a known candidate;
deliberately fail a readiness condition;
reconcile the previous revision;
verify the restored service through the same route and alert signal users depend on.

If that feels cumbersome in a test environment, it will feel worse during an incident.

Test notes: This note is based on a disposable Docker container limited to 0.25 CPU and 128 MiB memory. It simulated a declarative checkout release changing from 1.4.2 to 1.4.3, a failed readiness check, and restoration to 1.4.2. No production infrastructure, repository, cluster, registry, or database was accessed.