Original diagram for this note. It describes a Docker-only simulated release flow, not a production deployment or benchmark.
I do not trust a deployment plan until I can explain what happens when the new version is the problem.
That sounds obvious. It is still easy to build a delivery process around the happy path: create an image, update a manifest, watch the rollout, call it done. The awkward part starts when a release looks healthy enough to leave the pipeline but is not healthy enough to keep.
To keep this small, I ran a toy release flow in a disposable Docker container. There was no host change, cluster, registry, or real service. The container held a desired-state file for a fictional checkout service, plus a saved copy of the known-good release.
apply: checkout:1.4.3
healthcheck: /readyz -> FAIL (simulated)
rollback: checkout:1.4.2
healthcheck: /readyz -> OK (simulated)
final_desired_state: image=checkout:1.4.2 replicas=3
The experiment is intentionally boring. That is the point. A rollback should not require somebody to reconstruct the previous state from memory while a production graph turns red.
The release needs a return address
A deployment is a state transition. The candidate release is only one half of that transition; the other half is the state you can return to when the candidate fails.
In the tiny experiment, that state was just a saved file:
image=checkout:1.4.2
replicas=3
Real systems are less neat. There may be configuration changes, schema compatibility, feature flags, asynchronous workers, or traffic shifts. But the basic question does not change: what exact state are we restoring, and can the deployment system express it?
If the answer is “we will figure it out,” the rollback plan is not really a plan.
Rollback is not an apology button
People often talk about rollback as if it is the opposite of deployment. It is not. It is another deployment, with the same need for identity, evidence, and verification.
A useful rollback path has at least three properties:
- The previous artifact or desired state is identifiable.
- The path to apply it is known before the incident.
- The system has a signal that says the restored version is actually healthy.
The third item gets skipped surprisingly often. Reverting an image tag is not proof that the service recovered. It only proves that the deployment controller accepted another instruction.
Why declarative state helps
This is where GitOps and other declarative delivery patterns earn their keep. They make the intended state visible. They also make a reversal more concrete: restore a reviewed revision, reconcile it, and watch the same health signals that justified the rollout.
That does not make every rollback safe. Database migrations can make a simple reversal impossible. A downstream dependency may have changed underneath you. A feature flag may be the safer first lever. Declarative state is not magic; it just removes one common source of panic: having to guess which version and configuration were running before the change.
The part I would test next
The toy flow did not cover the hard cases. It did not include a database, traffic management, or an actual Kubernetes controller. It only checked the shape of the idea: a failed health check should lead to a named previous state, then to a second health check.
The next useful step is to run the same exercise against a non-critical service in an isolated environment:
- deploy a known candidate;
- deliberately fail a readiness condition;
- reconcile the previous revision;
- verify the restored service through the same route and alert signal users depend on.
If that feels cumbersome in a test environment, it will feel worse during an incident.
Test notes: This note is based on a disposable Docker container limited to 0.25 CPU and 128 MiB memory. It simulated a declarative checkout release changing from 1.4.2 to 1.4.3, a failed readiness check, and restoration to 1.4.2. No production infrastructure, repository, cluster, registry, or database was accessed.
Related work: Turning deployments into a repeatable platform capability