A cloud provider can remove a platform-wide attack path and still leave every customer with work to prove that their own recovery is complete.
On July 30, 2026, Wiz published CosmosEscape, a report about a critical vulnerability chain in Azure Cosmos DB's Gremlin API. The report says the chain could have produced read and write access across Cosmos DB accounts, including accounts isolated from the public network. Microsoft says it remediated the issue, found no evidence of unauthorized activity beyond the researchers' testing, and says no customer action is required.
That is good news. It is also not the same thing as an account owner having evidence that their Cosmos DB estate is ready for the next credential or isolation failure.
This article is about the operational work that remains useful after a cloud-service incident is fixed: separating vendor remediation from local recovery evidence, reducing the power of long-lived account keys, and making data-plane and control-plane activity reviewable. It does not reproduce CosmosEscape, interact with Azure, send Gremlin queries, or publish exploit material.
What CosmosEscape was, at the level operators need
According to Wiz, the entry point was the Apache Gremlin API in Azure Cosmos DB. Its researchers describe finding a way to escape the restrictions around query execution while running queries against a database they controlled. The chain allegedly reached a multi-tenant internal query service, then credentials available to that service, then a platform-wide signing capability Wiz calls the “Cosmos Master Key.”
Wiz says that key could retrieve an account's primary key across tenants, regions, and API families. It also says the same capability reached an internal configuration store containing account metadata such as account names, tenant and subscription identifiers, network settings, and tags. In the reported chain, the configuration store made targeting possible and the account-key path made access possible.
That is the part worth sitting with: a tenant-controlled query reached a service boundary that could resolve another tenant's account credentials. The report is not mainly a story about graph queries. It is a story about privileged service-to-service authority sitting too close to tenant-controlled execution.
Microsoft's statement, published in the Wiz report, says it investigated access logs and found no unauthorized activity outside the researchers' testing and no customer data access. It says Microsoft blocked the reported Gremlin entry point within 48 hours, performed penetration testing for similar vectors, then completed additional hardening around service-to-service authentication, network protections, monitoring, and detection.
Those statements should be quoted accurately:
- Microsoft says the issue is fully remediated and no customer action is required for this incident.
- Wiz says Microsoft eliminated the reported Cosmos Master Key and rolled out a long-term architectural fix across regions before public disclosure.
- Neither statement means a customer should invent evidence of compromise, rotate every key blindly, or treat an unrelated 401 as proof of exploitation.
The responsible response is narrower: record the vendor statement, decide what local assurance is still appropriate for the account's risk, and avoid claiming that a platform fix proves every customer environment is recovered.
Why private networking was not the whole boundary
Private endpoints and firewall rules are still worth deploying. Microsoft recommends restricting Cosmos DB access to a virtual network, disabling public network access, and using private endpoints for connecting services. Those controls reduce ordinary internet exposure and narrow which network paths can reach an account.
CosmosEscape is a reminder that network isolation is not a complete answer when an internal service that enforces or brokers that isolation is itself compromised. Wiz reports that the affected query service could reach private and network-isolated accounts, and that the configuration store held network settings. If that chain is accurate, the control plane and the enforcement point become part of the threat boundary.
This does not make private endpoints useless. It changes the claim we should make about them.
Private endpoint only
-> reduces direct client reachability
-> does not prove every provider-internal path is unable to reach the account
Data-plane RBAC + managed identity
-> removes or reduces application reliance on a shared account key
-> does not replace provider-side tenant isolation
Provider remediation
-> removes the reported platform attack path
-> does not create local audit evidence, rotation records, or restore tests
A mature design uses more than one of those controls. It also records what each control cannot prove.
The shared account key problem
The most useful customer-side lesson is not “rotate keys because CosmosEscape happened.” Microsoft says customers do not need to take incident action. The lesson is that account keys have a large authorization surface and are hard to reason about once copied into applications, jobs, CI systems, vaults, and third-party integrations.
Microsoft's Cosmos DB security guidance recommends managed identities, control-plane RBAC, data-plane RBAC, separate identities for control-plane and data-plane work, and regular rotation of primary and secondary keys when key-based authentication remains in use. The guidance also recommends moving toward Microsoft Entra ID authentication rather than embedding credentials in code.
The account-key rotation documentation makes the operational cost explicit. Primary and secondary keys are designed for an overlap period, but regeneration can take from one minute to multiple hours. An application needs to use one key consistently before the other can be validated and the old key regenerated. In June 2026, Microsoft documented a preview feature that reports when account keys were last used before rotation or before local authentication is disabled. Microsoft also labels that preview as unsuitable for production workloads under an SLA.
That last detail matters. Key-usage metadata can support a migration decision. It should not become the only thing that proves an application has stopped using keys.
A reasonable target state is:
| Capability | Transitional state | Preferred state |
|---|---|---|
| Application data access | Primary or secondary account key from a secret store | Managed identity with a narrowly scoped data-plane RBAC role |
| Administrative changes | SDK operations through an account key | Azure RBAC through a dedicated control-plane identity |
| Network path | Public endpoint with allowlists | Public network disabled and private endpoints used where the workload permits |
| Evidence | Ad hoc portal screenshots | Diagnostic logs, deployment records, and tested runbooks |
| Recovery | “the vendor fixed it” | Documented account review, tested restore, and an explicit decision about keys |
There will be exceptions. Legacy SDKs, API compatibility constraints, migration windows, and partner products can keep keys in service longer than anyone wants. The point is to make that an explicit temporary state with an owner and a measurable exit path.
Do not confuse key rotation with a universal incident requirement
There are two bad reactions to a public cloud vulnerability.
The first is doing nothing because the vendor says the service is fixed. The second is rotating every credential across every account without knowing which applications use which key, then calling the resulting outage “security.” Both are avoidance.
For CosmosEscape specifically, Microsoft's published position is that no customer action is required. An organization should not contradict that statement with a blanket claim that every Cosmos DB key is known compromised. There is no public evidence for that claim, and Microsoft says its review found no unauthorized activity beyond research testing.
But a high-assurance organization can still choose a post-incident review. The review should ask questions that have useful answers even if the incident never touched the account:
- Which Cosmos DB accounts use Gremlin, and which accounts have a business reason to retain that API?
- Which workloads still authenticate with account keys rather than Entra ID and data-plane RBAC?
- Do primary and secondary keys have named consumers, an owner, and a documented rotation procedure?
- Is public network access disabled where the service dependencies allow it?
- Can the team retrieve data-plane and control-plane evidence for a defined investigation window?
- Has the restore path been tested, not merely configured?
Those are posture questions, not indicators of compromise.
Evidence is different on the data plane and the control plane
Cosmos DB has two operational surfaces that teams often log separately, if they log them at all.
The data plane is where applications read, write, delete, and query data. Microsoft documents Cosmos DB resource logs as per-request data-plane logs. Diagnostic settings can send them to Log Analytics, storage, Event Hubs, or another destination. Microsoft also warns that enabling full-text query logging reveals deobfuscated query text and can increase cost; it recommends disabling that troubleshooting feature afterward.
The control plane is where an account's configuration changes: regions, throughput, failover, virtual networks, and other resource settings. Microsoft documents control-plane diagnostic logging as start and complete events. It specifically gives firewall-rule changes and region changes as cases where alerts are useful.
These logs answer different questions.
| Question | Better evidence |
|---|---|
| Did a client read, write, delete, or query data? | Data-plane diagnostic logs |
| Did someone change a network rule, add a region, or alter account configuration? | Control-plane diagnostic logs and Azure Activity Log |
| Did the service still work after key migration? | Application metrics, synthetic transactions, and deployment evidence |
| Can we restore a database or container to a useful state? | A timed restore exercise with documented result |
There is an important sequencing rule in Microsoft's control-plane audit guidance. Before auditing control-plane operations, it recommends disabling key-based metadata write access through disableKeyBasedMetadataWriteAccess. Once enabled, SDK clients using account keys cannot make metadata-changing calls; those changes need an identity with appropriate Azure role permissions through Azure Resource Manager, CLI, PowerShell, or the portal.
That is a security improvement only after dependency discovery. Turning it on without inventory can break workflows that create containers, update throughput, or change resources through an SDK. The correct change is staged: observe, identify dependencies, migrate them, validate, then enforce.
I tested the recovery distinction without touching Azure
I built a small synthetic checker to make one narrow point: a vendor remediation flag does not create account-level recovery evidence.
The checker contains no Azure SDK, no credentials, no network client, no Gremlin parser, and no exploitation logic. It evaluates two invented account records:
platform-remediated-account-without-local-evidenceaccount-recovery-evidence-complete
The first record confirms vendor remediation but has no documented investigation, no account-key usage review, no primary or secondary key-rotation evidence, no data-plane or control-plane logs, no tested restore, and no evidence of public-network reduction, local-auth disablement, or data-plane RBAC.
The second record has the same vendor-remediation confirmation plus all of those local controls and evidence markers.
I ran the checker in a disposable Docker container with no network, a read-only root filesystem, a 16 MiB noexec temporary filesystem, all Linux capabilities dropped, no-new-privileges, UID 65532, one CPU, 128 MiB memory, and a PID limit of 64.
The result for the first synthetic record was:
{
"case": "platform-remediated-account-without-local-evidence",
"platform_remediation_confirmed": true,
"status": "vendor_remediation_not_account_recovery",
"missing_recovery_evidence": [
"documented account-scoped investigation",
"account-key usage review",
"primary key rotation evidence",
"secondary key rotation evidence",
"data-plane diagnostic logs",
"control-plane diagnostic logs",
"tested restore evidence"
]
}
The complete synthetic record returned:
{
"case": "account-recovery-evidence-complete",
"platform_remediation_confirmed": true,
"status": "recovery_evidence_complete",
"missing_recovery_evidence": [],
"future_blast_radius_hardening_gaps": []
}
This is not a scanner. It cannot tell whether a real Cosmos DB account was exposed, whether a key was used by an attacker, or whether Microsoft telemetry is complete. It is a guard against a weaker claim: that a vendor remediation announcement alone is proof of a customer's recovery state.
A bounded review runbook
If I were responsible for a Cosmos DB estate after this disclosure, I would open a time-boxed review ticket rather than an uncontrolled incident.
1. Record the vendor position and scope the estate
Record the Wiz disclosure date, Microsoft's statement that no customer action is required, and the fact that the reported entry point involved Gremlin. Inventory accounts by API type, environment, data classification, public-network setting, and authentication mode. Do not put account keys, connection strings, tenant identifiers, or customer data into the ticket.
2. Preserve the evidence path before changing it
Confirm whether data-plane and control-plane diagnostics are enabled, where they go, their retention period, and who can query them. If logs need to be enabled, capture the activation timestamp. Evidence does not exist retroactively just because a setting is enabled today.
3. Review authentication deliberately
For every account still using keys, name the primary and secondary key consumers. Check whether the application can move to managed identity and data-plane RBAC. If rotation is chosen, use the dual-key sequence and validate the replacement key before regenerating the old one. Treat key-usage metadata as supporting evidence, not the only evidence.
4. Reduce future blast radius without breaking the workload
Plan private endpoints and public-network disablement where feasible. Separate identities for data access from identities that alter account configuration. Identify SDK metadata operations before enabling disableKeyBasedMetadataWriteAccess. Run the migration as a change with a rollback plan, not as a checkbox.
5. Test recovery
Continuous backup is useful only if the organization can restore a database or container, validate application behavior, and account for time and cost. Microsoft recommends regular restore testing. Record the objective, point-in-time selected, recovery time, validation result, and cleanup.
6. Close with an explicit conclusion
The conclusion should be one of these, not a vague green status:
No local action selected: vendor statement accepted; posture review deferred with owner/date.
Posture review complete: account inventory, logging, auth posture, and restore evidence recorded.
Migration in progress: named key consumers and enforcement changes tracked to a dated plan.
A decision that nothing more is needed can be correct. It should still be a decision with evidence behind it.
What this incident changes
CosmosEscape does not establish that Azure Cosmos DB customers were compromised. Microsoft's published position is the opposite: it found no evidence of unauthorized activity or customer-data access, and it says the vulnerability has been fully remediated.
It does establish a useful design lesson. In a multi-tenant cloud service, the most dangerous boundary is often not the public endpoint. It is the place where tenant-controlled execution can touch an internal service identity, a credential broker, or a configuration system with authority beyond that tenant.
Customers cannot audit provider internals. They can avoid duplicating that same pattern in their own architecture. Keep tenant-controlled input away from privileged brokers. Prefer identity-scoped access over shared account keys. Treat private networking as one layer, not a certificate that no internal path exists. Keep enough data-plane and control-plane evidence to answer a real question later.
The platform fix closes the reported path. The customer work is to make the next path smaller, easier to see, and less powerful.
Experiment notes: I created a synthetic recovery-contract checker, not an Azure client or vulnerability reproducer. It made no network calls and used no cloud credentials. It ran once in Docker using python:3.12-alpine with network disabled, read-only root filesystem, 16 MiB noexec tmpfs, UID 65532, all Linux capabilities dropped, no-new-privileges, one CPU, 128 MiB memory, and PID limit 64.
Sources: Wiz: CosmosEscape · Microsoft Learn: Secure your Cosmos DB account · Microsoft Learn: Rotate keys · Microsoft Learn: data-plane diagnostic logs · Microsoft Learn: control-plane audit logs