Grafana Breach Exposed Private Repositories After a Missed GitHub Workflow Token Rotation

A malicious TanStack npm dependency stole CI/CD workflow credentials and one token left active after incident response was later used to access Grafana's private repositories

Grafana confirmed its breach stemmed from a compromised CI/CD environment after malicious TanStack npm packages exfiltrated GitHub workflow tokens and one token missed during rotation was later used to access private repositories.

The incident is a clear lesson in credential governance: rotation must be complete, validated and supported by controls that limit token scope, lifetime and reuse. Grafana said it rotated a significant number of GitHub workflow tokens after detecting malicious activity on May 1, but a subsequent review found a workflow initially judged unaffected had also been compromised. The result was source code theft and exposure of business operational information, even though Grafana said customer production systems and production data were not impacted. The operational failure was not only the initial supply-chain compromise but the inability to fully invalidate stolen credentials across CI/CD workflows. This highlights a broader cloud security issue: tokens used across build systems, repositories and service integrations require the same governance rigour as IAM credentials, including inventory, scoping, issuance controls and provable revocation.

What went wrong

What’s happening Cause Action
A stolen CI/CD token remained valid after incident response Grafana said malicious TanStack packages executed in its GitHub environment and exfiltrated GitHub workflow tokens. Although many tokens were rotated, one token was missed and later used to access private repositories. Validate that all CI/CD, workflow and service tokens can be fully inventoried and revoked during incident response. SkySiege would assess whether credential rotation processes are complete, repeatable and verified rather than assumed.
Token rotation depended on incomplete impact scoping Grafana said a specific GitHub workflow was originally deemed not impacted but was later confirmed compromised. That misclassification left an active credential in place. Validate that credential revocation is triggered by exposure conditions, not only by initially confirmed compromise. SkySiege would assess whether organisations rely on partial workflow scoping instead of broad forced rotation when build environments are exposed.
Workflow tokens lacked sufficient usage constraints The available evidence shows a GitHub workflow token could be stolen from the CI environment and reused by the attacker to access private repositories. The source does not provide token configuration details, but the misuse indicates the token retained meaningful access after theft. Validate token scope, TTL, repository restrictions and whether ephemeral or tightly bounded credentials are used wherever possible. SkySiege would assess whether long-lived or over-permissive service and workflow tokens create reusable access paths across code and operational systems.
Supply-chain compromise propagated directly into the build environment Grafana’s CI/CD workflow consumed a malicious npm package, allowing credential-stealing code to run inside the GitHub environment. Validate dependency trust controls in CI/CD, including package provenance, allow-listing and isolation of build credentials from untrusted execution. SkySiege would assess whether external package ingestion can expose privileged automation tokens.
Operational and business data was exposed beyond source code Grafana said the attacker downloaded operational information and business contact details in addition to source code. Validate repository content boundaries and ensure business-sensitive operational data is not unnecessarily accessible through engineering systems. SkySiege would assess whether source control platforms hold mixed sensitivity data that expands breach impact.

Why this matters

This incident shows that token rotation is not just a cleanup step after compromise; it is a core containment control. A single missed token was enough to extend a supply-chain event into unauthorised repository access and data theft. That creates a direct lesson for cloud and platform teams: if credentials exist in automation, every token must be discoverable, scoped and revocable under pressure.

The detection gap is also clear. Grafana detected malicious activity and initiated response, but detection alone did not prevent continued access because revocation was incomplete. Organisations need visibility into where workflow tokens are issued, what they can access, how long they remain valid and whether they are actually invalidated after exposure. Without that, incident response can appear complete while usable credentials remain active.

From a governance perspective, this is a failure of credential lifecycle control. Tokens in CI/CD should be treated like privileged IAM access: least privilege, short lifetime, restricted audience and strong issuance controls. The same principle applies broadly to service token mechanisms and metadata-issued credentials; if token parameters are weak or unbounded, theft becomes materially more damaging.

Business impact extends beyond technical compromise. Grafana reported source code theft and exposure of operational business information. Even without customer production impact, this introduces reputational damage, legal review, disclosure pressure and potential diligence concerns around software supply-chain resilience, access governance and response maturity.

References

Original Article