Grafana Breach Exposed Private Repositories After a Missed GitHub Workflow Token Rotation

A malicious TanStack npm dependency stole CI/CD workflow credentials and one token left active after incident response was later used to access Grafana's private repositories

Grafana confirmed its breach stemmed from a compromised CI/CD environment after malicious TanStack npm packages exfiltrated GitHub workflow tokens and one token missed during rotation was later used to access private repositories.

The incident is a clear lesson in credential governance: rotation must be complete, validated and supported by controls that limit token scope, lifetime and reuse. Grafana said it rotated a significant number of GitHub workflow tokens after detecting malicious activity on May 1, but a subsequent review found a workflow initially judged unaffected had also been compromised. The result was source code theft and exposure of business operational information, even though Grafana said customer production systems and production data were not impacted. The operational failure was not only the initial supply-chain compromise but the inability to fully invalidate stolen credentials across CI/CD workflows. This highlights a broader cloud security issue: tokens used across build systems, repositories and service integrations require the same governance rigour as IAM credentials, including inventory, scoping, issuance controls and provable revocation.

What went wrong

What Happened Cause Action
A stolen CI/CD token remained valid after incident response Grafana said malicious TanStack packages executed in its GitHub environment and exfiltrated GitHub workflow tokens. Although many tokens were rotated, one token was missed and later used to access private repositories. Validate that all CI/CD, workflow and service tokens can be fully inventoried and revoked during incident response. SkySiege provides this visibility including age governance, last rotation and number of tokens attached to an identity.
Token rotation depended on incomplete impact scoping Grafana said a specific GitHub workflow was originally deemed not impacted but was later confirmed compromised. That misclassification left an active credential in place. Validate that credential revocation is able to be triggered on demand and be able to be linked back to intended operating environments. SkySiege surfaces human readable information on token activity and usage to provide direct data on what the real activity of tokens are.
Workflow tokens lacked sufficient usage constraints The available evidence shows a GitHub workflow token could be stolen from the CI environment and reused by the attacker to access private repositories. The source does not provide token configuration details, but the misuse indicates the token retained meaningful access after theft. Validate token scope, TTL, repository restrictions and whether ephemeral or tightly bounded credentials are used wherever possible.
Supply-chain compromise propagated directly into the build environment Grafana’s CI/CD workflow consumed a malicious npm package, allowing credential-stealing code to run inside the GitHub environment. Validate dependency trust controls in CI/CD, including package provenance, allow-listing and isolation of build credentials from untrusted execution.
Operational and business data was exposed beyond source code Grafana said the attacker downloaded operational information and business contact details in addition to source code. Validate repository content boundaries and ensure business-sensitive operational data is not unnecessarily accessible through engineering systems. SkySiege has a number of detections of cloud infrastructure that analyse the ease of exfiltration of data and the general accessibility and capability of lateral movement across an environment.

Why this matters

This incident shows that token rotation is not just a cleanup step after compromise; it is a core containment control. A single missed token was enough to extend a supply-chain event into unauthorised repository access and data theft. That creates a direct lesson for cloud and platform teams: if credentials exist in automation, every token must be discoverable, scoped and revocable under pressure.

The detection gap is also clear. Grafana detected malicious activity and initiated response, but detection alone did not prevent continued access because revocation was incomplete. Organisations need visibility into where workflow tokens are issued, what they can access, how long they remain valid and whether they are actually invalidated after exposure. Without that, incident response can appear complete while usable credentials remain active.

From a governance perspective, this is a failure of credential lifecycle control. Tokens in CI/CD should be treated like privileged IAM access: least privilege, short lifetime, restricted audience and strong issuance controls. The same principle applies broadly to service token mechanisms and metadata-issued credentials; if token parameters are weak or unbounded, theft becomes materially more damaging.

Business impact extends beyond technical compromise. Grafana reported source code theft and exposure of operational business information. Even without customer production impact, this introduces reputational damage, legal review, disclosure pressure and potential diligence concerns around software supply-chain resilience, access governance and response maturity.

References

Original Article