A service account nobody owned.
And a second key nobody made.
Service accounts do not offboard. They hold long-lived credentials, they outlive the teams that created them, and nothing about them changes when a person leaves. This one kept running a nightly export for ten months after the engineer who provisioned it was gone. By the time it surfaced in an identity sweep, its production key had already minted a second key from a German hosting provider. The standard remediation, disable the account, would have broken the finance close and told the operator they had been found.
sa-bq-export-legacy, provisioned by Terraform in 2023. No owner label, no listed owner, no surviving team. 1,269 days old, 399 of them observable, still writing to a production bucket every night at 02:00.The first question is not “is this malicious.”
It is “what is normal for this account.”
The Worker’s first query was not about the service account. It was about which of the 61 connected sources could say anything about a cloud identity at all. Thirteen could. Four of the places worth looking, Terraform state, source control, CI/CD, and Kubernetes manifests, could not be reached. That was written down before any conclusion depended on it.
The second query looked for the account’s creation event and returned nothing. The Cloud Logging sink keeps 400 days. The account is 1,269 days old. So the first real finding was a boundary: anything claimed about this identity before 2025-08-02 has to come from somewhere other than GCP.
Inside the window, the shape was unmistakable. 1,228 events. 1,191 of them from a single internal address, three operations, always in the 02:00 hour, on 397 of 399 nights. A backup job with a heartbeat.
Then the same query returned 37 events that did not fit any of that.
Baseline · 399 days
- Source
- 10.128.0.37
- Hour
- 02:00, always
- Operations
- 3
- Datasets
- 1 insights_v2
- Failures
- 0
- Credential
- a3f19c4e (2023)
Last six days
- Source
- 46.4.118.207 / .211
- Hour
- 09, 11, 16, 19, 23
- Operations
- 4
- Datasets
- 5
- Failures
- 12
- Credential
- 7d2b0491 (2026-08-28)
One row decided the case. On 2026-08-28 at 23:47 UTC, the service account created a new key for itself from a German hosting provider, and the call was authenticated with the production key. That is not an indicator. It is proof that the credential the nightly job has used since 2023 was in somebody else’s hands.
It also created the problem the rest of the investigation is about. The compromised credential and the production credential are the same credential.
Nobody remembering it does not mean it has no owner.
GCP was asked first and had nothing. Owner field empty on all 37 resource records. Labels empty. This is the ordinary condition of most cloud estates and it is exactly why the ticket existed.
One query against four other systems answered it. Not because the query was clever, but because those four systems were reachable in the same breath as the fifth.
| Asked | Answered |
|---|---|
| GCP Cloud Audit | Owner unknown. 37 records, owner field empty on every one. |
| ServiceNow | CHG0031204, 2023-03-14. Requested by D. Patel, Data Platform Engineering. Built by terraform module tf-modules/bq-export v1.2.0. Key stored in CI variable GCP_EXPORT_KEY. |
| Google Workspace Directory | D. Patel disabled 2025-11-07, offboarding. Owning group data-platform-eng now has zero members. |
| Cribl | Pipeline gcs-insights-export reads the bucket this account writes. Two downstream routes: finance-close-recon, exec-daily-brief. |
| AWS CloudTrail ×2 | Nothing. Clean negative, and it bounds the blast radius to one cloud. |
Four queries, three systems, and the two questions the ticket could not answer were both closed. The owner left ten months ago and the team that owned it no longer exists. And disabling this account breaks the finance close reconciliation and the executive daily brief, silently, at 02:00, with nobody left to notice.
The change record also handed over the most important lead in the case, and the mesh could not follow it. The key is in a CI variable, on a pipeline, in a system that is not connected. That is recorded as an open question, not resolved with a guess.
The denied calls were worth more than the successful ones.
Thirty-six events on the new key across six days. Dataset enumeration across the project, then table enumeration against four datasets the account had never touched in 399 days. Twenty-four of those calls succeeded. Twelve were denied.
The twelve denials were all against customer_pii_restricted and
hr_people_analytics. Zero denials in the preceding 393 days. A scheduled export
job does not repeatedly probe datasets it has no permission on. Somebody was mapping what this
credential could reach, and least privilege was holding at precisely the two datasets that
mattered most.
The one restricted-adjacent dataset the credential could read, finance_close.gl_detail,
it read. Six times.
Both source addresses geolocated to AS24940, Hetzner, Germany. No threat-intel records on either. The Worker declined to raise confidence on that and said why: a clean reputation on commodity hosting rented last week carries almost no information. The signal was the deviation from 399 days of baseline, not the address.
Data Access logging was off. So the honest answer is “I don’t know.”
Before writing any conclusion about data movement, the Worker checked what the source could actually see. Cloud Storage had Admin Activity logs and no Data Access logs, which is the GCP default: Data Access audit logs are opt-in per service, and in this project they had been turned on for BigQuery only.
So the nightly write to the export bucket is visible, and any read of that bucket is not. This account has written an export every night for eight days while under third-party control, and whether anything read it cannot be determined, and cannot be determined retroactively.
That is not softened into a low-likelihood judgement anywhere in the report. It is recorded as unanswerable, alongside what would change the answer.
- Open · unanswerable from connected sourcesHow the 2023 key was obtained. The change record puts it in a CI variable on the
insights-v2pipeline. That system is not in the mesh. - Open · not loggedWhether the export bucket was read. Data Access logging was off for Cloud Storage. Turning it on fixes the future, not the past.
- Open · not connectedWhat else
tf-modules/bq-exportbuilt. If the module ran more than once, there are sibling accounts in the same condition and nobody has looked. - Open · outside retention870 days of this account’s life sit before the log window opens. The baseline is strong and it is 399 days long, not 1,269.
The receipts
Seventeen queries. Every one of them shown, including the ones that returned nothing.
Q02Find the creation event0 rows
Q03What is normal for this account1,228 events
Q05Which credential did whatsplit is not clean
keys/a3f19c4e · 46.4.118.207 · 1
keys/7d2b0491 · 46.4.118.207 · 18
keys/7d2b0491 · 46.4.118.211 · 18
Q06Isolate the one production-key event that was not the pipelinethe case turns here
Q08Ask the systems where humans write things down3 sources answered
Q13The failures12 denials, all in 6 days
bigquery.tables.list deniedhr_people_analytics · 6 ·
bigquery.tables.list deniedZero failures in the preceding 393 days.
Q15Check what the source can actually see, before concluding anything about itgap found
Q17Directed by senior review · pivot on the source networkchanged the scope
Thirteen months of nothing, then ten days.
Contain quietly. In this order, for these reasons.
| # | Action | Why this position in the sequence |
|---|---|---|
| 1 | Enable Data Access audit logs for Cloud Storage and Secret Manager | Watch before you contain. Without this, step 4 is blind and its tripwires cannot fire. |
| 2 | Deny bigquery.tables.getData outside insights_v2 | Caps the reachable blast radius. Changes nothing the nightly job does. |
| 3 | Revoke key 7d2b0491 only | Costs the operator their working credential. Reads as an ordinary key rotation. |
| 4 | Leave key a3f19c4e live, 72 hours, watched | HUMAN DECISION Known compromised, and knowingly so. |
| 5 | Export the full 400-day partition | The oldest rows expire in 24 hours and they are evidence. |
| 6 | Rotate a3f19c4e with the owning pipeline, after the watch closes | Needs a CI change, and there is no owning team yet. |
Step 4 is the one that matters. Leaving a demonstrably compromised credential active for 72 hours has a real downside, taken to preserve visibility and to avoid breaking the finance close. The Worker does not make that call. It assembles the evidence, states the trade in the open, attaches five named tripwires that end the watch immediately, and hands it to the on-call incident lead.
Senior analyst review
Auto-triggered on the HIGH verdict. Four findings. Severity raised, scope widened, containment sequence amended, verdict upheld.
The report describes a 400-day baseline. The account is 1,269 days old and 399 days of it are observable. The Worker established that ceiling in its second query, wrote it down correctly, and then spent it anyway.
It probably is representative. A job that ran identically on 397 of 399 nights was very likely doing the same thing in 2024. But “very likely” is a different claim from “did,” and 870 unobserved days is enough room for an earlier compromise. Required: bound every baseline claim at 2025-08-02.
The Worker scored HIGH partly on “no data movement observed.” Its own coverage check established why nothing was observed: reads against the export bucket produce no records at all. It stated that correctly in one section and let the reassurance leak into the score in another.
Severity raised to CRITICAL. Not because exfiltration is confirmed, but because the only reason it is neither confirmed nor excluded is that nobody was logging it. Uncertainty of this shape raises severity. It does not lower it.
The Worker enriched both Hetzner addresses, drew the right conclusion about reputation data being uninformative, and stopped. It never asked whether that infrastructure had touched anything else in the estate.
Directed re-query found one failed sign-in from the same network against the disabled account of the service account’s original owner, two days before the key was minted. That reframes the case from opportunism to targeting, and it means the operator had a source of information about who owned what.
The sequencing is right and ungated. If the pipeline team is found and briefed during the watch, the natural thing for them to do is rotate the production key immediately, which lands both revocations in the same window and tells the operator exactly what happened.
Required: gate step 6 on the watch closing, and brief the pipeline team on the constraint before handing them the rotation.
Three of the four findings are the same habit. The Worker identified a limit in its own evidence, recorded it accurately, and then reasoned past it. The gaps were never hidden. They just stopped being load-bearing once they had been acknowledged, which is the failure mode a review exists to catch.
Source completeness
| Source | Status | What it contributed |
|---|---|---|
| GCP Cloud Audit (BigQuery) | HIT | 1,228 events. Timeline, baseline, drift, key attribution |
| ServiceNow | HIT | Owner, provenance, terraform module, key location |
| Google Workspace Directory | HIT | Owner disabled, owning group empty |
| Cribl Audit Logs | HIT | Two downstream routes. The reason not to disable |
| Entra sign-in | HIT | The reconnaissance attempt. Found only at review |
| AWS CloudTrail ×2 | CLEAN NEG | 0 rows. Bounds the blast radius to one cloud |
| Geolocate IPs · Shodan | HIT | AS24940, Hetzner, DE. No exposed services |
| AlienVault OTX | NO DATA | Treated as uninformative, not as exculpatory |
| GCS Data Access logs | GAP | Not enabled in the customer’s project. Blocks the exfiltration question permanently |
| Terraform state | GAP | Not connected. Cannot find sibling accounts from the same module |
| Source control | GAP | Not connected. Cannot trace key references |
| CI/CD | GAP | Not connected. Cannot answer where the key leaked from |
Four gaps. Three are missing connections and one is a logging setting that was off in the customer’s own project. All four appear in the report as named limits, not as inferences.
What this case teaches the next cleanup ticket.
Nothing alerted, and nothing was going to.
No detection covers a service account behaving unlike itself, because nothing had ever established what itself meant. Eight days of a stolen credential enumerating production datasets produced no alert at all. The compromise surfaced because the Worker builds a baseline before it recommends an action, and the baseline is what turned a routine identity sweep into a confirmed threat.
Ownership is never in the cloud that hosts the account.
GCP knew nothing about who owned this identity. The ticketing system, the directory, and the pipeline audit log knew all of it between them. The point is not that Query queried four systems. It is that asking them cost one query and eleven minutes rather than twenty Slack threads.
Know what your logs cannot see before you conclude anything.
Data Access logging was off for Cloud Storage, so the exfiltration question has no answer and never will for these eight days. That gets stated. An investigation that quietly converts a logging gap into a reassuring finding is worse than one that stops short.
The hard call goes to a person, with the trade in the open.
Leaving a compromised credential live for 72 hours is a real risk taken for a real reason. The Worker assembled everything needed to decide it, wrote down what it costs if it is wrong, attached the tripwires, and handed it over. That is the division of labour, and it does not change when the Worker is confident.
Three claims, made on the Workers page, tested here.
Five of the six findings came from outside GCP: the owner, the offboarding, the dead team, the downstream dependency, and the reconnaissance attempt. A GCP-only investigation reaches the same drift signal and none of the context that turns it into a decision.
Four sources could not answer, and each one is named in the report with what it would have told us. The Worker did not fill an unlit corner with a plausible guess, and the reviewer’s job was catching the three places where it nearly did.
Seventeen queries, thirteen sources, eleven minutes, and exactly one decision left open. Not because the Worker was uncertain about the facts, but because the trade it was recommending belongs to somebody accountable for the finance close.
How many unowned service accounts are running in your estate tonight?
Bring your cloud, your ticketing system, and your directory. We will show you what a baseline looks like before anyone touches anything.
