Check for Anomalous GCP Access After the Railway Incident
How to check for anomalous GCP access after the Railway incident: disable and rotate service account keys, Cloud Audit Logs filters, IAM and billing checks.
If you stored a Google Cloud service account key in Railway, treat it as exposed after Railway's October 21, 2026 security disclosure. Create a new key, disable the old one with gcloud (the console cannot disable keys), update Railway, then search Cloud Audit Logs for everything that service account did during the exposure window. Look for new service accounts, keys, IAM bindings, Compute Engine instances, storage buckets and calls from unfamiliar IP addresses, and compare billing with the previous month.
This guide follows Railway's published GCP guidance [1], adds a few extra commands from Google Cloud documentation, and explains what each step is for. It is for checking your own projects. For background, read our Railway security incident explainer.
Key takeaways
- Railway's scope: the guide applies to a service account key used with Railway, such as one stored in
GOOGLE_APPLICATION_CREDENTIALS_JSON[1]. - Service account keys do not expire by default, so an exposed key keeps working until you disable it [1].
- Disabling a key is not instant. Tokens already issued can stay valid for up to an hour [1].
- Admin Activity logs are always on. Data Access logs are off by default, except for BigQuery, so some activity may not be recorded [1].
- Check for attacker footprints: new accounts, keys, IAM bindings, instances, buckets and SSH keys.
Before you start
Gather:
- The service account email and the key ID of every Google Cloud key stored in Railway.
- The project ID(s) the service account can access.
- An exposure window, from the earliest time the key could have been readable until you rotated it. Railway's guide does not give a fixed length, so use a conservative one.
- The
gcloudCLI authenticated as a separate admin identity, not the exposed key.
Placeholders used below: SERVICE_ACCOUNT_EMAIL, KEY_ID, PROJECT_ID.
Step 1: Create a replacement key
In the Google Cloud console, open Service accounts, select the account, go to the Keys tab and choose Add key, then Create new key. The JSON file downloads once [1]. Store it securely.
You can also list keys from the CLI to find the key ID of the exposed one:
bashgcloud iam service-accounts keys list \
--iam-account=SERVICE_ACCOUNT_EMAIL --project=PROJECT_ID
Step 2: Disable the exposed key
Railway notes that disabling a key can only be done from the CLI, since the console cannot do it [1]:
bashgcloud iam service-accounts keys disable KEY_ID \
--iam-account=SERVICE_ACCOUNT_EMAIL \
--project=PROJECT_ID
Important caveats from Railway's guide [1]:
- Disabling blocks new authentication, but tokens already issued from the key remain valid for up to an hour.
- To cut access immediately, disable the whole service account or apply an IAM deny policy. Every workload that uses that account then loses access, including your Railway service until it has the new key.
- Wait at least 60 minutes before re-enabling the account.
Also review the account's IAM roles and remove any you do not recognise, especially Service Account Token Creator on other accounts [1].
Step 3: Put the new key into Railway
- Edit the variable in the service's Variables tab. If it is a shared variable, edit it once in Project Settings [1].
- Paste the new JSON as it is, since multiline values are supported, then deploy the staged change [1].
- Confirm the deployment is active and its logs show successful Google API calls, with no
invalid_grantorPERMISSION_DENIEDerrors [1]. - Update anything outside Railway that used the old key, such as local development or CI [1].
Delete the old key only after the new deployment is running cleanly and your review below is finished [1]:
bashgcloud iam service-accounts keys delete KEY_ID \
--iam-account=SERVICE_ACCOUNT_EMAIL
Step 4: Search Cloud Audit Logs
Open Logs Explorer, set the time range to your exposure window and run the query Railway suggests [1]:
logName:"cloudaudit.googleapis.com"
protoPayload.authenticationInfo.principalEmail="SERVICE_ACCOUNT_EMAIL"
To catch impersonation, where the service account was used to act as another account, filter on [1]:
protoPayload.authenticationInfo.serviceAccountDelegationInfo.firstPartyPrincipal.principalEmail="SERVICE_ACCOUNT_EMAIL"
You can run a similar search from the command line:
bashgcloud logging read \
'protoPayload.authenticationInfo.principalEmail="SERVICE_ACCOUNT_EMAIL"' \
--project=PROJECT_ID --freshness=30d --limit=200
If you use Security Command Center, check its Findings with the filter [1]:
access.principal_email="SERVICE_ACCOUNT_EMAIL"
What the logs may not show
Admin Activity audit logs are always on, but Data Access logs are off by default, except for BigQuery [1]. That means reads of your data may not be recorded. Absence of evidence is not proof nothing was read, so enable Data Access logs for sensitive services going forward.
Step 5: Know what suspicious looks like
Railway's guide lists these things to check for [1]:
| Look for | Why it matters |
|---|---|
| New service accounts or new keys | Persistence after you rotate |
| New IAM bindings | Privilege escalation |
| New Compute Engine instances | Compute abuse such as mining |
| New Cloud Storage buckets | Data staging or exfiltration |
| New project-level SSH keys | A way to log in to instances |
| Calls from unfamiliar IP addresses | Access from outside your app |
On IP addresses, Railway notes that if you use its static outbound IPs, any other source IP points to an attacker. Without them, your services share Railway egress IPs, so an unfamiliar IP is only a lead to investigate, not proof [1].
To see when the key last authenticated:
bashgcloud policy-intelligence query-activity \
--activity-type=serviceAccountKeyLastAuthentication --project=PROJECT_ID
Step 6: Check billing
Compare the exposure window with the previous month, by service and SKU. Railway names Compute, GPU and Vertex AI as common targets for abuse [1]. Set budgets and alerts in Cloud Billing so a spike is flagged quickly next time.
Step 7: Clean up carefully
- Keep copies of anything you may need for forensics first [1].
- Delete attacker-created resources, service accounts, keys and IAM bindings [1].
- Rotate any other credentials the account could reach.
- Contact Cloud Customer Care for help with logs or charges [1].
Step 8: Reduce the risk next time
- Avoid long-lived JSON keys where possible. Prefer attached service accounts or workload identity, which do not need a downloadable key.
- Least privilege. Give the account only the roles it needs.
- Separate service accounts per service and environment.
- Enable Data Access logs for sensitive data.
- Budgets, alerts and key-age monitoring.
See also AWS vs GCP vs Azure and our AWS equivalent of this guide. For the wider picture, read incident management and postmortems and observability: logs, metrics and traces.
Need a hand reviewing your cloud setup?
We help teams audit cloud access, remove long-lived keys and set up monitoring. Get a scope estimate for a cloud security review.
FAQ
How do I check for anomalous GCP access after the Railway incident?
Disable and replace the exposed service account key, then search Cloud Audit Logs in Logs Explorer for the service account's principal email over the exposure window, and check IAM, resources, IP addresses and billing for anything you did not create.
Can I disable a service account key in the console?
Railway says disabling a key can only be done from the CLI, using gcloud iam service-accounts keys disable.
Does disabling a key stop access immediately?
Not fully. Tokens already issued from the key can remain valid for up to an hour. To cut access immediately, disable the whole service account or apply an IAM deny policy.
Do service account keys expire?
Not by default, so an exposed key works until it is disabled or deleted.
Why might my logs show nothing suspicious?
Admin Activity logs are always on, but Data Access logs are off by default except for BigQuery, so some activity, such as data reads, may not have been recorded.
What resources do attackers usually create?
Railway's guide lists new service accounts and keys, IAM bindings, Compute Engine instances, Cloud Storage buckets and project-level SSH keys, with Compute, GPU and Vertex AI as common billing targets.
Should I delete the old key?
Yes, but only after the new key is working in production and your log review is complete.
References
- Railway Blog, Mykal Machon. "Security Disclosure Recommendation: Check for Anomalous GCP Access." 22 October 2026. blog.railway.com/p/security-disclosure-recommendation-check-for-anomalous-gcp-access
- Railway Blog. "Security Disclosure: Wednesday October 21st." 21 October 2026. blog.railway.com/p/security-disclosure-wednesday-october-21st
- Google Cloud Documentation: Cloud Audit Logs and Service account keys.
Disclaimer: independent, unofficial guidance for checking your own projects, not affiliated with Railway or Google Cloud. Verify commands against current Google Cloud documentation, and contact Google Cloud Customer Care or a security professional for serious incidents.