Railway says environment variables may have been decrypted: what to rotate while builds are paused

If you deploy on Railway, treat every secret stored there as exposed. That means API keys (including AI provider keys), database URLs and private registry credentials. Revoke them at the provider that issued them, then check your logs for unusual use. Railway’s security disclosure says an unauthorized third party “may have accessed encrypted environment variables and, for some of them, obtained the decrypted contents.” Railway recommends that all users rotate, not only the ones it later contacts.

Builder question this answers: My app runs on Railway. What exactly do I rotate, and how do I do it when I can’t redeploy?

What Railway has confirmed

From the October 21 blog post and the updates on status.railway.com:

• Railway detected anomalous activity on systems that store customer environment variables. Its investigation so far indicates that some variables were decrypted.
• Railway believes the attack is “part of a series of attacks recently targeting platform companies with the aim of compromising AI inference providers.” It says it has asked those upstream providers to revoke all keys proactively. So AI keys stored in Railway may stop working even if you do nothing.
• Railway thinks the impact may be limited to a subset of users. It will email affected customers directly as it confirms them, but it still asks everyone to rotate.
• All deployments are paused, including for Enterprise workspaces. Until builds come back, you can’t roll out new versions.
• Railway is rotating all storage bucket credentials itself. The new credentials appear in your dashboard automatically, but services can’t pick them up until redeploys are possible. In audit logs you may see these revocations attributed to the Railway employee “Faraz Patankar”. Railway says that is expected.
• OAuth sessions were invalidated as part of the mitigation, so you may be signed out and need to log in again.

At the time of writing, the latest status update (October 22, 00:22 UTC) said Railway was still working to bring builds back. It also said it would publish guides on the recommended next steps. The Hacker News thread on the disclosure passed 270 points within hours.

What to do now

1. Revoke at the source first. Rotating a value inside Railway does nothing if the old key still works. Go to each issuer: your LLM provider dashboards, Stripe, cloud IAM, the registry you pull private images from, and your database. Create new credentials there and revoke the old ones.

2. Decide on the downtime trade-off. Your services can’t redeploy right now, so they keep running with the old values. If you revoke a key a running service depends on, that service breaks until builds resume. Railway’s advice is to roll immediately. For high-value keys (cloud admin, payment, AI keys with large spend limits), accepting the outage is usually the safer choice. This is the same situation we covered for hosts that ran the compromised LiteLLM 1.107.0 release yesterday: assume the secret is gone, and rotate it.

3. Stage the new values with the CLI. The Railway CLI variable docs list a --skip-deploys flag for set, and a --stdin option that keeps the secret out of your shell history:

# See what is stored for a service in a given environment
railway variable list --service api --environment production --kv

# Set a new value from stdin without triggering a deploy
printf '%s' "$NEW_KEY" | railway variable set OPENAI_API_KEY --stdin --skip-deploys --service api --environment production

Run railway variable list --kv only on a trusted machine, because it prints secrets in plain text. Do this for every service and every environment. Staging and preview environments often hold the same production keys.

4. Review logs and audit trails. Railway asks you to review service logs for unusual activity. Also check the usage pages at your providers (unexpected token spend is an early sign that an AI key is being abused). Workspace admins can open Railway’s audit logs, which record changes to variables, services and deployments.

5. Watch the status page. Railway says status.railway.com is the fastest place for updates. Plan a redeploy as soon as builds resume, so services pick up both your new variables and the rotated bucket credentials.

Why this matters beyond Railway

This is the second incident this week where the target was AI provider credentials sitting in developer infrastructure. If your keys are spread across several platforms, now is a good time to list where each one lives and give each deployment its own scoped key, so a single leak can be revoked without taking everything down. For continuous checks on your own code, see our guide to setting up Claude Security continuous scanning. More builder news is on the 7minai homepage.