Set the variables before you create the volume
Postgres reads its password once, when the data directory is empty. The order of operations is configuration.
The problem
Production on Railway, about 15 hours after the first pull request. A Postgres service, a volume, the variables. Every migration (dbmate up) died with 28P01: password authentication failed.
What happened
- On Railway, creating a volume deploys the service within the second.
- The
postgresimage appliesPOSTGRES_PASSWORDonly while it initialises an empty data directory. - We created the volume first and set the variables after.
The database was born with the password the service had at that second, which was staging’s. Changing the variable later touched nothing: the cluster already existed.
The fix
On an empty database: delete the volume and create it again, with the variables already set. The deploy log proves the cluster is fresh: PG_VERSION=missing.
Two more things learned on the way:
- A deleted volume lingers as
isPendingDeletion, andenvironment { volumeInstances }keeps returning it.project { volumes }shows which one is real. - Neither environment had a repository connected, so the dashboard’s Deploy button had nothing to build. Every release was
railway upfrom a working copy. That became the first line of “deploy on merge”.
Takeaway
Any image that initialises on first start (Postgres, MySQL, MongoDB) reads its credentials once. Variables first, storage second. Write the order down: it is the step nobody re-reads.