Adopting Payload migrations without nuking production

I shipped a new feature and Vercel turned the build red:

error: relation "series" does not exist

The code was fine. The build failed because of where Postgres was in its life, not because of anything I'd written. This is the story of getting a live Payload + Postgres app off dev "push" mode and onto tracked migrations, and every trap I hit on the way. There were more than I expected, and a couple could have taken production down if I'd been less lucky.

What push mode is, and why it felt fine for so long

Payload's dev workflow uses drizzle's push: it diffs your config against the database and syncs the schema on the fly. No migration files and no ledger. You change a collection, the tables reshape themselves, and you keep moving. For local iteration it's great.

The problem is how that habit reaches production. This app spent its early life on push. The schema was auto-synced in dev and pushed to prod in bulk, which means prod's tables were never created by a migration runner. Postgres has all the tables (users, posts, media, and the rest), but Payload's payload_migrations bookkeeping table has no real history in it. Just a single marker row that push leaves behind, named dev, batch -1.

None of that hurt as long as I only touched existing tables. Add a field here, widen a column there, push handles it and the site keeps serving. The cliff is a brand-new table that gets read at build time. That's the day push stops being invisible.

Payload's own docs say two things I'd been half-ignoring: use payload migrate && next build as your build command, and don't mix push and migrate. I was about to learn why, in order.

Trap 1: the build prerenders against production

Here's the mechanism that produced that first red build.

A Vercel production deploy runs next build, and next build prerenders pages. Prerendering runs your data-fetching code, which queries Postgres. So when I added a Series collection and shipped a page that read from the series table, the build tried to prerender that page against the production database, where the table didn't exist yet, and got relation "series" does not exist.

Nothing was wrong with the feature. The failure was purely ordering: the code that reads a table shipped before the table existed in prod. Push had been hiding this the whole time, because in dev push had already synced the table locally. The one environment where you can't casually push, production at build time, is exactly where it surfaced.

The fix isn't "create the table faster." It's to make schema changes land in prod before the code that depends on them. Which is the entire reason to adopt migrations. So I did.

Trap 2: a naive payload migrate tries to recreate tables that already exist

The obvious next move is to run payload migrate against prod. Don't, not yet.

Remember that prod has no real migration history, only that dev marker. So when the migration runner looks at prod, it sees zero applied migrations, concludes it must start from the very first one, and runs the initial-schema migration: CREATE TABLE "users", CREATE TABLE "posts", and so on. Against tables that already exist. It errors out before it ever reaches the migration I actually wanted to apply.

The migration ledger and the real schema disagree. The schema is ahead; the ledger is empty. To fix that, you don't run DDL. You baseline: you tell Payload the initial-schema migration is already applied, without touching any tables. It's a bookkeeping insert.

INSERT INTO payload_migrations (name, batch)
SELECT '20260731_211106_initial_schema', 1
WHERE EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name='users')
  AND NOT EXISTS (SELECT 1 FROM payload_migrations WHERE name='20260731_211106_initial_schema');

Now payload migrate skips the initial schema and runs only the new migrations.

Two things make this safe to run against production more than once. The WHERE clause guards it, so it only fires when the app tables are present and the initial-schema migration isn't already recorded. Once prod is tracked it's a harmless no-op. And notice what it keys on: "initial_schema not recorded," not "payload_migrations is empty." That distinction matters, because push left that dev marker behind. The table is rarely actually empty, so an emptiness check would be wrong. That marker turned out to matter in a second, worse way too.

Trap 3: the interactive prompt that hangs CI

After baselining, I ran the migration in a build. It didn't error. It just hung. No output, no progress, no timeout, a build sitting there forever.

The culprit was that same dev marker. While the push marker is present, Payload treats the database as push-managed and stops on an interactive "data loss will occur" confirmation prompt. Interactive prompt, meet non-interactive CI with no TTY: the prompt waits for a keystroke that will never come, and the build blocks until something kills it.

The fix has two parts. First, remove the marker during baseline, since we're adopting migrations and the push marker should go:

DELETE FROM payload_migrations WHERE name = 'dev' AND batch = -1;

Second, run the migration with stdin closed as a hang guard:

With the marker gone, an additive migration doesn't prompt at all. But if a future one ever does, < /dev/null gives it EOF, so it aborts instead of blocking forever. Both my manual migrate-prod.sh and the Vercel build script use it. It's cheap insurance against the failure mode that's hardest to diagnose, because a hang leaves no error to grep for.

Trap 4: ordering, migrate before or migrate after

Trap 1 was really an ordering bug, and expand/contract is the rule that stops it recurring:

• Additive change (add a table or column): migrate before you deploy the code. The new schema is there when the new code prerenders.
• Destructive change (drop a table or column): migrate after. First deploy code that no longer touches it, then drop it in a follow-up.

Keep each migration additive and backward-compatible so that during a rollout, when old and new code briefly run against the same database, both are happy. This is standard expand/contract, but Payload's build-time prerendering makes the "before" case bite harder than usual, because getting the order wrong doesn't cause a subtle runtime error. It fails the build outright.

Trap 5: the generated down() was wrong

You should be able to roll a migration back. I wrote the Series migration, then tested the down path against a local database. It failed.

Payload generates the down() for you, and the generated one dropped the series table with CASCADE, which already removes the dependent payload_locked_documents_rels_series_fk foreign key, and then tried to drop that same foreign key again. The second drop hit a constraint that no longer existed, and the rollback errored.

The fix is to make the whole down() idempotent with IF EXISTS guards:

ALTER TABLE IF EXISTS "series" DISABLE ROW LEVEL SECURITY;
ALTER TABLE IF EXISTS "series_rels" DISABLE ROW LEVEL SECURITY;
DROP TABLE IF EXISTS "series" CASCADE;
DROP TABLE IF EXISTS "series_rels" CASCADE;
ALTER TABLE "payload_locked_documents_rels" DROP CONSTRAINT IF EXISTS "payload_locked_documents_rels_series_fk";
DROP INDEX IF EXISTS "payload_locked_documents_rels_series_id_idx";
ALTER TABLE "payload_locked_documents_rels" DROP COLUMN IF EXISTS "series_id";

Now the CASCADE can remove the constraint and the explicit drop is a no-op instead of an error.

The lesson generalizes past this one bug: a generated down() is a draft, not gospel. The only way I found the double-drop was by actually running up, then down, then up again against a real database. A rollback you've never executed is a rollback you don't have.

Automating it so future-me can't forget

By this point I had a safe manual path. pnpm db:migrate-prod takes a full backup of prod, prints the migration status, asks for confirmation, baselines if the database is still untracked, evicts the dev marker, and only then migrates. The backup and the migrate are fused into one command specifically so the backup can't be skipped by accident. The safe pairing is the default, not a thing you have to remember.

On top of that I added two opt-in automation layers, both gated behind an environment flag so that merging the tooling changes nothing until I explicitly turn each on:

• Previews migrate their own database branch. With RUN_PREVIEW_MIGRATIONS=true, a preview deploy applies pending migrations to its own Neon branch before building, so a page can prerender against a schema change the PR itself introduces. This path is fail-closed: if the resolved database host is production, or if no branch connection is present at all, the build refuses to run rather than risk migrating prod from a preview.
• Production optionally auto-migrates on deploy. With RUN_PROD_MIGRATIONS=true, a production build takes an instant Neon snapshot branch first and only then migrates. If it can't take the snapshot, the build fails, so prod is never migrated without a restore point sitting right behind it.

Preview branches clone the already-baselined prod, so they inherit a clean migration ledger and no dev marker. The whole thing composes.

The one rule

Where this lands is Payload's recommended build command, payload migrate && next build, and a single rule that ties every trap above together: don't mix push and migrate. Push is a local convenience for throwaway iteration. The moment a schema change is headed for an environment you can't afford to lose, it belongs in a migration.

The short version, if you're staring at your own push-mode prod:

• The build reads production, so schema has to lead code. Think expand/contract.
• Baseline an untracked database with a guarded, idempotent bookkeeping insert, and delete the dev marker while you're there.
• Never trust a generated down() you haven't actually run up-then-down.
• Automate with backups and fail-closed guards, opt-in, so the safe path is the default path.

Push mode is a good way to build and a bad way to deploy. The gap between those two is the set of traps above, and now you know where each one is.