Skip to content

Refresh Staging From Production Data

How to reload staging.wippidu.net with a current snapshot of the production database — anonymized by default.

This workflow exists because synthetic test data on staging only goes so far when reproducing real bugs or trying out a new feature against realistic shapes of data. The full-database import/export (#459) lets an admin take a dump from production, upload it to staging, and have all PII scrubbed during the load.

Prerequisites

  • Admin role on both instances (production and staging).
  • Staging started with the env var APP_ALLOW_PROD_IMPORT=true. Production must not have this set — that is what keeps the import button off the production UI.
  • A browser session for each.

Step 1 — Export from production

  1. Log in to app.wippidu.net as admin.
  2. Open the gear menu → Datenmanagement → Import/Export.
  3. Leave Include attachments ticked unless you specifically want a smaller dump without binaries.
  4. Click Download export. The browser saves a file named wippidu-export-YYYYMMDD-HHMMSS.json.gz.

Production never offers an import button on this page, so there's no risk of confusing the two halves of the workflow on the same screen.

Step 2 — Import into staging

  1. Log in to staging.wippidu.net as admin.
  2. Open the same Import/Export page. The page now also has an Import section (gated by APP_ALLOW_PROD_IMPORT).
  3. Anonymization checkboxes are all ticked by default:
  4. Replace email addresses → user-<id>@staging.local
  5. Replace names → deterministic fake German names
  6. Blank contact details (address, phone)
  7. Reset passwords → all set to staging123
  8. Scrub message and parental-letter bodies → [scrubbed for staging]
  9. Replace attachments with a 1×1 placeholder PNG (filename and MIME type preserved)

Leave them all on for the normal refresh path. To opt out of any, you must type I_UNDERSTAND_THE_PII_RISK into the risk-acknowledgement field. The server enforces this independent of the form.

  1. Hostname confirmation: type the staging hostname (e.g. staging.wippidu.net) into the confirmation field. The server compares against Request.Host, so a tampered form won't bypass it.

  2. Choose the wippidu-export-…json.gz file and click Start import. The browser asks once more to confirm.

Step 3 — Verify

The page redirects to itself with a success banner and a result panel showing:

  • Source host and original export timestamp
  • Rows imported per table
  • Which anonymizers ran (and which were skipped, in bold)
  • Skipped/missing tables, if the schemas don't fully align
  • A reminder of the staging password (staging123) when password hashes were reset

The home page now also shows a yellow "staging data" banner at the top, visible to every logged-in user — so anyone bumping into staging immediately knows they're not looking at production data.

Behavior

  • Transactional. The whole import runs in a single GORM transaction. Any failure rolls back to the prior state; the success path commits atomically.
  • IDs preserved. Foreign keys in the dump just work after import. Postgres sequences are reset past MAX(id) automatically so subsequent inserts on staging don't collide.
  • Soft-deletes survive. Rows soft-deleted on production are still soft-deleted on staging.
  • Audit log is local. The ImportExportAudit table is excluded from both export and import: staging keeps its own running history of refreshes across reloads.
  • Schema tolerance. If production is on a newer release with extra tables, those are listed as skipped and ignored. If staging expects tables the dump doesn't have, they're listed as missing and left empty.

Logging into staging after a refresh

If you ticked the password-reset anonymizer (the default), every account on staging now logs in with staging123. The original bcrypt hashes from production never reach staging in this mode.

If you opted out (with the risk acknowledgement), production password hashes were imported intact. Don't do this casually — production password material on a less-locked-down host is exactly what the scrubber exists to prevent.

Troubleshooting

Symptom Likely cause
"Import is not enabled on this instance." APP_ALLOW_PROD_IMPORT is unset on the target. Set it and restart.
"The hostname you entered does not match this instance." Typo in the confirmation field, or you're on the wrong instance. Re-read the page URL and the expected value shown next to the field.
"Dump schema version N is newer than this instance supports." Production is running a release ahead of staging. Update staging first, then retry.
Audit row shows failure with no other detail Check application logs. The full error message is also stored in the audit row's ErrorMessage column.