Skip to content

ingest(databases): online schema changes + autovacuum/wraparound#5

Open
choiyounggi wants to merge 1 commit into
mainfrom
wiki/postgres-migration-and-vacuum
Open

ingest(databases): online schema changes + autovacuum/wraparound#5
choiyounggi wants to merge 1 commit into
mainfrom
wiki/postgres-migration-and-vacuum

Conversation

@choiyounggi

Copy link
Copy Markdown
Owner

What

Adds two databases-domain wiki pages, derived from the Hatchet "Postgres survival guide" and cross-checked against the official PostgreSQL docs:

  • schema-design/online-schema-changes — applying ALTER TABLE / CREATE INDEX to a live table without the ACCESS EXCLUSIVE outage: the non-volatile DEFAULT fast path, ADD CONSTRAINT ... NOT VALID + VALIDATE (only SHARE UPDATE EXCLUSIVE), the CHECK-proves-NOT NULL trick, CREATE INDEX CONCURRENTLY, ADD FOREIGN KEY's lighter lock, lock_timeout for lock-queue pileups, and expand-and-contract to decouple the DB migration from the app deploy.
  • operations/autovacuum-and-wraparound (new operations category) — keeping autovacuum ahead of a hot table and off the wraparound cliff: per-table scale_factor/cost_limit tuning, age(datfrozenxid)/relfrozenxid + n_dead_tup monitoring, the ~3M-XID read-only cliff and superuser VACUUM recovery (not FULL/FREEZE), and VACUUM FULL vs pg_repack.

Plumbing: registers both in databases/index.md (new operations category), appends log.md, and adds two-way related: links from column-data-types and index-write-cost.

Decision Log

  • Intent: fill two databases gaps a grep of the domain confirmed absent (not valid/expand and contract 0 hits; vacuum only mentioned as index-bloat/visibility side notes, no wraparound/autovacuum-tuning page). Both are production-severe for write-heavy workloads.
  • Routing: online DDL → existing schema-design category (adjacent to column-data-types's live type-change note). Autovacuum/wraparound is operational maintenance, which no existing category (indexing/query-optimization/schema-design/transactions/sqlite) owns → new operations category, per the ingest "create a category only if none fits" rule.
  • Rejected: folding autovacuum into index-write-cost (that page is about index write amplification, a different trigger — would violate one-case-per-page); duplicating SKIP-LOCKED / CONCURRENTLY content already covered in transactions/isolation-level-selection and indexing/index-write-cost (linked instead).
  • Sourcing: every claim cites postgresql.org (sql-altertable, sql-createindex, routine-vacuuming); blog-specific numbers ("10×", "90%") were excluded as unverifiable.

Cross-Check: independently fact-checked against the official PostgreSQL docs by a separate agent — 8 of 9 load-bearing claims CONFIRMED verbatim (ADD FOREIGN KEY SHARE ROW EXCLUSIVE, VALIDATE SHARE UPDATE EXCLUSIVE, non-volatile default fast path, CHECK→SET NOT NULL scan skip, binary-coercible type change, ~3M-XID cliff, per-table storage-param syntax, CONCURRENTLY semantics). One error caught and fixed pre-merge: autovacuum_vacuum_scale_factor default is 0.2, not 0.1 (and the derived 1M-row example → ~200k dead tuples, not ~100k).

Verification

  • All related: ids and inline [page-id] refs resolve to existing pages.
  • Both new pages registered in databases/index.md with load when lines; log.md entry appended.
  • Body ≤120 lines each; no banned vague qualifiers in directives.

…m Postgres survival guide)

Two gaps the databases domain didn't cover, both production-severe for a
write-heavy startup, both cross-checked against the official PostgreSQL docs:

- schema-design/online-schema-changes — avoiding the ACCESS EXCLUSIVE outage:
  non-volatile DEFAULT fast path, ADD CONSTRAINT NOT VALID + VALIDATE (SHARE
  UPDATE EXCLUSIVE), the CHECK-proves-NOT-NULL trick, CREATE INDEX CONCURRENTLY,
  ADD FOREIGN KEY's lighter lock, lock_timeout for lock-queue pileups, and
  expand-and-contract to decouple DB migration from app deploy.
- operations/autovacuum-and-wraparound (new category) — per-table scale_factor/
  cost_limit tuning for hot tables, age(datfrozenxid)/relfrozenxid + n_dead_tup
  monitoring, the ~3M-XID read-only cliff and superuser VACUUM recovery (not
  FULL/FREEZE), VACUUM FULL vs pg_repack.

Registers both in databases/index.md (new operations category), appends log.md,
and adds two-way related links from column-data-types and index-write-cost.

Source: Hatchet "Postgres survival guide" + postgresql.org docs (sql-altertable,
sql-createindex, routine-vacuuming). Independently fact-checked against the
official docs; one default corrected (autovacuum_vacuum_scale_factor 0.2, not 0.1).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant