ingest(databases): online schema changes + autovacuum/wraparound#5
Open
choiyounggi wants to merge 1 commit into
Open
ingest(databases): online schema changes + autovacuum/wraparound#5choiyounggi wants to merge 1 commit into
choiyounggi wants to merge 1 commit into
Conversation
…m Postgres survival guide) Two gaps the databases domain didn't cover, both production-severe for a write-heavy startup, both cross-checked against the official PostgreSQL docs: - schema-design/online-schema-changes — avoiding the ACCESS EXCLUSIVE outage: non-volatile DEFAULT fast path, ADD CONSTRAINT NOT VALID + VALIDATE (SHARE UPDATE EXCLUSIVE), the CHECK-proves-NOT-NULL trick, CREATE INDEX CONCURRENTLY, ADD FOREIGN KEY's lighter lock, lock_timeout for lock-queue pileups, and expand-and-contract to decouple DB migration from app deploy. - operations/autovacuum-and-wraparound (new category) — per-table scale_factor/ cost_limit tuning for hot tables, age(datfrozenxid)/relfrozenxid + n_dead_tup monitoring, the ~3M-XID read-only cliff and superuser VACUUM recovery (not FULL/FREEZE), VACUUM FULL vs pg_repack. Registers both in databases/index.md (new operations category), appends log.md, and adds two-way related links from column-data-types and index-write-cost. Source: Hatchet "Postgres survival guide" + postgresql.org docs (sql-altertable, sql-createindex, routine-vacuuming). Independently fact-checked against the official docs; one default corrected (autovacuum_vacuum_scale_factor 0.2, not 0.1).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds two
databases-domain wiki pages, derived from the Hatchet "Postgres survival guide" and cross-checked against the official PostgreSQL docs:schema-design/online-schema-changes— applyingALTER TABLE/CREATE INDEXto a live table without theACCESS EXCLUSIVEoutage: the non-volatileDEFAULTfast path,ADD CONSTRAINT ... NOT VALID+VALIDATE(onlySHARE UPDATE EXCLUSIVE), theCHECK-proves-NOT NULLtrick,CREATE INDEX CONCURRENTLY,ADD FOREIGN KEY's lighter lock,lock_timeoutfor lock-queue pileups, and expand-and-contract to decouple the DB migration from the app deploy.operations/autovacuum-and-wraparound(newoperationscategory) — keeping autovacuum ahead of a hot table and off the wraparound cliff: per-tablescale_factor/cost_limittuning,age(datfrozenxid)/relfrozenxid+n_dead_tupmonitoring, the ~3M-XID read-only cliff and superuserVACUUMrecovery (notFULL/FREEZE), andVACUUM FULLvspg_repack.Plumbing: registers both in
databases/index.md(newoperationscategory), appendslog.md, and adds two-wayrelated:links fromcolumn-data-typesandindex-write-cost.Decision Log
grepof the domain confirmed absent (not valid/expand and contract0 hits;vacuumonly mentioned as index-bloat/visibility side notes, no wraparound/autovacuum-tuning page). Both are production-severe for write-heavy workloads.schema-designcategory (adjacent tocolumn-data-types's live type-change note). Autovacuum/wraparound is operational maintenance, which no existing category (indexing/query-optimization/schema-design/transactions/sqlite) owns → newoperationscategory, per the ingest "create a category only if none fits" rule.index-write-cost(that page is about index write amplification, a different trigger — would violate one-case-per-page); duplicating SKIP-LOCKED / CONCURRENTLY content already covered intransactions/isolation-level-selectionandindexing/index-write-cost(linked instead).sql-altertable,sql-createindex,routine-vacuuming); blog-specific numbers ("10×", "90%") were excluded as unverifiable.Cross-Check: independently fact-checked against the official PostgreSQL docs by a separate agent — 8 of 9 load-bearing claims CONFIRMED verbatim (ADD FOREIGN KEY SHARE ROW EXCLUSIVE, VALIDATE SHARE UPDATE EXCLUSIVE, non-volatile default fast path, CHECK→SET NOT NULL scan skip, binary-coercible type change, ~3M-XID cliff, per-table storage-param syntax, CONCURRENTLY semantics). One error caught and fixed pre-merge:
autovacuum_vacuum_scale_factordefault is 0.2, not 0.1 (and the derived 1M-row example → ~200k dead tuples, not ~100k).Verification
related:ids and inline[page-id]refs resolve to existing pages.databases/index.mdwithload whenlines;log.mdentry appended.