[#856] Backend: Persist write-ahead audit log entries to Prisma for multi-instance durability - #923
Conversation
|
please fix conflict |
…ries to Prisma writeAheadAuditLog.ts (the Issue Junirezz#707 prepare/commit/rollback flow for admin configuration changes) stored entries in a process-local array capped at 10,000 records. Entries were lost on restart and invisible to peer instances in multi-pod deployments, so operators couldn't reconstruct partial config changes after a crash between prepare and commit. Note on this PR: the existing branch/PR for this issue was 847 commits behind current main, and several files it touched (schema.prisma, writeAheadAuditLog.ts itself, check-postgres-drift.js) had since been created or substantially changed by unrelated work upstream. Rather than force a mechanical rebase across that much drift (high risk of silently corrupting the Prisma schema or duplicating models), this reimplements the feature fresh against current main, replacing this branch's history. What changed: - Added a WriteAheadAuditEntry Prisma model (backend/prisma/schema.prisma) and a matching migration (backend/prisma/migrations/20260727000000_add_write_ahead_audit_entries), following the same field shape and index conventions as the existing AdminConfigChange model - Rewrote writeAheadAuditLog.ts: prepare/commit/rollback/getEntry/list/ getPendingEntries/getMetrics/clear are now async and persist via Prisma, following the exact same test-mode/no-DATABASE_URL fallback convention already established by adminConfigChangeAudit.ts (shouldUseInMemoryWalFallback: NODE_ENV=test, JEST_WORKER_ID, or no DATABASE_URL). Unlike that simpler precedent, this store is read back within the same request lifecycle (prepare then commit/rollback by id, then list), so the in-memory fallback is a fully functional store (not just a stub), and any unexpected Prisma error at runtime also falls back to the in-memory path per-call rather than throwing - Added a stalePending count to getMetrics() (pending entries older than a configurable WAL_STALE_PENDING_TTL_MS, default 15 minutes) so operators can spot admin config changes stuck between prepare and commit/rollback after a crash - this is an additive field, so existing consumers of the WAL metrics response shape are unaffected - Updated the 4 GET /admin/wal/* route handlers in index.ts to be async and await the now-async store, with try/catch around each returning a 500 on genuinely unexpected errors (the store itself degrades gracefully on Prisma errors, so this is a backstop). HTTP response shapes are unchanged - Updated the existing Issue Junirezz#707 describe block in issues705-707-719-723.test.ts to await the now-async calls, and added 2 new tests covering the stalePending metric and confirming a stale pending entry can still be rolled back Verification: - npm install succeeds - tsc --noEmit: 0 errors in any changed file. Confirmed against a clean upstream/main checkout: baseline is 95 pre-existing errors, entirely in files this PR does not touch (apiKeyAuth.ts, walletNonce.ts, etc.) - identical count and files before and after this change - eslint: 0 new errors. 1 new instance of an already-established no-non-null-assertion warning pattern in the new stale-pending test, matching the existing style used throughout this same test file (baseline 13 warnings -> 14 with this change, same rule, same file) - node scripts/check-migrations.js: passes (the new migration doesn't match any high-risk pattern the safety scanner checks for) - tsx scripts/check-schema-snapshots.ts: passes (WAL endpoints aren't covered by these API contract snapshots, confirmed by inspecting apiContractSnapshots.ts) - Could not run `prisma generate` or the actual Jest suite: this sandbox's network policy blocks binaries.prisma.sh (confirmed via 403 on every engine-fetch attempt, including with PRISMA_ENGINES_CHECKSUM_IGNORE_MISSING=1 and forcing PRISMA_CLIENT_ENGINE_TYPE=wasm), and no engine binary exists anywhere on this filesystem to substitute. Jest itself fails before any test runs, at PrismaClient construction, with "@prisma/client did not initialize yet. Please run prisma generate" - this happens even though Jest sets NODE_ENV=test and my code would have exercised only the in-memory fallback path, never actually querying Prisma, so it's specifically the client construction step that's blocked here. Please run `prisma generate && npm test` in CI/locally to confirm. Closes Junirezz#856
0b2f507 to
ca0a318
Compare
|
@broda-spendy Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
|
Checked the three failing checks ( Confirmed this is pre-existing and unrelated: the identical three checks are currently failing on Locally, |
Summary
Closes #856.
writeAheadAuditLog.ts(the Issue #707 prepare/commit/rollback flow for admin configuration changes) stored entries in a process-local array capped at 10,000 records. Entries were lost on restart and invisible to peer instances in multi-pod deployments, so operators couldn't reconstruct partial config changes after a crash between prepare and commit.Note on this update: this branch was 847 commits behind current
main, and several files it touched (schema.prisma,writeAheadAuditLog.tsitself,check-postgres-drift.js) had since been created or substantially changed by unrelated work upstream. Rather than force a mechanical rebase across that much drift (real risk of silently corrupting the Prisma schema or duplicating models), I reimplemented the feature fresh against currentmain, replacing this branch's history.What changed
WriteAheadAuditEntryPrisma model (backend/prisma/schema.prisma) and a matching migration (backend/prisma/migrations/20260727000000_add_write_ahead_audit_entries), following the same field shape and index conventions as the existingAdminConfigChangemodelwriteAheadAuditLog.ts:prepare/commit/rollback/getEntry/list/getPendingEntries/getMetrics/clearare now async and persist via Prisma, following the exact same test-mode/no-DATABASE_URLfallback convention already established byadminConfigChangeAudit.ts(NODE_ENV=test,JEST_WORKER_ID, or noDATABASE_URL). Unlike that simpler precedent, this store is read back within the same request lifecycle (prepare then commit/rollback by id, then list), so the in-memory fallback is a fully functional store, not just a stub — and any unexpected Prisma error at runtime also falls back to the in-memory path per-call rather than throwingstalePendingcount togetMetrics()(pending entries older than a configurableWAL_STALE_PENDING_TTL_MS, default 15 minutes) so operators can spot admin config changes stuck between prepare and commit/rollback after a crash — this is an additive field, so existing consumers of the WAL metrics response shape are unaffectedGET /admin/wal/*route handlers inindex.tsto be async and await the now-async store, with try/catch around each returning a 500 on genuinely unexpected errors. HTTP response shapes are unchangedissues705-707-719-723.test.tsto await the now-async calls, and added 2 new tests covering thestalePendingmetric and confirming a stale pending entry can still be rolled backVerification
npm installsucceedstsc --noEmit: 0 errors in any changed file. Confirmed against a cleanupstream/maincheckout: baseline is 95 pre-existing errors, entirely in files this PR doesn't touch (apiKeyAuth.ts,walletNonce.ts, etc.) — identical count and files before and after this changeeslint: 0 new errors. 1 new instance of an already-establishedno-non-null-assertionwarning pattern in the new stale-pending test, matching the existing style throughout this same test file (baseline 13 warnings → 14, same rule, same file)node scripts/check-migrations.js: passestsx scripts/check-schema-snapshots.ts: passes (WAL endpoints aren't covered by these API contract snapshots — confirmed by inspectingapiContractSnapshots.ts)prisma generateor the actual Jest suite: this sandbox's network policy blocksbinaries.prisma.sh(confirmed 403 on every engine-fetch attempt, including withPRISMA_ENGINES_CHECKSUM_IGNORE_MISSING=1and forcingPRISMA_CLIENT_ENGINE_TYPE=wasm), and no engine binary exists anywhere on this filesystem to substitute. Jest fails before any test runs, atPrismaClientconstruction, with"@prisma/client did not initialize yet. Please run prisma generate"— this happens even though Jest setsNODE_ENV=testand my code would only exercise the in-memory fallback path, never actually querying Prisma, so it's specifically the client construction step that's blocked here.Please run
prisma generate && npm testin CI/locally to confirm test execution.Closes #856