Skip to content

[#856] Backend: Persist write-ahead audit log entries to Prisma for multi-instance durability - #923

Merged
Junirezz merged 1 commit into
Junirezz:mainfrom
broda-spendy:fix/856-backend-persist-write-ahead-audit-log-entries-to-prisma-for-multi-instance-durability
Jul 29, 2026
Merged

[#856] Backend: Persist write-ahead audit log entries to Prisma for multi-instance durability#923
Junirezz merged 1 commit into
Junirezz:mainfrom
broda-spendy:fix/856-backend-persist-write-ahead-audit-log-entries-to-prisma-for-multi-instance-durability

Conversation

@broda-spendy

@broda-spendy broda-spendy commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #856. writeAheadAuditLog.ts (the Issue #707 prepare/commit/rollback flow for admin configuration changes) stored entries in a process-local array capped at 10,000 records. Entries were lost on restart and invisible to peer instances in multi-pod deployments, so operators couldn't reconstruct partial config changes after a crash between prepare and commit.

Note on this update: this branch was 847 commits behind current main, and several files it touched (schema.prisma, writeAheadAuditLog.ts itself, check-postgres-drift.js) had since been created or substantially changed by unrelated work upstream. Rather than force a mechanical rebase across that much drift (real risk of silently corrupting the Prisma schema or duplicating models), I reimplemented the feature fresh against current main, replacing this branch's history.

What changed

  • Added a WriteAheadAuditEntry Prisma model (backend/prisma/schema.prisma) and a matching migration (backend/prisma/migrations/20260727000000_add_write_ahead_audit_entries), following the same field shape and index conventions as the existing AdminConfigChange model
  • Rewrote writeAheadAuditLog.ts: prepare/commit/rollback/getEntry/list/getPendingEntries/getMetrics/clear are now async and persist via Prisma, following the exact same test-mode/no-DATABASE_URL fallback convention already established by adminConfigChangeAudit.ts (NODE_ENV=test, JEST_WORKER_ID, or no DATABASE_URL). Unlike that simpler precedent, this store is read back within the same request lifecycle (prepare then commit/rollback by id, then list), so the in-memory fallback is a fully functional store, not just a stub — and any unexpected Prisma error at runtime also falls back to the in-memory path per-call rather than throwing
  • Added a stalePending count to getMetrics() (pending entries older than a configurable WAL_STALE_PENDING_TTL_MS, default 15 minutes) so operators can spot admin config changes stuck between prepare and commit/rollback after a crash — this is an additive field, so existing consumers of the WAL metrics response shape are unaffected
  • Updated the 4 GET /admin/wal/* route handlers in index.ts to be async and await the now-async store, with try/catch around each returning a 500 on genuinely unexpected errors. HTTP response shapes are unchanged
  • Updated the existing Issue Backend: Add write-ahead audit log for admin configuration changes #707 describe block in issues705-707-719-723.test.ts to await the now-async calls, and added 2 new tests covering the stalePending metric and confirming a stale pending entry can still be rolled back

Verification

  • npm install succeeds
  • tsc --noEmit: 0 errors in any changed file. Confirmed against a clean upstream/main checkout: baseline is 95 pre-existing errors, entirely in files this PR doesn't touch (apiKeyAuth.ts, walletNonce.ts, etc.) — identical count and files before and after this change
  • eslint: 0 new errors. 1 new instance of an already-established no-non-null-assertion warning pattern in the new stale-pending test, matching the existing style throughout this same test file (baseline 13 warnings → 14, same rule, same file)
  • node scripts/check-migrations.js: passes
  • tsx scripts/check-schema-snapshots.ts: passes (WAL endpoints aren't covered by these API contract snapshots — confirmed by inspecting apiContractSnapshots.ts)
  • Could not run prisma generate or the actual Jest suite: this sandbox's network policy blocks binaries.prisma.sh (confirmed 403 on every engine-fetch attempt, including with PRISMA_ENGINES_CHECKSUM_IGNORE_MISSING=1 and forcing PRISMA_CLIENT_ENGINE_TYPE=wasm), and no engine binary exists anywhere on this filesystem to substitute. Jest fails before any test runs, at PrismaClient construction, with "@prisma/client did not initialize yet. Please run prisma generate" — this happens even though Jest sets NODE_ENV=test and my code would only exercise the in-memory fallback path, never actually querying Prisma, so it's specifically the client construction step that's blocked here.

Please run prisma generate && npm test in CI/locally to confirm test execution.

Closes #856

@Junirezz

Copy link
Copy Markdown
Owner

please fix conflict

…ries to Prisma

writeAheadAuditLog.ts (the Issue Junirezz#707 prepare/commit/rollback flow for
admin configuration changes) stored entries in a process-local array
capped at 10,000 records. Entries were lost on restart and invisible
to peer instances in multi-pod deployments, so operators couldn't
reconstruct partial config changes after a crash between prepare and
commit.

Note on this PR: the existing branch/PR for this issue was 847 commits
behind current main, and several files it touched (schema.prisma,
writeAheadAuditLog.ts itself, check-postgres-drift.js) had since been
created or substantially changed by unrelated work upstream. Rather
than force a mechanical rebase across that much drift (high risk of
silently corrupting the Prisma schema or duplicating models), this
reimplements the feature fresh against current main, replacing this
branch's history.

What changed:

- Added a WriteAheadAuditEntry Prisma model (backend/prisma/schema.prisma)
  and a matching migration
  (backend/prisma/migrations/20260727000000_add_write_ahead_audit_entries),
  following the same field shape and index conventions as the existing
  AdminConfigChange model
- Rewrote writeAheadAuditLog.ts: prepare/commit/rollback/getEntry/list/
  getPendingEntries/getMetrics/clear are now async and persist via
  Prisma, following the exact same test-mode/no-DATABASE_URL fallback
  convention already established by adminConfigChangeAudit.ts
  (shouldUseInMemoryWalFallback: NODE_ENV=test, JEST_WORKER_ID, or no
  DATABASE_URL). Unlike that simpler precedent, this store is read back
  within the same request lifecycle (prepare then commit/rollback by
  id, then list), so the in-memory fallback is a fully functional store
  (not just a stub), and any unexpected Prisma error at runtime also
  falls back to the in-memory path per-call rather than throwing
- Added a stalePending count to getMetrics() (pending entries older
  than a configurable WAL_STALE_PENDING_TTL_MS, default 15 minutes) so
  operators can spot admin config changes stuck between prepare and
  commit/rollback after a crash - this is an additive field, so
  existing consumers of the WAL metrics response shape are unaffected
- Updated the 4 GET /admin/wal/* route handlers in index.ts to be
  async and await the now-async store, with try/catch around each
  returning a 500 on genuinely unexpected errors (the store itself
  degrades gracefully on Prisma errors, so this is a backstop). HTTP
  response shapes are unchanged
- Updated the existing Issue Junirezz#707 describe block in
  issues705-707-719-723.test.ts to await the now-async calls, and
  added 2 new tests covering the stalePending metric and confirming a
  stale pending entry can still be rolled back

Verification:
- npm install succeeds
- tsc --noEmit: 0 errors in any changed file. Confirmed against a
  clean upstream/main checkout: baseline is 95 pre-existing errors,
  entirely in files this PR does not touch (apiKeyAuth.ts, walletNonce.ts,
  etc.) - identical count and files before and after this change
- eslint: 0 new errors. 1 new instance of an already-established
  no-non-null-assertion warning pattern in the new stale-pending test,
  matching the existing style used throughout this same test file
  (baseline 13 warnings -> 14 with this change, same rule, same file)
- node scripts/check-migrations.js: passes (the new migration doesn't
  match any high-risk pattern the safety scanner checks for)
- tsx scripts/check-schema-snapshots.ts: passes (WAL endpoints aren't
  covered by these API contract snapshots, confirmed by inspecting
  apiContractSnapshots.ts)
- Could not run `prisma generate` or the actual Jest suite: this
  sandbox's network policy blocks binaries.prisma.sh (confirmed via
  403 on every engine-fetch attempt, including with
  PRISMA_ENGINES_CHECKSUM_IGNORE_MISSING=1 and forcing
  PRISMA_CLIENT_ENGINE_TYPE=wasm), and no engine binary exists
  anywhere on this filesystem to substitute. Jest itself fails before
  any test runs, at PrismaClient construction, with "@prisma/client
  did not initialize yet. Please run prisma generate" - this happens
  even though Jest sets NODE_ENV=test and my code would have exercised
  only the in-memory fallback path, never actually querying Prisma, so
  it's specifically the client construction step that's blocked here.
  Please run `prisma generate && npm test` in CI/locally to confirm.

Closes Junirezz#856
@broda-spendy
broda-spendy force-pushed the fix/856-backend-persist-write-ahead-audit-log-entries-to-prisma-for-multi-instance-durability branch from 0b2f507 to ca0a318 Compare July 27, 2026 10:08
@drips-wave

drips-wave Bot commented Jul 27, 2026

Copy link
Copy Markdown

@broda-spendy Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@broda-spendy

Copy link
Copy Markdown
Contributor Author

Checked the three failing checks (Backend build, Backend lint + test, backend-governance). All three trace to the same root cause: Property 'sql'/'join'/'empty' does not exist on type 'typeof Prisma' in backend/src/exportJobs.ts and backend/src/apiKeyAudit.ts — neither file is touched by this PR.

Confirmed this is pre-existing and unrelated: the identical three checks are currently failing on main itself (commit f18ae966), with the same error. Looks like the generated Prisma client is currently missing the Prisma.sql/Prisma.join/Prisma.empty tagged-template helpers that those two files depend on — worth a separate look, but out of scope for #856 and not something introduced by this change.

Locally, tsc --noEmit produces an identical 95-error baseline with and without this PR applied (see PR description), which is consistent with this being pre-existing breakage rather than something this PR caused.

@Junirezz
Junirezz merged commit 3bfa6be into Junirezz:main Jul 29, 2026
11 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Backend: Persist write-ahead audit log entries to Prisma for multi-instance durability

3 participants