fix(perf): paginate GET /streams/pending to prevent unbounded memory fetch

[irefavor15-dotcom]

Summary

Closes #337

The processing worker called GET /streams/pending with no pagination. During a backlog (e.g. after downtime) this returned all pending events in a single HTTP response, causing an unbounded memory spike and unpredictable worker footprint.

Changes

api/src/streams/streams.controller.ts

New GET /streams/pending endpoint with limit and cursor query params:

• limit: batch size (default 100, max 1000)
• cursor: offset for the next page (default 0)
• Returns { data: PendingStreamEvent[], nextCursor: number | null }
• nextCursor is null when the returned batch is smaller than limit (final page)

api/src/streams/streams.service.ts

Added getPendingEvents(limit, offset) delegating to the repository layer.

api/src/streams/repository/streams-db.repository.ts

SQL implementation: queries stream_data ordered by timestamp ASC (FIFO) with LIMIT / OFFSET.

api/src/streams/repository/streams.repository.ts (in-memory)

Stub returning { data: [], nextCursor: null } — the in-memory repository has no stream_data store and is used only in unit tests.

xstreamroll-processing/src/config.ts

Added POLL_BATCH_SIZE to the zod schema (default '100', transformed to a positive integer).

xstreamroll-processing/src/worker.ts

Rewrote pollOnce() to paginate:

• Starts at cursor=0, fetches up to POLL_BATCH_SIZE events per request
• Advances cursor via nextCursor until it is null or until the high-watermark is reached (back-pressure)
• Backward-compatible: still accepts plain-array responses from the old un-paginated endpoint

xstreamroll-processing/.env.example

Documents POLL_BATCH_SIZE with description and trade-offs.

Behaviour

Scenario	Before	After
Backlog of 10 000 events	10 000 events in one response (~memory spike)	≤100 events per batch, loops until drained
High-watermark reached mid-poll	N/A	Stops paging, logs a warning, waits for next interval
Last page	N/A	nextCursor is null → loop exits cleanly

Testing

• TypeScript compile passes (type-check runs on push)
• Worker test suite unaffected (mocks remain compatible with both response shapes)