fix(workspace-search): repair wedged search indexing

[posthog]

Problem

• Hybrid workspace search (shipped in feat(workspaces): add hybrid workspace search #701) can wedge permanently for a workspace: newly added documents and files never become searchable, search silently returns stale results, and nothing surfaces the failure.
• Root cause: kernel_search_vector_deletes is created with CREATE TABLE IF NOT EXISTS outside the version-reset drop set, because its rows reference vectors still live in Vectorize and must survive resets. A Durable Object whose table was created by an earlier iteration of the schema never gains the attempts column — IF NOT EXISTS is a no-op and the reset never touches it.
• flushVectorDeletes is the first statement in WorkspaceSearchIndexer.processBatch and queries WHERE attempts < …, so an affected object throws SqlError: no such column: attempts on every alarm tick and dies before indexing a single item — forever, with no self-healing path.
• Secondary: embedWorkspaceSearchTexts calls ai.run("@cf/baai/bge-m3", …) with no retry, so a single transient AiError: 3043 burns one of the five index attempts and opens a fresh error-tracking issue.

Changes

• Real migration in initializeWorkspaceSearchStorage: PRAGMA table_info check, then ALTER TABLE kernel_search_vector_deletes ADD COLUMN attempts INTEGER NOT NULL DEFAULT 0 when absent. Chosen over moving the table into the reset drop set — a drop would discard queued deletes and orphan their vectors in Vectorize.
• Bounded retry with backoff around ai.run: retryable AiErrors (3xxx internal, 5xx-class upstream) are retried in-process (4 attempts, exponential backoff capped at 2s); permanent client/validation codes fail fast. A transient hiccup no longer consumes an index attempt or pages anyone.

Note

Failed items are already visible via the lexical fallback — searchKeyword includes vector_status = 'failed' and getStatus reports partial — so exhausted items keep surfacing in keyword results rather than vanishing.

Tests

• New workspace-search-embeddings.test.ts: retryable-vs-permanent classification, retry-then-succeed, attempt-budget exhaustion, and fail-fast on permanent errors.

Why

Restore a brand-new flagship surface (hybrid workspace search) that is silently, permanently broken for any affected workspace, with no client-visible error and no recovery short of a code fix.

---

Created with PostHog Desktop from this inbox report.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[github-actions]

React Doctor found 1 new issue in 1 file · 1 warning · score 92 / 100 (Great) · 1 fixed · vs main

1 warning

src/features/workspaces/search/workspace-search-embeddings.ts

• ⚠️ L21 await inside a loop async-await-in-loop

_{Reviewed by React Doctor for commit b42088e. See inline comments for fixes.}

[github-actions]

React Doctor · react-doctor/async-await-in-loop (warning)

This makes the for…of loop slow because each await runs one after another, so collect the independent calls & run them together with await Promise.all(items.map(...))

Fix → Collect the items, then use await Promise.all(items.map(...)) so independent work runs at the same time

Docs

[urjitc]

Closing as partially superseded and no longer suitable to merge as one PR. The critical kernel_search_vector_deletes.attempts migration already landed on main as a3cb6ca7. If Workers AI embedding retries are still needed based on current telemetry, they should return as a separate focused PR against current main.