Multi-model embedding: normalized chunk_vectors table + model switching

[EthanThatOneKid]

Overview

Multi-model embedding support for Worlds: normalized chunk_vectors table, model switching, and async backfill. This is the post-beta design — the July 31 private beta ships with one model per world.

Current design (beta)

• Single vector F32_BLOB column on chunks table
• embedding_model TEXT on worlds_metadata identifies which model produced vectors
• On model switch, synchronous client.reindex() re-embeds all chunks
• One model per world, no rollback without full reindex

Problem

Users will eventually want to:

1. Switch embedding models without losing search availability
2. Run queries against a specific model's vector space
3. Compare search quality across models on the same data
4. Roll back to a previous model if quality degrades

The current single-vector design can't support any of these without a full reindex.

Proposed design

Normalized chunk_vectors table

CREATE TABLE chunk_vectors (
  chunk_id INTEGER NOT NULL,
  model TEXT NOT NULL,
  vector F32_BLOB NOT NULL,
  created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now')),
  PRIMARY KEY (chunk_id, model),
  FOREIGN KEY (chunk_id) REFERENCES chunks(id) ON DELETE CASCADE
);

Replace the vector column on chunks with this normalized table. The idx_chunks_vector index moves here, scoped per model.

Per-model vector indexes

-- Partial index per active model (created dynamically on model registration)
CREATE INDEX idx_cv_vector_{model} ON chunk_vectors(vector)
  WHERE model = '{model}';

Or use a single index with model-aware query:

CREATE INDEX idx_cv_vector ON chunk_vectors(vector, model);

Query routing

vector_top_k scoped to model:

SELECT chunk_id, row_number() OVER (...) AS rank_number
FROM vector_top_k('idx_cv_vector_{model}', vector32(?), ?)

Or filter in a CTE:

WITH model_vectors AS (
  SELECT chunk_id, vector FROM chunk_vectors WHERE model = ?
),
vec_matches AS (
  SELECT chunk_id AS rowid, row_number() OVER (...) AS rank_number
  FROM vector_top_k('idx_cv_vector', vector32(?), ?)
  WHERE chunk_id IN (SELECT chunk_id FROM model_vectors)
)

Model switching flow

1. User sets embedding_model on world via PATCH /worlds/{id}
2. API stores new model name but does NOT immediately reindex
3. New chunks get vectors for the new model
4. Background reindex job (triggered by API or CLI) backfills old chunks:
   • Embed old chunks with new model
   • Insert rows into chunk_vectors with new model name
   • Mark model as "active" when backfill completes
5. During backfill, search uses old model's index (no downtime)
6. After backfill, switch vector_top_k to new model's index
7. Optionally delete old model's vectors after validation period

Import path changes

Current: embed once, insert into chunks.vector.
New: embed once, insert into chunk_vectors with model = embedding_model.

Reindex path changes

Current: rebuildLibsqlSearchIndexFromQuads() re-embeds all chunks.
New: rebuildLibsqlSearchIndexFromQuads() embeds into chunk_vectors for active model.

Search path changes

Current: vector_top_k('idx_chunks_vector', ...) over chunks.vector.
New: vector_top_k('idx_cv_vector_{model}', ...) over chunk_vectors.vector WHERE model = ?.

Write amplification

• Import: No change (one embedding call per chunk)
• Model switch + backfill: N embedding calls for N existing chunks (one-time cost)
• Multi-model queries: No write amplification (queries read from existing vectors)

Open questions

1. Should we support model registration metadata (embedding_models table) or keep embedding_model as a free-text field?
2. Should backfill be automatic (triggered by embedding_model change) or manual (CLI/API command)?
3. How long do we keep old model vectors after switching? (Validation period)
4. Should SearchRequest accept a model override for cross-model quality comparison?
5. What's the storage budget? (32-dim TFJS vs 1536-dim OpenAI = ~48x per vector)

Migration path

The v0 schema has chunks.vector. The migration:

1. Create chunk_vectors table
2. Migrate existing vectors: INSERT INTO chunk_vectors (chunk_id, model, vector) SELECT id, embedding_model, vector FROM chunks WHERE vector IS NOT NULL
3. Drop chunks.vector column
4. Drop idx_chunks_vector index
5. Create per-model partial indexes on chunk_vectors

Related

• worlds-api#2 (inference cost offset research — beta one-model design)
• worlds-api MemoryProfile PR (adds embedding_model to world metadata)
• Wazoo Beta deadline: July 31 (this is post-beta)