Sort submissions by metadata count

[jerbaroo]

This PR makes changes to how the chunk selection query of the PreferDistinct strategy. In the Files changed tab, you can see the impact on the benchmark SVG .

Key changes:

• Includes a test of fairness in the Python test suite.
• We no longer do index walks over all chunks, instead we sort submissions by metadata.
• Already-reserved check is from SQLite via FFI, commit from @ReinierMaas
• Metadata counts are not passed to SQLite via the generated query, but rather checked from SQLite via FFI, commit from @ReinierMaas

[Copilot]

Pull request overview

Moves PreferDistinct selection from chunk scans to metadata-ranked submission selection with indexed chunk lookup.

Changes:

• Ranks submissions using reservation metadata exposed through SQLite FFI.
• Adds submission random-order indexing and simplifies metastate tracking.
• Adds fairness and reservation-filtering tests.

Reviewed changes

Copilot reviewed 12 out of 15 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
workspace-hack/Cargo.toml	Updates generated dependency features.
opsqueue/src/consumer/strategy.rs	Reworks strategy query generation.
opsqueue/src/consumer/dispatcher/reserver.rs	Exposes reservation lookup.
opsqueue/src/consumer/dispatcher/mod.rs	Registers SQLite FFI callbacks.
opsqueue/src/consumer/dispatcher/metastate.rs	Simplifies counts and adds JSON export.
opsqueue/src/common/submission.rs	Reuses one SQL parameter.
opsqueue/src/common/chunk.rs	Adds signed chunk-index conversion.
opsqueue/migrations/20260803133844_add_random_order_index_to_submissions.up.sql	Adds submission random ordering.
opsqueue/migrations/20260803133844_add_random_order_index_to_submissions.down.sql	Reverts submission random ordering.
opsqueue/Cargo.toml	Adds SQLite FFI dependency.
libs/opsqueue_python/tests/test_roundtrip.py	Tests PreferDistinct fairness.
Cargo.toml	Defines the workspace SQLite dependency.
Cargo.lock	Updates resolved dependencies.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.