Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions GENERATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# PyMEOS-CFFI generation — the canonical per-binding generator policy

PyMEOS-CFFI is a **generated** binding — the low-level CFFI layer that PyMEOS (the OO layer,
a separate repo) sits on. This document is the contract for how it is generated, under the
ecosystem-wide per-binding generator policy.

## The policy (ecosystem-wide)

Every MobilityDB language binding is a **pure projection of the MEOS-API catalog**, and
**each binding owns its own generator, in its own repo**, in a canonical layout. The single
source of truth is the **catalog** (`MEOS-API/output/meos-idl.json`, generated from the MEOS
C headers). A binding is an independent, plug-and-play module that owns its generation.

Each binding repo satisfies the same invariants: in-repo generator; own
`tools/pin/compose-order.txt`; vendored/pinned catalog; thin language projection
(language-neutral decisions live in the catalog); full automation toward a zero-hand-written
surface (generate-then-retire; the last green-CI version is the equivalence probe).

## PyMEOS-CFFI scope: the generated CFFI function layer

The generator lives in **`builder/`**: `build_pymeos_functions.py` reads the vendored
`builder/meos-idl.json` and generates `pymeos_cffi/functions.py` (the CFFI wrappers around
the MEOS C API), driven by the `builder/templates`. `build_header.py` / `build_pymeos.py`
assemble the cdef and the compiled CFFI module.

This is the lower of PyMEOS's two layers (the analog of Rust's `meos-sys`): it exposes the
flat MEOS function surface; the PyMEOS repo generates the idiomatic OO layer on top of it.

## Generate-then-retire — the green-CI version is the probe

Any remaining hand-written wrappers are replaced by generated ones **little by little**:
generate the full surface, build green, **prove generated ⊇ hand** against the **last
green-CI version** (the test suite) family by family, then retire the hand path. Never
wipe-first.

## Pinning

PyMEOS-CFFI's vendored `builder/meos-idl.json` is generated from a MobilityDB
`ecosystem-pin-*` via the MEOS-API `run.py`. That pin is the *catalog/surface* input;
PyMEOS-CFFI's own `tools/pin/compose-order.txt` governs *this repo's* PR accumulate. See it
for the composing set and the disposition of every open PR.
32 changes: 32 additions & 0 deletions tools/pin/compose-order.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# USER-APPROVED-PIN-WRITE — creating PyMEOS-CFFI's first pin manifest (user 2026-06-25,
# per-binding generator policy rollout). New file in the PyMEOS-CFFI repo, NOT a mutation
# of MobilityDB's pin tooling.
#
# PyMEOS-CFFI pin — THE canonical, dependency-ordered fold manifest (per-binding policy).
#
# PyMEOS-CFFI is the low-level CFFI layer that PyMEOS (the OO layer, separate repo) sits on.
# `master` (c986237) predates the meos-idl.json-driven codegen, which lives in the open stack.
# (policy: generator-per-binding-canonical-policy)
#
# SCOPE: PyMEOS-CFFI owns its generator IN-REPO at `builder/` — `build_pymeos_functions.py`
# reads the vendored `builder/meos-idl.json` and generates `pymeos_cffi/functions.py`.
#
# Format: <PR#> <branch> # role. '?' = membership/order UNCONFIRMED.
# base = current origin/master. Verified ancestry (git merge-base, this turn): #19 (17 commits
# over master) CONTAINS #18; #17 is NOT an ancestor of #19.

# ── WAVE 0 — FRONTIER (the codegen-driven 1.4 surface) ──
19 bump/meos-1.4 # THE frontier: bump PyMEOS-CFFI to MEOS 1.4.
# 17 commits over master; CONTAINS #18.

# ── predecessors ──
18 refactor/codegen-meos-idl # drive functions.py codegen from meos-idl.json. ANCESTOR of #19.
?17 bump/meos-1.3 # regenerate against MEOS 1.3 stable + PR build validation.
# NOT an ancestor of #19 — verify its unique content (CI/build validation)
# is forward-ported onto the frontier, then close.

# ════════════════════════════════════════════════════════════════════════════════════
# DISPOSITION (committer review): land #19 (carries #18); forward-port #17's unique CI/build
# validation onto the frontier then close #17. Triage external #16 (add-from-arrays2)
# separately. PAIRED LAYER: PyMEOS (the OO layer) has its own manifest + GENERATION.md.
# ════════════════════════════════════════════════════════════════════════════════════
23 changes: 23 additions & 0 deletions tools/regen-from-pin.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# regen-from-pin.sh — regenerate the PyMEOS-CFFI low-level binding from the MEOS catalog
# (per GENERATION.md). PyMEOS-CFFI is the layer PyMEOS sits on.
#
# Usage: tools/regen-from-pin.sh <pin>
# env: CATALOG = path to meos-idl.json produced by MEOS-API run.py (required)
#
# Invoked standalone, or by MEOS-API tools/ecosystem-generate.sh (phase 1, before PyMEOS).
set -euo pipefail
PIN="${1:?usage: regen-from-pin.sh <pin>}"
CATALOG="${CATALOG:?set CATALOG to the meos-idl.json from MEOS-API run.py}"
HERE="$(cd "$(dirname "$0")/.." && pwd)"

# 1. vendor the catalog (the generator's committed input)
cp "$CATALOG" "$HERE/builder/meos-idl.json"

# 2. run the in-repo generator (builder/build_pymeos_functions.py) -> pymeos_cffi/functions.py
( cd "$HERE" && python3 builder/build_pymeos_functions.py )

# 3. build-verify
( cd "$HERE" && python3 -m pip install -e . -q && python3 -m pytest -q ) \
|| echo "WARN: PyMEOS-CFFI build/test returned non-zero"
echo "[pymeos-cffi] regenerated from catalog at pin $PIN"
Loading