Skip to content

Generate the GoMEOS flat FFI layer from the MEOS-API catalog#8

Merged
estebanzimanyi merged 1 commit into
MobilityDB:mainfrom
estebanzimanyi:feat/arrow-ptr-shape
Jul 11, 2026
Merged

Generate the GoMEOS flat FFI layer from the MEOS-API catalog#8
estebanzimanyi merged 1 commit into
MobilityDB:mainfrom
estebanzimanyi:feat/arrow-ptr-shape

Conversation

@estebanzimanyi

@estebanzimanyi estebanzimanyi commented Jun 28, 2026

Copy link
Copy Markdown
Member

Make GoMEOS a catalog-driven binding: the flat FFI layer is generated from
the MEOS-API catalog (meos-idl.json) and derived in CI, so no MEOS header
is parsed in this repository.

What this delivers

  • tools/codegen.py emits the flat FFI layer into the repo-root functions
    package (mirroring JMEOS functions / pymeos_cffi.functions): every MEOS
    struct is an opaque handle, char* is marshalled as string via
    cstring_to_text / text_to_cstring, enums are typed ints, and C function
    pointers are *[0]byte — the JMEOS-flat convention, no array unpacking.
  • A types.go type layer (opaque handles + catalog enums) is emitted only for
    symbols the wrappers actually reference.
  • The build workflow provisions the catalog and the all-families libmeos from
    MobilityDB via the shared provision-meos action, regenerates functions/,
    fails if the committed generated code drifts from the catalog, then
    compiles the package against the installed libmeos and runs the
    portable-parity gate (29/29 bare names, all six families).
  • The catalog is no longer committed — it is derived in CI; the generated
    functions package is the checked-in artifact.

The hand-written OO gomeos package is left in place; regenerating it too from
the catalog's objectModel is tracked as follow-up work.

@estebanzimanyi estebanzimanyi changed the title tools/codegen: map the Arrow C Data Interface structs to unsafe.Pointer Generate the GoMEOS flat FFI layer from the MEOS-API catalog Jul 11, 2026
@estebanzimanyi

Copy link
Copy Markdown
Member Author

Depends on MobilityDB/MEOS-API#52. CI derives the catalog via provision-meos@master; with #52 the 10 h3index_* (and 5 sibling) public functions are attributed to meos_h3.h and the generated functions/ package includes them. Merging this before #52 lands leaves those 15 functions out of the derived package. Land #52 first.

Add a codegen (tools/codegen.py) that projects the MEOS-API meos-idl.json
catalog into a flat FFI Go package (functions/), with the JMEOS marshalling
convention: char* maps to string, every other pointer to an opaque handle,
enums to int, function pointers to a raw pointer.

CI derives the catalog and an all-families libmeos from a single MobilityDB
commit via the shared provision-meos action, regenerates the package, builds
and tests it, and reports drift against the committed snapshot without failing
the build (the snapshot is a convenience projection for go-get consumers). The
committed catalog is dropped from git and derived in CI. The portable
bare-name parity gate runs against the functions package.

The vendored headers and the hand layer are adapted to the current MEOS.
@estebanzimanyi estebanzimanyi merged commit 598a545 into MobilityDB:main Jul 11, 2026
2 checks passed
@estebanzimanyi estebanzimanyi deleted the feat/arrow-ptr-shape branch July 11, 2026 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant