ci: 👷 build installable plugin bundles for four platforms - #8
Conversation
Installing a plugin required a Rust toolchain, a sibling augur-rs checkout and a working cargo, which made every measurement PC a development machine. Build all runtime plugins on each pull request and each push to main for macOS arm64/x86_64, Linux x86_64 and Windows x86_64, staged in the exact layout ~/.augur/plugins/ expects, so installing is a copy. main also publishes a rolling plugins-latest release, because workflow artifacts need a login and expire after 90 days while a bench should be able to curl a URL. The workspace depends on the host by path, so the job lays out two sibling checkouts. build-runtime-plugins.sh patches a git source whenever it finds a sibling augur-rs checkout; with path deps that patch matches nothing but still costs a fetch, so the checkout's .git is dropped right after cloning. CI calls the repository's own build and install scripts instead of restating the install layout in YAML — those scripts already own plugin discovery, library naming, A1's protocols folder and the macOS install-name rewrite. Plugins are dlopened into the host process, so pin rust-toolchain.toml to the same 1.95.0 augur-rs pins and read the channel out of that file rather than naming a version in the workflow. Every bundle carries a BUILD-INFO.txt with the augur-rs revision and rustc version behind it, which is what makes an ABI mismatch reported from the bench answerable.
Two defects the first run exposed, both independent of the plugin sources. The Linux job borrowed augur-rs/.github/scripts/install-linux-deps.sh from the host checkout to avoid keeping a second dependency list. That script does not exist on every augur-rs revision the job can be pointed at, so the Linux build failed on the value of augur_rs_ref rather than on anything in this repository. It was also a superset: it installs the X11/Wayland/GL stack for the GUI, which no plugin crate links. Install what the plugins actually need instead — pkg-config and libudev-dev for serialport. setup-rust-toolchain injects RUSTFLAGS="-D warnings" by default. That is right for a lint job and wrong for one that ships artifacts: a dead-code warning in one plugin would have denied the bench a bundle for all of them. Lint gating belongs in its own job. Also record what the run proved about the repository itself: these plugins do not compile against augur-rs main, which lacks the TableSchema, host-view and dataset-descriptor API they use.
augur-rs main still has a two-field TableSchema and no Scatter3dFromTable, HostDatasetDescriptor.relations/display or HostViewRegistry.actions, all of which the plugins in this repository already use. Defaulting AUGUR_RS_REF to main is therefore a guaranteed red build that never hands the bench a bundle. Default to the open host branch that does carry the API instead, and record the coupling in the brief and the ADR. BUILD-INFO.txt already names the exact host ref and SHA behind every library, so this stays visible rather than becoming folklore. Move the default back to main in the same commit that the host API lands there.
The eveSMLM chain expressed its stage dependencies directly: fitting depended on the candidates crate, post-processing on the fitting crate. Plugin crates are cdylibs that each export augur_plugin_vtable, so linking one plugin's rlib into another pulled that symbol in twice. Apple's linker tolerates the duplicate. rust-lld and MSVC's link.exe do not: rust-lld: error: duplicate symbol: augur_plugin_vtable LNK2005: augur_plugin_vtable already defined … fatal error LNK1169 That went unnoticed for as long as the only build machine was a Mac. The first CI run on four platforms found it: macOS produced a complete bundle while Linux and Windows failed to link, which also denied the bench a Windows bundle for the Stage-A plugins, since the build is all-or-nothing. Move everything that crosses a stage boundary into evesmlm-types, a plain library crate that exports no vtable — the wire contract plus the current-localization dataset and registry builders that both fitting and post-processing publish. Plugin-private state stays with its plugin: the candidate tracker's TrackedCluster moves back into the candidates crate. Each plugin still re-exports the names it used to own, so downstream use paths keep compiling. This generalizes what stage-a-plugin-contract already does for the Stage-A owners, and replaces the repo convention that shared types belong in the producing plugin's crate.
Port discovery filtered candidates by the two Unix name patterns (`cu.usbmodem`, `ttyACM`) before probing. Windows names no device — every port is `COMn` — so an attached, correctly driven Teensy was filtered out before any probe could run, and both plugins reported "no USB serial device found (looked for usbmodem/ttyACM)": the two things Windows cannot produce. Move the filter into `stage-a-io::transport::candidate_ports()`, where it is platform-aware: the callout node on macOS, `ttyACM*` on Linux, and every USB-classified port on Windows, falling back to the whole list when the OS classifies nothing. What identifies the device is still the probe (HELLO on the command port, PDA1 frames on the stream port); the filter only keeps probes off unrelated ports. Also open every port with DTR asserted. macOS and Linux do this implicitly, Windows does not, so a sketch gating on `if (Serial)` would stay silent even once the right port was found. The filter existed in four places in two implementations; it is now one function with unit tests covering both platform branches, and the failure message names the ports the OS actually enumerated. Refs ADR 032
Update: the first runs found three real defects, two of them mine1. Linux job borrowed the host's dependency script. 2. 3. And one real defect in the repository (ADR 031)With the host API resolved, macOS built a complete bundle while Linux and Windows failed to link:
Fixed by moving everything that crosses a stage boundary into a new plain library crate Status
Local verification of the refactor: 9/9 runtime plugins build, 324 tests pass, |
Excel's "CSV UTF-8" — the obvious save format on a Windows bench — writes a UTF-8 byte-order mark. Unstripped it becomes part of the first header cell, so `mean_u` stops matching `mean_u` and the protocol is refused for missing a required column that is plainly there. The TOML form fails its parse outright. Neither message points at an invisible character. Strip the BOM once for both readers. CRLF was already handled by `str::lines()`; it now has a test so it stays that way. Refs ADR 027
1 participant
Why
Installing a plugin required a Rust toolchain, a sibling
augur-rscheckout and a workingcargo. That is a reasonable ask of a contributor and an unreasonable ask of the bench machine that runs the experiment.What
Every pull request and every push to
mainbuilds all runtime plugins on four platforms and stages them in the exact layout~/.augur/plugins/expects — so installing becomes a copy.macos-arm64macos-latest.dylibmacos-x86_64macos-13.dyliblinux-x86_64ubuntu-latest.sowindows-x86_64windows-latest.dllPRs publish workflow artifacts.
mainadditionally publishes a rollingplugins-latestrelease (one zip per platform +SHA256SUMS.txt), because artifacts need a GitHub login and expire after 90 days while a bench should be able tocurla URL.workflow_dispatchtakes anaugur_rs_refinput for building against a host revision other thanmain.Design notes
augur-core = { path = "../augur-rs/augur-core" }), so a single-repo checkout cannot resolve the dependency at all.augur-rs/.gitis deleted after checkout.build-runtime-plugins.shadds[patch]flags whenever it finds a siblingaugur-rsgit checkout. With path deps that patch matches nothing — verified by runningcargo metadatawith the exact flags:Patch … was not used in the crate graph, exit 0 — but it still costs cargo a fetch. Removing.gitmakes the detection fail and the path deps win.build-runtime-plugins.shandinstall-built-plugins.shalready own plugin discovery, library naming, A1'sprotocols/copy and the macOS install-name rewrite. Restating any of that in YAML would be a second, silently divergent definition of what an installed plugin is.rust-toolchain.tomlpins 1.95.0, matchingaugur-rs. Plugins arecdylibs the hostdlopens into its own process, so the compiler that builds them and the compiler that buildsaugur-guihave to agree. The workflow parses the channel out of that file rather than naming a version, so CI cannot drift from the pin.augur-rs/.github/scripts/install-linux-deps.shin the checkout the job already has, instead of a second list that can go stale.serialportneedslibudev.zipis not in the Windows runner's bash by default, so packaging per-runner would have needed a per-platform branch for no benefit.Each bundle carries a
BUILD-INFO.txtrecording theaugur-pluginscommit, theaugur-rsref/SHA and the exactrustcversion — which is what turns an "ABI mismatch" report from the bench into an answerable question.Verification
actionlint1.7.7: cleanscripts/build-runtime-plugins.sh --profile release→ 9 runtime plugins, 0 skippedscripts/install-built-plugins.sh --profile release --dest dist/macos-arm64→ 9 plugin folders / 30 files, withstage-a-a1/protocols/{example.csv,example.toml}travelling alongotool -Don the staged A1 dylib →@loader_path/libaugur_plugin_stage_a_a1.dylib, so the macOS identity fix applies to CI bundles toozip -qrover the staged bundle → 30 files, matching what the release job expectsThe four-platform matrix itself is exercised for the first time by this PR.
Base branch
Opened against
fix/stage-a-pockels-parameter-semantics, notmain, to keep the diff at one commit —mainis 40 commits behind, and itsinstall-built-plugins.shneither copies A1'sprotocols/nor rewrites the macOS dylib id, so bundles built frommaintoday would be incomplete. GitHub retargets this PR automatically once the base branch merges.Note
Local
cargoon the dev machine is/opt/homebrew/bin/cargo(1.87.0), which ignoresrust-toolchain.toml. Sinceaugur-rspins 1.95.0, locally built plugins are currently compiled by a different compiler than the host. Putting~/.cargo/binahead of/opt/homebrew/binonPATHpicks up the pin.Docs
Feature Brief
docs/features/ci-prebuilt-plugin-bundles.md, ADR 030, plus README anddocs/installing-plugins.mdinstall routes.🤖 Generated with Claude Code