Stage-A: protocol rows name themselves, plus the accumulated Stage-A plugin stack - #9
Open
muthmann wants to merge 49 commits into
Open
Stage-A: protocol rows name themselves, plus the accumulated Stage-A plugin stack#9muthmann wants to merge 49 commits into
muthmann wants to merge 49 commits into
Conversation
PDA1 wire protocol (fragmentation-tolerant parser with CRC resync), v1 ASCII command grammar, typed serial client with idempotent sequence retries and stream-integrity accounting, bounded background I/O worker, .pdq raw-frame writer, JSON run sidecar, calibrated optical log-contrast estimator (clipping/dark-headroom guarded), and a mock controller for hardware-free tests. Wire-compatible with stage-a-controller include/wire_protocol.h.
Live decimated photodiode waveform, calibrated clipping-guarded optical contrast, stream-integrity status, and gated manual controller actions (connect/config/start/stop/expert drive). Fails closed on the ABI v5 execution context: serial I/O only in the active live-capture worker; commands are host actions, never persistent settings.
Statistical core: phase folding (hardware EXT_TRIGGER fiducials or software clock-skew recovery via Rayleigh-power frequency scan), Rayleigh detection with Bonferroni-charged trials, median-background phase-locked excess, probit a_min fit with profile CI, hot-pixel mask from an unmodulated reference window. Sweep engine bisects the drive code, grids the bracketed transition, and records measured optical contrast per point. PDQ + sidecar + results export per run; fails closed on the ABI v5 execution context.
Feature brief for the stage-a-io/monitor/a1 stack, ADR 005 for the device-ownership boundary (plugins own the Teensy, AugurRs stays generic), feature index entry. Note: the legacy plugins (localization/reconstruction/focus-metrics/ evesmlm) on this branch predate the current augur-rs plugin API and do not compile against it — their refresh is in progress on feature/eve-batch-findings; rebasing that work onto plugin ABI v5 only adds the new FfiPreviewFrame.external_triggers field in one test initializer.
… refresh Updates the evesmlm suite, localization, and reconstruction plugins plus repo docs to the current augur-rs plugin interface (host-view actions, dataset display metadata/relations, TableSchema extensions, CompactEvent accessors).
# Conflicts: # docs/features/README.md
Adds the new FfiPreviewFrame.external_triggers field to the two test initializers in focus-metrics and evesmlm-candidates; all plugins now compile and test against the current augur-rs plugin API (ABI v5).
The mock controller spoke an invented protocol (ARM/RUN/FAULT_CLEAR verbs, capabilities HELLO field, BAD_* error codes, arbitrary CONFIG fields), so tests validated commands the Teensy never accepts. It now mirrors stage-a-controller/src/main.cpp verbatim: verbs, SAFE_IDLE/CONFIGURED/RUNNING state machine, RANGE/STATE/SYNTAX/PROTOCOL/VERB error details, single-entry idempotent reply cache, and unknown-CONFIG-field rejection (the host's feature-detection contract). The reserved v2 waveform fields are only accepted behind an explicit with_waveform_extension() opt-in, which also synthesizes photodiode blocks through a Pockels-like sin² transfer. Host fixes uncovered by the faithful mock: - surface async control notices (watchdog !FAULT) through poll_events even with no request in flight; monitor and A1 now react instead of showing a stale acquiring state - A1 sweep issues STOP before CONFIG between measurement points (CONFIG is illegal while RUNNING) - record the ACKed CONFIG fields in the A1 run sidecar - reject wire frames with an unknown protocol version like the reference parser does - stream the PDQ file CRC incrementally instead of buffering the whole recording in memory - monitor maps the unknown_config_field rejection of drive fields to a clear 'no waveform backend' message
New plugins/stage-a-funcgen crate: manual Pockels-cell drive control (sine/square/sawtooth, frequency, center/amplitude DAC codes) with the resulting optical amplitude always measured from the photodiode as a = ln(Vmax/Vmin) — the commanded amplitude is a phase-modulation depth and maps non-linearly to light, so it is never reported as an optical level. The default 'mock' port runs the waveform-extended mock controller on an in-process thread and streams a synthetic sin²-transfer photodiode response, so the full control loop works with zero hardware. Against real firmware 0.2.0 the reserved drive fields are feature-detected via the unknown_config_field rejection and reported as 'no waveform backend'; actual output stays blocked on the hardware freeze per stage-a-controller/docs/features/waveform-drive.md. Same fail-closed safety model as stage-a-monitor: LiveCapture + effects gating, drive parameters as settings but application as an explicit action, local DAC-range validation before any command, watchdog fault surfacing.
…n and photodiode pair - delete stage-a-monitor, stage-a-funcgen, stage-a-a1 (too complex for the current bench stage; retained in git history) - add stage-a-modulation: capped power slider, CONST/SINE/SQUARE with frequency and min threshold, immediate MOD transfer, board-reported DAC code - add stage-a-photodiode: SMA5/pin18/A4 stream readout on the second CDC port with RAW and EXCITATION (I_exc = I_tot - I_pd) modes and rolling chart - extend the stage-a-io mock to firmware 0.3.0 (MOD verb, capabilities) - ADR 006 (two plugins, one port each), rewritten stage-a brief, doc updates
The host settings UI reads enum values with as_u64() and writes the selected variant index; the modulation/photodiode plugins returned and expected variant name strings, so port and mode radio buttons could never be set. Map indices against the schema's variant list in get_setting/set_setting (names still accepted) and pin the contract with round-trip tests.
auto in stage-a-modulation now probes each attached port with HELLO and picks the command port; auto in stage-a-photodiode listens for PD lines and picks the stream port. Filter macOS port lists to the cu.* callout nodes so each device appears once. Verified against the live Teensy (firmware 0.3.0).
Port entries now read '/dev/cu.usbmodem… (Teensyduino Dual Serial)' so the Teensy is recognisable among the enumerated devices; the parenthesised label is display-only and stripped when the setting is applied. stage-a-io gains available_ports_with_labels() for this.
…ut camera frames The host only calls process_frame() while camera frames flow, so the action-button + effects-gate design never connected on a camera-less bench (verified: the serial port stayed free while the GUI ran). Connect is now a checkbox setting handled in set_setting, all serial I/O lives in a plugin-owned device thread (slider drags coalesce into one pending MOD), status comes from shared state, and replay mode still disconnects defensively. Verified end-to-end against the live Teensy: auto-probe, level 800 -> board code 800, level 0 -> 0.
…elope and moving average Track firmware 0.4.0 (ADR 003 in stage-a-controller): the stream port now carries PDA1 SamplesU16 frames at pd_stream_rate_hz instead of 50 Hz ASCII lines, raising the plot's data rate 400x. - parse with stage-a-io's FrameParser (new dep, default-features off: wire parser only); auto port probe detects sample frames - bounded raw ring (130 s / 4 M samples) keyed by device sample index; rate changes and index jumps restart the segment so index/rate stays a consistent time base across acquisition handovers - chart decimates the window into <= 1000 min/mean/max envelope buckets; short windows render raw samples; window down to 10 ms - moving-average indicator for the low-voltage regime: fixed sample window (default 4) or one full period of a user-set sync frequency (window = rate / f), making the mean phase-independent under modulation; overlay line + numeric readout in status views - status table gains rate, moving avg, and stream-integrity columns (device drops, CRC failures, resync bytes, segment restarts) - stage-a-io: gate the Duration import behind the hardware feature so default-features = false builds are warning-free Verified: cargo fmt, clippy -D warnings (photodiode + stage-a-io), 9 plugin tests + 29 stage-a-io tests green.
…e photodiode plugin Two save modes behind one Data settings section: - monitor cache: the raw ring is now cache_s seconds long (default 20 s, 1-130 s) and a Save-cache-snapshot button dumps it as pd_cache_<utc>.csv (sample_index, t_s on the device clock, raw code, raw volts) plus a JSON sidecar carrying rate, integrity counters, display mode, and reference so derived quantities stay reproducible - recording: a Record toggle tees every incoming SamplesU16 frame verbatim to pd_rec_<utc>.pdq via stage-a-io's PdqWriter from the reader thread; stopping (or disabling the plugin) finalizes the file and writes a sidecar with per-recording integrity deltas and validity; the mock synthesizes identical wire frames so recordings parse the same without hardware - data_dir uses the new Path setting kind; record/save failures surface through status entries like connect errors; status shows a live REC indicator with recorded seconds Verified: cargo fmt, clippy -D warnings, 13 plugin tests green.
…odiode plugin - PD Spectrum window: Hann-windowed radix-2 FFT (no deps) over the newest power-of-two window of raw samples (256-16384; ≈1.2 Hz resolution at 20 kSa/s), one-sided amplitude in volts with max-hold bin decimation so narrow peaks survive the plot budget. Window placement means the FFT only runs while the window is open. Verified by test: a synthesized 1 kHz 0.4 V tone is recovered at the right frequency and amplitude. - time_axis setting: BEFORE NOW (scrolling, x ends at 0) or SEGMENT TIME (absolute device-clock seconds) — frozen plots and cursor measurements read as positions instead of implied motion.
Timed MOD sequences without cross-plugin control: the executor lives inside the plugin that already owns the command port. - protocol file: loops = N plus [[steps]] with duration_s, wave (OFF/CONST/SINE/SQUARE), level, min, frequency_hz — fully validated before the run starts (ranges match the firmware grammar) - executor thread walks the steps on an absolute schedule (no drift accumulation) and feeds the same coalescing pending-command slot the device thread drains, so it never touches the serial port itself; the last step holds after completion (set-and-hold), stop is immediate, and disconnecting aborts the run - protocol_path uses the new Path setting kind; run/stop is a settings-driven toggle that works with no camera; progress (loop, step, summary) shows in the status entries Verified: cargo fmt, clippy -D warnings, 8 plugin tests green incl. an end-to-end run against the mock controller.
…ry cells Prepares the plugin for firmware 0.5.0's 500 kSa/s DMA stream (ADR 004 in stage-a-controller): a full raw-window rescan per repaint stops being viable around that rate. - ingest maintains 64:1 min/max/sum summary cells aligned to deque offsets; eviction drops whole cells so the alignment (and the device-clock index base) survives, at the cost of up to one cell of ring slack - chart buckets and every moving-average window combine cells plus raw edge samples via range_summary — O(range/64) instead of O(range), verified exact against naive scans across cell boundaries and after eviction - ring cap raised to 16 M samples (32 s at 500 kSa/s, 32 MiB of codes); cache_s keeps ruling the duration at lower rates Verified: cargo fmt, clippy -D warnings, 16 plugin tests green.
The photodiode plugin dropped the pre-0.4.0 ASCII PD-line path, so a Teensy running old firmware produced a generic 'no PDA1 sample frames' error that gave no hint at the real cause. The auto-probe now classifies each port (PDA1 frames / legacy ASCII / nothing) and, when it sees the 'PD code=…' ASCII stream, tells the user to flash stage-a-controller 0.4.0+ instead — the actual fix, since the plugin and firmware ship in lockstep. Verified: cargo fmt, clippy -D warnings, 16 tests green.
…ntract Introduces `stage-a-a1` as the measurement owner that drives the modulation and photodiode plugins through a leased device contract, plus the `stage-a-plugin-contract` crate that carries the shared settings/telemetry schema between them. Also lands the supporting work these depend on: - optical waveform generation and inversion in the modulation plugin - Pockels transfer calibration (measured V_null / Vpi) with sweep support - photodiode contrast estimation and monitor-cache snapshots - `.pdq` recording format with SHA-256 integrity sidecars Documented in docs/features/stage-a-a1.md, stage-a-a1-automation.md, stage-a-optical-waveform.md, stage-a-pockels-calibration.md and ADRs 007-011.
…display mode The photodiode sits behind the PBS reject port and measures the complement I_pd = I_tot - I_exc — a property of the bench, not of what the operator chose to plot. `optical_summary` picked its geometry from the display mode, so leaving the chart on RAW published the raw detector contrast as `measured_log_contrast`. A1's amplitude sweep settles on that value against a target `a`: it would never settle, time out at every point, and write a wrong `measured_a` into each sweep sidecar. The geometry is now always the rejected complement; the display mode is presentational. A withheld `a` now reports which gate rejected the window instead of silently showing nothing. Also in the photodiode plugin: - dark level is a measured setting with a capture action, applied to both the detector samples and the I_tot anchor so it cancels out of the complement instead of biasing it; `dark_id` names it honestly - phase-0 marker frames are written into the .pdq, so a recorded run stays phase-attributable offline - `save_cache_snapshot` copies the ring and releases the lock before writing the CSV, instead of blocking the reader across millions of writes - the UI mirror keeps the operator's connect intent rather than clearing it every control tick - a 0-byte read backs off instead of spinning a core - the spectrum max-hold seeds each bucket with its own first bin
…ol step `send_modulation` was silenced for an automation lease and a calibration sweep but not for a running protocol, which queues its steps into the same `pending` slot. Because the host re-applies the whole settings snapshot on every sync, any settings change — from this plugin or another — dropped the operator's armed drive on top of the protocol's queued step, and the board held it until the next step boundary. Also in the modulation plugin: - `mod_freq_mhz` is parsed once as f64 and rounded to millihertz; the second u64 parse returned None as soon as the firmware echoed a decimal, which published frequency_millihz: 0 and cost A1 its fallback modulation period - a leased `SetOpticalDepth` now parks the operator's armed depth and `end_lease` restores it, so the board no longer holds the last sweep point's depth after an A1 amplitude sweep finishes
…emoise the fold `S_p(t)` divided by the whole sensor (`width * height`) while `q_p` on the same screen divided by ROI area minus masked pixels, and the rolling numerator counted events from outside the ROI and from masked pixels. With a small ROI that under-reported `S_p` by the ROI/frame ratio, and the status readout printed both numbers under the same "valid pixels" label. The fold is now built from ROI-filtered events and both quantities divide by `valid_pixel_count()`. `current_fold()` is memoised on a fingerprint of its inputs. It is called from `rolling_dataset`, `latest_rolling` (twice), `current_windows` and `current_response`, each allocating a `Vec<FoldedEvent>` over up to MAX_EVENTS — a single repaint could allocate and discard hundreds of megabytes at bench event rates. Also in the A1 plugin: - the no-EventStore fallback trims by the analysis window instead of growing to MAX_EVENTS and then freezing on a stale buffer while the plots still looked live - a failed pilot-window freeze clears the previously loaded windows, so the sidecar cannot record an earlier pilot's windows as this run's
The photodiode's display mode used to select the optical geometry the published log-contrast was computed in, so a UI toggle changed a scientific quantity that A1's amplitude sweep settles against. ADR 012 records that geometry follows the bench, the measured dark level is applied to both sides of the complement (where it cancels), and a withheld `a` states its reason. Also documents the A1 `N_valid` definition (ROI area minus masked pixels, the same denominator `q_p` uses) and generalises the rule in architecture.md: a published field's meaning must not depend on the publisher's UI state.
…etry Ports the per-frequency exact-event-count depth lock (ADR 013, renumbered from 012 to clear the contrast-geometry ADR) onto the branch that carries the photodiode and modulation fixes, and repairs what that combination exposes. The lock closes `a_cmd ← a_cmd · a₀/a_measured` against the photodiode-measured log contrast. On its own branch that value's geometry followed the photodiode's *display toggle*, so under the default RAW mode it was the reject-port detector contrast rather than the excitation contrast — every locked depth would have been wrong. It is correct only together with the fixed geometry. Then the measurement itself: `a = ln(I_exc,max / I_exc,min)` is peak-to-peak, but the photodiode estimated it over a fixed 16 384-sample window — 0.82 s at 20 kSa/s, less than one cycle for every f < 1.2 Hz. Below one cycle the robust extrema see an arc of the waveform, so `a` comes out low and phase-dependent. That is exactly the sub-hertz plateau reference the A1 protocol normalises |H(f)| against, and the lock *divides* by it: a truncated estimate does not add noise, it drives the commanded depth up trial after trial until it rails at 6.0 or the detector clips. The contrast window is now sized from the phase-0 markers to cover CONTRAST_WINDOW_CYCLES whole cycles, floored at the old fixed window and capped by what the ring retains, and `a` is withheld outright below one cycle. The retained markers cannot measure a period longer than the ring — once the ring holds under a cycle it holds at most one marker — so the interval is remembered as markers go past instead of recovered from what survived eviction. `window_seconds` and `covered_cycles` join the optical summary (additive in V1). In the lock: - Find a₀ refuses up front when the published window is under one cycle at the current frequency, naming the cache length to raise. A1 always knows f, so this also covers an owner whose own markers cannot prove it. - the per-trial dwell is at least one estimator window, so a trial cannot average the depth it just replaced; the deadline grows with it - readings are spaced by half a window instead of per service_revision. Consecutive revisions share nearly their whole window, so three of them said no more than one - the trial value is the median and the spread is a stability gate: readings straddling a₀ abort the lock instead of locking onto a drifting drive - the clip warning threshold sits below the estimator's own refusal, where it can actually fire, instead of above it where it never could - a lock-table save failure is appended to the result instead of being overwritten by it
An A1 event-count block is 7-12 frequencies over two or three decades, each one a Find a₀ and a Record a₀ point, repeated over three blocks. ADR 013 left that manual because the ordering decisions are scientific — but they are also expressible, and the checklist wants them frozen in the session plan anyway. Every gap between the two presses was also a gap in which a modulation settings sync could re-apply the operator's own depth on top of the found one. Start frequency sweep runs the whole ladder on ONE modulation lease: per point it retargets the drive frequency, waits for the phase-0 trigger to confirm the new period, runs the unchanged a₀ lock, and records the unchanged event-count point. Both children gained an inherited-lease mode, so they run on the ladder's lease instead of taking their own — which is the substantive guarantee: the operator's drive settings are locked out from the first frequency to the last, so the amplitude provably cannot move between a lock and the point that replays it. `ModulationCommandV1::SetDriveFrequency` is the frequency counterpart of SetOpticalDepth (additive in V1, same scoping: leased only, re-derived through the owner's own drive_command, refused for a manual DAC or constant drive). The owner parks the operator's armed frequency on the first retarget and restores it with the depth when the lease ends, so a finished ladder does not leave the bench on its last point. Robustness, which is most of the work: - the trigger confirms the frequency, not the firmware ACK. A point starts only once enough phase-0 markers at the *new* period agree with the commanded one - retained markers and events are dropped on every frequency change: the measured period is their mean spacing, so keeping them would confirm the new frequency against a mixture of the old drive and the new - pilot windows are dropped with them. Windows frozen at one period are a phase interval of that period; carrying them across would score a point in the wrong window, silently, because a fold always produces something - the plan is validated before the drive moves. The photodiode estimates `a` over one window for the whole ladder, so its lowest frequency decides whether the ladder is measurable at all — checked at the button press, not at the ninth point two hours in - an unreachable a₀, an unconfirmed frequency or a failed recording skips that point and names it in the summary; the remaining decades are still recorded. A refused frequency carries the owner's own wording into the skip The schedule is data: log spacing (|H(f)| is read per decade), ascending / descending / alternating / seeded-random order, and an optional low-frequency reference interleaved every N points. Every point's sidecar gains a [frequency_sweep] section with the executed position, the order and the seed, so a block is interpretable from its files rather than from a notebook. See ADR 014.
Starting an A1 recording appeared to succeed and then reported a finished run
immediately. What landed was a config sidecar in the chosen output folder, a
truncated camera .raw in the host process's working directory, and no .pdq at
all. Three defects combined:
- Every photodiode-leg failure jumped straight to stop_camera, so the host had
been recording for a few hundred milliseconds and produced a stub RAW that
still carried a complete finalization receipt. Photodiode faults also set
stop_requested, conflating them with an operator stop. The camera now runs its
full duration and closes as a camera-only recording instead.
- The specific cause ('set the data directory first') was overwritten by the
generic 'was incomplete' on the way out. The first cause is now preserved and
named in the closing message.
- The RAW, the PDQ, and the sidecar are written by three owners against three
roots, and the host's relative output path resolved to its working directory.
Once both recorders report finalization their files are closed and hashed, so
A1 now gathers them into <output folder>/<id>/ and records the final paths.
PDQ receipts report a label relative to the photodiode data directory, which
the owner now publishes in its summary so the path can be resolved.
Also pre-flights the photodiode before starting the camera (not reporting, not
connected, no data directory, leased elsewhere), surfaces the blocker in the
status view while idle, and stops A1's own pipeline restart from wiping the
row's pilot windows, background floor, and collected response points.
Cherry-picked from fix/stage-a-a1-recording onto this branch, because the
frequency ladder records every one of its points through this coordinator:
without these fixes an unattended ladder would write a folder of truncated
RAWs and no PDQ at all. Its ADR is renumbered 012 to 015 — the third branch to
have claimed 012 independently.
Refs ADR 015, revises ADR 009 decision 3.
The gather-after-finalization from the previous commit left the photodiode's own Data directory in the critical path: an A1 run still failed when it was unset, and changing it mid-experiment could move files out from under a measurement. `PdqStartSpecV1` gains an additive `root_dir: Option<String>` — an absolute directory the client wants the recording written below, replacing the owner's configured data directory for that run. Every safety rule the owner already had survives below the new root (the path stays relative, `..` and non-normal components refused, parent components must be real directories rather than symlinks, the resolved target must stay below the root), plus the root itself must be absolute. An A1-driven run therefore no longer depends on the photodiode's Data directory at all, and the pre-flight stops checking it. The camera RAW still has to be gathered after finalization: the host resolves plugin recording paths below its own output directory and rejects absolute ones, and that rule lives in the other repository. The gather now also runs over the PDQ, where it is normally a no-op because the file was opened in place — which means a run that dies before finalization still leaves its PDQ in the measurement folder. PDQ receipts report the label the client requested, so A1 resolves it against the root it named, falling back to the owner's published `data_dir` and preferring whichever exists — an owner too old to honour `root_dir` still yields a correct path. Cherry-picked from fix/stage-a-a1-recording; revises ADR 015 decision 4.
# Conflicts: # plugins/stage-a-a1/src/runtime.rs # plugins/stage-a-modulation/src/lib.rs # plugins/stage-a-photodiode/src/lib.rs
# Conflicts: # docs/features/README.md # docs/features/stage-a-optical-waveform.md # plugins/stage-a-modulation/README.md # plugins/stage-a-modulation/src/lib.rs
Reworks the Stage-A bench across all four crates so a whole `q_p(a, f)` survey runs unattended and every recording carries its own provenance. A1 (ADR 020-023, 027, 028) - `a` comes from the photodiode (measured) or the commanded calibrated drive (open loop), and the source is recorded in every artefact - `Find a₀` and the lock table apply only to a measured depth; a commanded depth has nothing to search for - the frequency ladder becomes an outer loop over the amplitude sweep, one `a₀` point or a whole depth sweep per `f` - surveys run from a `.csv` or `.toml` protocol naming `ū`, `f` and `a` per recording; examples ship in `protocols/` and install with the plugin - die temperature, pixel dead time and illumination land in every sidecar, absent rather than `0` when the host cannot report them - host sensor telemetry is compacted column-wise into the measurement folder under the run's own name Photodiode and calibration (ADR 017, 019, 024) - the detector learns `I_tot` from the brightest reading it takes; the four anchor settings are gone and dark cancels in the complement - rail detection is span-relative, and a withheld `a` names its gate across the plugin boundary - the transfer sweep judges residual and hysteresis against its own measured noise, over the window it swept Modulation (ADR 025, 026) - `ū` and `a` clamp into the achievable range instead of reverting, so every mode stays selectable - the applied lobe crosses the UI-mirror/live-worker boundary through a process-global generation Gating fixes (ADR 018): A1 requires only the output folder; the measurement and `I_k` ids are provenance, not preconditions.
Both device owners cap the automation lease TTL they hand out at 60 s — a dead-man switch, and correct — but the clamp is silent: the request comes back `Applied`, so A1 believed it held the drive for a whole survey when it held it for a minute. Renewing once per point was therefore only ever right for points shorter than the cap. The shipped example protocol has a 40 s row, every row also pays the camera and photodiode handshakes, and `acquire_photodiode` asks for `duration_s + 60 s`, so any longer recording outlived its own leases. Past the deadline the modulation owner does what an expired lease must do — STOP, output off — and that single event surfaced as three unrelated-looking faults at once: the owner refusing every later command for want of a lease, the sidecar refusing to write because the photodiode had no fresh optical summary, and a "no trigger signal" line that read exactly like an unplugged EXT_TRIGGER cable but was the drive being off. A1 now heartbeats both leases against `expires_at_unix_ms` from the owner's own snapshot — which both owners already published and A1 never read — renewing once less than 20 s of the granted window is left, no more often than every 2 s. The owners' cap is untouched: raising it to survey length would fix the symptom by deleting the safety property that motivated it. Also fixes `on_discontinuity` asking `recording.is_active() || sweep.is_some()` to decide whether a SourceChanged was self-inflicted. Starting and stopping the host recorder raises it twice per recording, and between two points of a protocol or a ladder neither is true — so the run's own boundary was treated as an idle-time reset and wiped the survey's pilot windows, background floor and response curve mid-run. It now asks `automation_active()`: the same set `request_stop` winds down. Refs ADR 029. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The compacted `<stem>.sensor.json` depends on a companion CSV the host writes only when its own "Record sensor monitoring" switch is on. A1 cannot set that switch and cannot query it, so an absent readout was indistinguishable from a camera with no monitoring block — and the sidecar field's own doc comment said exactly that, which is what made the absence so hard to trace. A finished run that produced no readout now says so in the panel and names the switch, instead of leaving a survey to discover months later that it kept none of its bench conditions. The single-point readings in `[sensor]` ride the context bus and are unaffected either way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Installing a plugin required a Rust toolchain, a sibling augur-rs checkout and a working cargo, which made every measurement PC a development machine. Build all runtime plugins on each pull request and each push to main for macOS arm64/x86_64, Linux x86_64 and Windows x86_64, staged in the exact layout ~/.augur/plugins/ expects, so installing is a copy. main also publishes a rolling plugins-latest release, because workflow artifacts need a login and expire after 90 days while a bench should be able to curl a URL. The workspace depends on the host by path, so the job lays out two sibling checkouts. build-runtime-plugins.sh patches a git source whenever it finds a sibling augur-rs checkout; with path deps that patch matches nothing but still costs a fetch, so the checkout's .git is dropped right after cloning. CI calls the repository's own build and install scripts instead of restating the install layout in YAML — those scripts already own plugin discovery, library naming, A1's protocols folder and the macOS install-name rewrite. Plugins are dlopened into the host process, so pin rust-toolchain.toml to the same 1.95.0 augur-rs pins and read the channel out of that file rather than naming a version in the workflow. Every bundle carries a BUILD-INFO.txt with the augur-rs revision and rustc version behind it, which is what makes an ABI mismatch reported from the bench answerable.
Two defects the first run exposed, both independent of the plugin sources. The Linux job borrowed augur-rs/.github/scripts/install-linux-deps.sh from the host checkout to avoid keeping a second dependency list. That script does not exist on every augur-rs revision the job can be pointed at, so the Linux build failed on the value of augur_rs_ref rather than on anything in this repository. It was also a superset: it installs the X11/Wayland/GL stack for the GUI, which no plugin crate links. Install what the plugins actually need instead — pkg-config and libudev-dev for serialport. setup-rust-toolchain injects RUSTFLAGS="-D warnings" by default. That is right for a lint job and wrong for one that ships artifacts: a dead-code warning in one plugin would have denied the bench a bundle for all of them. Lint gating belongs in its own job. Also record what the run proved about the repository itself: these plugins do not compile against augur-rs main, which lacks the TableSchema, host-view and dataset-descriptor API they use.
augur-rs main still has a two-field TableSchema and no Scatter3dFromTable, HostDatasetDescriptor.relations/display or HostViewRegistry.actions, all of which the plugins in this repository already use. Defaulting AUGUR_RS_REF to main is therefore a guaranteed red build that never hands the bench a bundle. Default to the open host branch that does carry the API instead, and record the coupling in the brief and the ADR. BUILD-INFO.txt already names the exact host ref and SHA behind every library, so this stays visible rather than becoming folklore. Move the default back to main in the same commit that the host API lands there.
The eveSMLM chain expressed its stage dependencies directly: fitting depended on the candidates crate, post-processing on the fitting crate. Plugin crates are cdylibs that each export augur_plugin_vtable, so linking one plugin's rlib into another pulled that symbol in twice. Apple's linker tolerates the duplicate. rust-lld and MSVC's link.exe do not: rust-lld: error: duplicate symbol: augur_plugin_vtable LNK2005: augur_plugin_vtable already defined … fatal error LNK1169 That went unnoticed for as long as the only build machine was a Mac. The first CI run on four platforms found it: macOS produced a complete bundle while Linux and Windows failed to link, which also denied the bench a Windows bundle for the Stage-A plugins, since the build is all-or-nothing. Move everything that crosses a stage boundary into evesmlm-types, a plain library crate that exports no vtable — the wire contract plus the current-localization dataset and registry builders that both fitting and post-processing publish. Plugin-private state stays with its plugin: the candidate tracker's TrackedCluster moves back into the candidates crate. Each plugin still re-exports the names it used to own, so downstream use paths keep compiling. This generalizes what stage-a-plugin-contract already does for the Stage-A owners, and replaces the repo convention that shared types belong in the producing plugin's crate.
Port discovery filtered candidates by the two Unix name patterns (`cu.usbmodem`, `ttyACM`) before probing. Windows names no device — every port is `COMn` — so an attached, correctly driven Teensy was filtered out before any probe could run, and both plugins reported "no USB serial device found (looked for usbmodem/ttyACM)": the two things Windows cannot produce. Move the filter into `stage-a-io::transport::candidate_ports()`, where it is platform-aware: the callout node on macOS, `ttyACM*` on Linux, and every USB-classified port on Windows, falling back to the whole list when the OS classifies nothing. What identifies the device is still the probe (HELLO on the command port, PDA1 frames on the stream port); the filter only keeps probes off unrelated ports. Also open every port with DTR asserted. macOS and Linux do this implicitly, Windows does not, so a sketch gating on `if (Serial)` would stay silent even once the right port was found. The filter existed in four places in two implementations; it is now one function with unit tests covering both platform branches, and the failure message names the ports the OS actually enumerated. Refs ADR 032
Excel's "CSV UTF-8" — the obvious save format on a Windows bench — writes a UTF-8 byte-order mark. Unstripped it becomes part of the first header cell, so `mean_u` stops matching `mean_u` and the protocol is refused for missing a required column that is plainly there. The TOML form fails its parse outright. Neither message points at an invisible character. Strip the BOM once for both readers. CRLF was already handled by `str::lines()`; it now has a test so it stays that way. Refs ADR 027
Formatting drift left by the two previous commits, split out so the change that follows is only the change it claims to be. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A protocol run came off the bench as a folder of files that could not be told apart: same duration on every row, stems differing only in the second the row started. Mapping a .pdq back to the modulation it was recorded under meant sorting by timestamp and counting rows against the protocol file — a join that is only correct if no row was skipped, and a refused point is deliberately non-fatal so an overnight run finishes. A protocol is the one recording path where nothing in the panel is armed at anything that separates its recordings, and three places relied on exactly that: - The file stem took the empty `sweep_tag` branch, which is filled only by the amplitude sweep and the event-count workflow. `ProtocolPoint::tag()` already produced `u500m_f10Hz_a800m` and had a test, but nothing ever called it. It is now appended after the timestamp, behind a 1-based point index whose width follows the point count, so the files sort in protocol order rather than by u-bar. - The A1 sidecar had a section for every other automatic path and none for this one, so a row's `[sweep]` carried only the panel's min_a/max_a and the file did not record what the row had asked for on any axis. It gains `[protocol]`. - The recorder metadata described the drive as a frequency and two DAC codes, which name neither the lobe nor the operating point. u-bar is a whole axis of the survey and appeared nowhere. It now carries the waveform, the optical target, requested/resolved u-bar, the commanded depth, V_null/V_peak and the Pockels calibration id — all of it already published by the modulation owner. The metadata matters most, because the A1 sidecar is not guaranteed to exist: `write_sidecar` refuses without a fresh optical summary, which is what an expired lease or a drive switched off mid-row looks like after the fact. Such a run keeps only its .pdq and _pd.json, so the _pd.json has to stand alone. Free text out of the protocol file is clamped before it enters the map — the photodiode owner refuses the whole recording over an oversized value, and a chatty block name must not cost a run. What is written is the request; `[modulation]` says what the drive reported back and `[optical]` what the photodiode measured, so a row that missed its point is still distinguishable from one that hit it. Refs ADR 033. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
muthmann
changed the title
Every A1 run long enough to walk away from — the depth sweep, either mode
of the frequency ladder, an a₀ point, a protocol — now states its remaining
time on the button press and keeps stating it on the status pane:
Estimated time: ≈ 41 min left of ≈ 1 h 5 min — done by 03:41 UTC
Only the protocol answered this before, once, from the sum of its rows'
duration_s + settle_s. That number is too small — it omits the camera and
photodiode handshakes, the drive settling at a new depth, the four marker
periods a new frequency is confirmed over and the a₀ lock's trials — and it
goes stale the moment the next per-point message overwrites it.
So the estimate is the plan's own seconds scaled by the pace the run is
actually keeping: the plan alone (plus a fixed handshake allowance) until
the first point finishes, saying so in words, and re-scaled by every
finished point after that. The pace is clamped so one point that spent its
timeouts cannot extrapolate a night onto the rest. It is deliberately not
the lease TTL, which is a worst case and would quote half an hour for a
five-minute sweep.
One line, for the outermost run: a ladder's estimate already covers the
depth sweep inside it, and a protocol covers both.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mainand 0 behind. Only thetop two are from the latest session; the rest is the Stage-A plugin stack that has
been accumulating unmerged (A1 orchestration, photodiode, modulation, declarative
protocols, CI bundles, the evesmlm types crate). Merging this brings all of it in.
If only the recent fixes are wanted, they can be cherry-picked onto a narrow
branch instead.
The most recent change
fix(stage-a-a1): make every protocol row name itselfA protocol run came off the bench as a folder of files that could not be told
apart: same duration on every row, stems differing only in the second the row
started. Mapping a
.pdqback to its modulation meant sorting by timestamp andcounting rows against the protocol file — a join that is only correct if no row
was skipped, and a refused point is deliberately non-fatal so an overnight run
finishes.
A protocol is the one recording path where nothing in the panel is armed at
anything that separates its recordings. Three places relied on exactly that:
<id>_<ts>only —sweep_tagis filled only by the sweep buttons…_p03_u500m_f10Hz_a800m_config.toml[protocol]section_pd.jsonmetadataprotocol_*keys and waveform, optical target, requested/resolvedū, commanded depth,V_null/V_peak, calibration idProtocolPoint::tag()already producedu500m_f10Hz_a800mand had a test —nothing ever called it.
The metadata matters most, because the A1 sidecar is not guaranteed to exist:
write_sidecarrefuses without a fresh optical summary, which is what an expiredlease or a drive switched off mid-row looks like after the fact. Such a run keeps
only its
.pdqand_pd.json, so the_pd.jsonhas to stand alone.What is written is the request;
[modulation]says what the drive reported backand
[optical]what the photodiode measured, so a row that missed its point staysdistinguishable from one that hit it.
style(stage-a): apply rustfmt to the port-discovery testsFormatting drift left by the two preceding commits, split out so the change above
is only the change it claims to be.
Verification
cargo fmt --checkclean, no new clippy findings, all workspace tests pass(131 in
stage-a-a1, 5 of them new): the stem tag including two rows beingdistinguishable, the
_pd.jsonkeys plus the photodiode owner's 64-key bound,the
[protocol]section, the drive description, and that a hand-driven recordinginvents no
protocol_*keys.Docs per DoD: ADR 033, the A1 feature brief, and the feature index.
🤖 Generated with Claude Code