Stage-A A1: consolidate the three A1 branches, fix the a₀ measurement, add the frequency ladder

CONTRIBUTING.md

@@ -137,6 +137,18 @@ Plugins do not render `egui` directly. Instead, expose:
 
 The host owns rendering, export, caching, and window state for declared host views.
 
+When a table dataset should participate in the linked investigation workspace, also populate the additive metadata the host can use:
+
+- `coordinate_space_2d`
+- `coordinate_space_3d`
+- `row_id_column`
+- `time_column`
+- `layer_id`
+- `semantic_label`
+- `HostDatasetDescriptor.display`
+
+Prefer structured datasets for selection/linking and use overlays only for supplemental 2D annotations or hit-testing.
+
 ### 7. Write `plugin.toml`
 
 Use the runtime format:
@@ -160,6 +172,14 @@ cp plugins/my-plugin/plugin.toml ~/.augur/plugins/my-plugin/
 cp target/release/libaugur_plugin_my_plugin.dylib ~/.augur/plugins/my-plugin/
 ```
 
+On macOS, either run `./scripts/install-built-plugins.sh --profile release` instead of the manual
+copy steps or rewrite the installed dylib id yourself:
+
+```bash
+install_name_tool -id "@loader_path/libaugur_plugin_my_plugin.dylib" \
+  ~/.augur/plugins/my-plugin/libaugur_plugin_my_plugin.dylib
+```
+
 Then open `augur-gui`, go to **Plugins**, click **Scan for New Plugins**, and enable the plugin.
 
 ## Migrating Older Plugins

Cargo.toml

@@ -1,5 +1,10 @@
 [workspace]
 members = [
+  "stage-a-io",
+  "stage-a-plugin-contract",
+  "plugins/stage-a-a1",
+  "plugins/stage-a-modulation",
+  "plugins/stage-a-photodiode",
   "plugins/localization",
   "plugins/reconstruction",
   "plugins/focus-metrics",
@@ -25,3 +30,5 @@ egui = "0.27"
 rustfft = "6"
 serde = { version = "1", features = ["derive"] }
 serde_json = "1"
+serialport = "4"
+stage-a-plugin-contract = { path = "stage-a-plugin-contract" }

README.md

@@ -25,23 +25,42 @@ Use this repository for the plugin implementations, template crate, and repo-loc
 ## Runtime Model
 
 - Each plugin ships as a `plugin.toml` manifest plus one platform library (`.dylib`, `.so`, or `.dll`).
-- `augur-gui` discovers plugins from `~/.augur/plugins/`, loads the exported `augur_plugin_vtable`, and renders settings, status, and host views through the host.
+- `augur-gui` discovers plugins from `~/.augur/plugins/`, loads the exported `augur_plugin_vtable`, and renders settings, status, and linked investigation datasets/views through the host.
 - Host-owned built-in tools stay in `augur-gui`; they are not runtime plugins in this repository.
 - Host-owned experiment settings such as pixel scale, sensor geometry, acquisition time, and EventStore budget are published to plugins as `GlobalSettings` on `augur.global_settings`.
 - Standard shared scientific payloads can also live in companion crates such as `augur-plugin-types`.
 
+## Investigation Workspace Contract
+
+The host now owns a generic linked workspace across:
+
+- 2D preview
+- 3D inspection
+- host-rendered tables
+
+For plugins, that means:
+
+- structured datasets are the primary linking mechanism
+- stable row ids should be provided when possible
+- 2D/3D coordinate metadata should be declared when the plugin has it
+- layer/display metadata should describe visibility, color, marker shape, and size
+- overlays are supplemental annotations, not the primary integration surface
+
 ## In-Tree Runtime Plugins (work in progress)
 
 The plugin crates under `plugins/` are under active development and not yet ready for external use. The template crate and documentation are stable references for writing your own plugins.
 
 | Plugin | Phase | Notes |
 |---|---|---|
 | `localization` | `RawEvents` | Wavelet/Gaussian SMLM localization and standard `LocalizationResults` output |
-| `reconstruction` | `DerivedData` | Accumulated localization table plus host-rendered reconstruction windows |
+| `reconstruction` | `DerivedData` | Accumulated localization dataset with stable ids, time metadata, density rendering, and 3D inspection |
 | `focus-metrics` | `DerivedData` | Focus metrics from localization results or FFT preview sharpness |
-| `evesmlm-candidates` | `RawEvents` | Event-domain candidate clustering for eveSMLM |
-| `evesmlm-fitting` | `DerivedData` | Candidate fitting plus EVE and compatibility localization outputs |
-| `evesmlm-postproc` | `DerivedData` | Filtering, drift correction, evaluation, and the later EVE compact view provider |
+| `evesmlm-candidates` | `RawEvents` | Event-domain candidate clustering plus accepted/rejected raw-event investigation layers |
+| `evesmlm-fitting` | `DerivedData` | Candidate fitting plus shared current-localization datasets, stable ids, and linked 3D inspection |
+| `evesmlm-postproc` | `DerivedData` | Filtering, drift correction, evaluation, and the later shared EVE current-localization provider |
+| `stage-a-modulation` | control service | Sole owner of the Stage-A Teensy command port and ACKed modulation state |
+| `stage-a-photodiode` | control service | Sole owner of the Stage-A stream port, PDA1 ingestion, and PDQ persistence |
+| `stage-a-a1` | `RawEvents` + orchestration | A1 protocol/schedule validation, raw phase quicklooks, analysis core, and a safety-gated commissioning run through the two owner services |
 
 `plugin-template/` is the starting point for new plugin crates.
 
@@ -62,6 +81,8 @@ cp target/release/libaugur_plugin_localization.dylib ~/.augur/plugins/localizati
 ```
 
 On Linux, copy the `.so`. On Windows, copy the `.dll`.
+On macOS, prefer `./scripts/install-built-plugins.sh --profile release`; it rewrites the copied
+plugin dylib id so Plugin Manager reloads do not keep pointing at Cargo's build tree.
 
 Then open `augur-gui`, go to **Plugins**, click **Scan for New Plugins**, and enable the plugin.
 
@@ -101,7 +122,8 @@ The current authoring flow is:
 2. export the vtable with `export_plugin!`
 3. choose `input_kind()` and optional `PluginCapabilities`
 4. use `HostContext` for shared payloads, companion crates such as `augur-plugin-types` for reusable payload types, and `CTX_GLOBAL_SETTINGS` for host-owned calibration/settings
-5. declare host-rendered outputs with `host_views()` when needed
+5. declare host-rendered outputs with `host_views()` when needed and populate stable-id / coordinate / layer metadata when the dataset should participate in linked investigation
+   - to expose interactive operations, append `HostActionDescriptor`s to `HostViewRegistry.actions` (scope `Dataset`/`Row`/`Cluster`, optional `param_schema`); consume requests from the persistent context key `CTX_INVESTIGATION_ACTION_REQUESTS`
 6. build a `cdylib`
 7. install `plugin.toml` plus the compiled library into `~/.augur/plugins/<name>/`
 

docs/adr/005-investigation-workspace-datasets.md

@@ -0,0 +1,43 @@
+# ADR 005: Expose Generic Investigation Datasets From Plugins
+
+## Status
+
+Accepted
+
+## Context
+
+`augur-gui` now owns a generic linked investigation workspace across 2D preview, 3D inspection, and host-rendered tables.
+
+That host model depends on richer plugin-side dataset metadata than the older window-centric host-view integration used:
+
+- stable row ids
+- optional 2D and 3D coordinates
+- optional time columns
+- layer ids and display metadata
+
+The eveSMLM pipeline also needs stage-local investigation surfaces for tuning, especially at the candidate-finding stage where researchers need to compare accepted and rejected raw events directly.
+
+## Decision
+
+Plugins in this repository will align to the investigation workspace through generic structured datasets.
+
+Rules:
+
+1. Use table datasets as the primary linking surface for inspectable scientific outputs.
+2. Provide `row_id_column` when the plugin can produce stable ids.
+3. Provide `coordinate_space_2d`, `coordinate_space_3d`, and `time_column` when the data supports linked 2D/3D inspection.
+4. Use `layer_id` plus `HostDatasetDescriptor.display` for visibility and styling defaults.
+5. Keep intentionally shared dataset/view ids byte-for-byte identical across providers.
+6. Use overlays only for supplemental 2D annotation or hit-testing, not as the primary data contract.
+7. When one stage needs multiple logical layers, publish separate datasets/layer ids instead of keying style by plugin name.
+8. It is acceptable for multiple rows to share the same stable row id when the intended interaction is "select the whole cluster" rather than "select one raw sample".
+9. Stable row keys are dataset-scoped in the current host, so matching row ids across different datasets do not create cross-dataset selection on their own.
+
+## Consequences
+
+- the host can keep selection, styling, and filtering generic
+- candidate-finding can expose accepted and rejected raw events as separate investigation layers
+- candidate centroid overlays can select whole raw-event clusters by reusing `cluster_id` as the stable row key for accepted events
+- fitting and post-processing can safely reuse the same current-localization ids without breaking host linking
+- fitting can expose rejected fits as a first-class investigation dataset instead of hiding them behind aggregate counters
+- plugins carry a little more schema metadata, but avoid plugin-specific host hooks

docs/adr/005-stage-a-device-ownership.md

@@ -0,0 +1,47 @@
+# ADR 005 — Stage-A device ownership and the `stage-a-io` boundary
+
+- **Status:** Accepted
+- **Date:** 2026-07-13
+- **Amended by:** ADR 006 and ADR 007
+
+## Context
+
+The Stage-A camera calibrations (A1–A3) drive a Teensy stimulus/DAQ
+controller over USB serial while recording the event camera. Someone has
+to own the serial port, the experiment state machines, and the safety
+rules. The knowledge-base control-software spec fixes the boundary:
+AugurRs stays a generic camera recorder and plugin host and must not gain
+laboratory-instrument abstractions.
+
+## Decision
+
+1. **Device control lives in removable protocol plugins.** The permanent
+   command-port and stream-port owners are now `stage-a-modulation` and
+   `stage-a-photodiode` (ADR 006/007). Experiment workflows such as A1/A2/A3
+   orchestrate those owners through the host service plane and do not open the
+   ports themselves.
+2. **A shared plain-Rust library `stage-a-io`** (this repo, not a plugin)
+   owns everything protocol-shaped: PDA1 framing + CRC resync, the ASCII
+   command grammar with idempotent sequence retries, the bounded I/O
+   worker, `.pdq` persistence, the run sidecar, and the calibrated optical
+   contrast estimator. It contains **no experiment policy** (sweeps,
+   bisection, fits stay in the plugins) and **no augur types** (testable
+   without a host).
+3. **Effects are gated by the host's execution context** (plugin ABI v5):
+   plugins fail closed unless `LiveCapture && effects_allowed`. Hardware
+   commands are host actions, never persistent settings.
+4. **Wire compatibility is anchored to the firmware header**
+   (`stage-a-controller/include/wire_protocol.h`); `stage-a-io` mirrors it
+   with layout tests, and the mock controller implements the same
+   idempotency contract the firmware promises.
+
+## Consequences
+
+- A1/A2/A3 reuse the owner services and serde-only contracts; they may reuse
+  hardware-free `stage-a-io` parsing/analysis but not its serial transports.
+- The GUI knows nothing about Teensys; removing the three plugins removes
+  every trace of lab hardware from the product.
+- Protocol changes must land in the firmware header first, then in
+  `stage-a-io`, keeping a single source of truth for the wire format.
+- Plugins depend on `stage-a-io` by path; it is versioned with the
+  workspace and its API may still move until A2/A3 land.

docs/adr/006-stage-a-two-plugin-split.md

@@ -0,0 +1,52 @@
+# ADR 006 — Stage-A simplification: two plugins, one serial port each
+
+- **Status:** Accepted
+- **Date:** 2026-07-15
+- **Amends:** ADR 005 (Stage-A device ownership)
+- **Amended by:** ADR 007 (persistent owners with host-routed orchestration)
+
+## Context
+
+The three commissioning plugins (`stage-a-monitor`, `stage-a-funcgen`, `stage-a-a1`, ~3100 lines)
+bundled experiment state machines, contrast estimation, and drive control into UIs that were too
+complex and opaque for the current bench stage. What the bench actually needs now is:
+
+1. direct, immediate control of the laser modulation output (capped power slider,
+   constant/sine/square with frequency), and
+2. a plain readout of the photodiode (raw, or inverted to excitation power).
+
+Both need the same Teensy, but ADR 005 fixes one owner per serial port — and a context-bus
+coupling (one plugin republishing data for the other) would make the readout depend on the
+control plugin's connection.
+
+## Decision
+
+1. **The firmware enumerates two USB CDC ports** (`USB_DUAL_SERIAL`, `stage-a-controller`
+   ADR 002): port 1 keeps the v1 command protocol; port 2 free-runs the PDA1 photodiode
+   stream. ADR 005's rule is unchanged — one owner per port — there are simply two ports now.
+2. **Two minimal plugins replace the three commissioning plugins** (deleted 2026-07-15, retained
+   in git history):
+   - `stage-a-modulation` owns the command port (`docs/features/stage-a-modulation.md`);
+   - `stage-a-photodiode` owns the stream port (`docs/features/stage-a-photodiode.md`).
+3. **`stage-a-io` stays** as the protocol library (wire format, client, worker, firmware-faithful
+   mock — the mock now models firmware 0.3.0's `MOD` verb). Owner plugins use its transport/PDQ
+   pieces. A1/A2/A3 do not open transports; they use the host-routed owner contract (ADR 007)
+   and may use hardware-free parsing/analysis helpers.
+4. **Immediate transfer replaces the Apply-action pattern**, and **all device control is
+   settings-driven** (connect checkbox, slider changes sent as they happen). Host actions and
+   the per-frame effects gate are unsuitable here: the host only runs `process_frame()` while
+   camera frames flow, but the bench must work with no camera attached (amended 2026-07-16).
+   Replay mode still disconnects the modulation plugin defensively. The firmware output is
+   set-and-hold. ADR 008's 2026-07-23 amendment separates Manual/Calibrated drive method from
+   waveform mode and makes `max_level` the universal DAC ceiling.
+
+## Consequences
+
+- Each owner plugin has a single hardware concern; the photodiode plugin uses
+  `stage-a-io`'s PDA1 parser and PDQ persistence without taking command-port ownership.
+- Both plugins work independently — either can connect, disconnect, or crash without affecting
+  the other.
+- Wire-protocol changes still land firmware-first (`stage-a-controller/include/wire_protocol.h`
+  and command grammar), then in `stage-a-io`'s client/mock.
+- The A1 min-depth workflow is rebuilt as an orchestrator on this stable two-owner
+  stack; it never becomes a third Teensy owner.

docs/adr/007-stage-a-owner-orchestration.md

@@ -0,0 +1,68 @@
+# ADR 007 — Persistent Stage-A owners with host-routed orchestration
+
+- **Status:** Superseded in part (2026-07-20) — the persistent two-owner model and
+  host-routed control plane still hold, but `stage-a-a1` no longer orchestrates the
+  A1 acquisition. It was reduced to a read-only live-analysis plugin (two
+  phase-folded quicklooks + the photodiode-measured `a`); leases, recordings,
+  protocol/schedule freezing, references/epochs, and the minimum-depth logistic fit
+  were removed. See [Stage-A A1 Analysis](../features/stage-a-a1.md).
+- **Date:** 2026-07-20
+- **Amends:** ADR 005 and ADR 006
+
+## Context
+
+The A1 workflow must coordinate laser modulation, high-rate photodiode capture,
+and camera recording. The earlier handoff proposed that `stage-a-a1` open both
+Teensy ports while armed. That would create a third hardware owner and duplicate
+the control/readout logic already maintained by the two manual plugins.
+
+The existing Augur frame context cannot solve this safely: it is available only
+inside `process_frame`, while device control and reference acquisition must also
+progress without camera frames. Augur also loads a GUI mirror and a live-worker
+instance of each plugin, so an effectful setting copied between both instances
+can make them compete for the same port.
+
+## Decision
+
+1. `stage-a-modulation` is permanently the sole command-port owner and source of
+   truth for requested and controller-ACKed modulation state.
+2. `stage-a-photodiode` is permanently the sole stream-port owner and source of
+   truth for PDA1 ingestion, integrity accounting, and PDQ persistence.
+3. `stage-a-a1` is an orchestrator and camera-analysis plugin. It never opens a
+   Teensy port and never creates a `PdqWriter`.
+4. Coordination uses Augur's frame-independent, worker-owned plugin service
+   plane. Requests are atomic semantic operations with stable plugin IDs,
+   request IDs, leases, run IDs, expected revisions, explicit success/rejection,
+   and bounded versioned snapshots. The host routes messages but contains no
+   Stage-A logic.
+5. Manual controls and automation share the same owner-side validation. A held
+   automation lease prevents competing manual mutations; a deliberate manual
+   override revokes the lease, becomes a visible workflow fault, and commands
+   output-off where safe.
+6. Camera start/finalize uses the allow-listed plugin-to-host recording command
+   contract. RAW and PDQ receipts are correlated by immutable run ID and actual
+   finalized paths; the workflow never claims filesystem atomicity.
+7. Raw photodiode arrays do not cross JSON. A1 consumes small live summaries and
+   parses finalized PDQ data for replayable scientific results.
+
+## Safety and synchronization
+
+- Only the canonical live-worker instances may effect hardware. GUI mirrors,
+  replay, and offline instances are fail-closed.
+- Duplicate request IDs return the original terminal response without repeating
+  an effect.
+- Lease expiry, replay transition, plugin disable, worker shutdown, or hard fault
+  revokes control and requests output-off/finalization.
+- Current PDA1 frames do not carry a shared modulation/configuration revision.
+  Ordered ACKs establish operational order, but scientific cross-port identity is
+  reported as `UNSYNCED` until firmware supplies a common epoch or marker.
+
+## Consequences
+
+- A1/A2/A3 can reuse the same owner services without duplicating serial code.
+- The manual plugins remain independently useful and testable.
+- ADR 005's statement that each experiment plugin owns the serial port no longer
+  applies to A1/A2/A3; exclusive ownership now belongs to the two device plugins.
+- ADR 006's two-port/two-owner split becomes the stable architecture instead of a
+  temporary commissioning simplification.
+