diff --git a/UI_AUDIT.html b/UI_AUDIT.html new file mode 100644 index 0000000..d6d1382 --- /dev/null +++ b/UI_AUDIT.html @@ -0,0 +1,762 @@ + + + + + +The Discarded Five-Sixths + + + + +
+

OpenStrap · edge · v0.9.26+57 · UI/UX audit

+

The Discarded Five-Sixths

+

+ The analytics engine returns six fields per metric. The interface renders one of them. + Every other problem in this audit is downstream of that. +

+
+ 41,588 lines of UI + 93 files + 40 screens + 5 parallel auditors + 0 golden tests +
+
+ +
+

Verdict

+
+

5.9 /10

+

+ Not bad — undifferentiated. The craft is well above the Flutter-health median: + real tokens, real null-honesty copy, a measured contrast test already in CI. What it is not + is this product's interface. It renders like a competent WHOOP clone sitting on top + of an engine that could beat WHOOP on the one axis WHOOP can't compete on: telling the truth + about what it doesn't know. +

+
+
+ +
+

The finding

+

Four auditors, working blind to each other, found the same wall.

+

+ Analytics hands the UI a Metric. It carries a value, a confidence, a tier + (AUTH / HIGH / ESTIMATE / RELATIVE), + the inputs it used, the drivers that moved it, and a note saying why it's absent. Five of those + six survive the trip to a pixel almost nowhere. +

+ +
+ + + + + + + + ANALYTICS + SCREEN + + + + UI BOUNDARY + + + + value + + + 72 + + + + confidence + tier + inputs_used + drivers + note + + + + + + + + + + + + + + + + + — + 102 times + +

+ The band's whole claim is that it never guesses.
The interface renders that claim as a dash. +

+
+ +
+
+

102

+

bare — literals in lib/ui. 1 call site of needMessageFromNote, the function that turns need_baseline:have=3,need=14 into a sentence a human can act on. detail_cards.dart:818

+
+
+

1 of 13

+

ArcGauge call sites pass confidence; it defaults to 1.0. Every sparkline, bar and time series has no confidence parameter at all. An ESTIMATE respiratory rate and an AUTH heart rate render as identical pixels. charts.dart:36

+
+
+

3

+

reads of Metric.note across all 41,588 lines. The reason a number is missing is computed, stored, shipped to the device — and then dropped one layer before the eye.

+
+
+

0

+

places where a drivers list is shown as an attribution. Oura's Contributors panel is the category baseline in 2026; this app computes something strictly better and never draws it.

+
+
+
+ +
+

By discipline

+

Where the points went.

+
+
+

Design system — tokens, type, components

+

6.5

+
+
+
+

UX craft — states, a11y, motion

+

6.0

+
+
+
+

Information architecture — nav, screens, flows

+

5.5

+
+
+
+

Data visualisation — charts, gauges, hypnogram

+

5.5

+
+
+
+

+ The design system scores highest because it exists: a 4 pt spacing scale at 93% adoption, + 753 AppColors and 615 AppText call sites, dark mode re-derived rather than + inverted. The failure is not absence of a system — it is two systems, both re-exported + from the same barrel, with 21 files importing both. design.dart:36 +

+
+ +
+

Ranked findings

+

What actually costs quality.

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
SevFindingWhereFix
P0Muted caption text fails WCAG AA — 2.71:1, 2.40:1 and 2.20:1 depending on surface. It is the app's caption and label voice, so it fails everywhere.theme.dart:159,165
223 uses		Two hex values. A contrast test harness already exists.
P0Zero reduced-motion support. 13 infinite .repeat() loops, including a 2.6 s breathing icon on every empty state that never stops.motion.dart:32
state_card.dart:43		One guard inside the DsMotion extension fixes all 34 caller files.
P0A 958-line design gallery ships in release builds, routed from Settings under a "Developer" heading.profile_screen.dart:747Wrap the route in kDebugMode.
P0No downsampling anywhere. A night of 1 Hz HR is ~30,000 chart points per frame, with an O(n) linear scan on every drag pixel.charts.dart:381,624Min/max bucketing to 2× pixel width. _bucketAvg already exists — promote it.
P0The Insights screen returns SizedBox.shrink() while loading and when empty. Its content silently does not exist.coach_cards.dart
:176,365,436,534		Skeleton, then StateCard with the have/need line.
P1Today's tiles push Sleep/Heart/Body — the same classes that are already tabs 1/2/3. Two live instances with divergent scroll and drill state; "back" means two different things depending on how you arrived.today_screen.dart:441
vs app.dart:236		Tiles switch tabs. Never push a destination that is also a tab.
P1The onboarding validator contradicts its own copy: the screen promises "leave a field blank and only that metric stays unknown — never guessed", then requires all four fields.profile_setup_screen
.dart:139 vs :193		Allow blanks. Gate Continue on sex alone.
P1No escape from pairing. Bond-refused and picker-cancelled both fall through to one generic "Couldn't pair". This is the drop-off cliff of a hardware app.pairing_screen.dart
:178,190		Distinct refused/cancelled states, plus a skip-to-shell path.
P1The app's most visible element ignores the type scale. BigStat and MetricCard override AppText.hero/display/metric; hero numbers render at 26, 27, 36, 40, 44, 46, 48, 52, 54, 64 and 72 px across screens, at 14 different letter-spacings.big_stat.dart:66
metric_card.dart:112		Promote metricXl…metricSm into AppText; delete the overrides.
P1A shadow chart library. Five line-chart implementations, three gauges, two hypnograms, two sparklines — and coach_render.dart's copies interpolate straight across null gaps, which is a lie in a null-honest app.coach_render.dart
coach_chart.dart		One engine each. ~400 lines net deletion.
P1A dual-axis chart normalises each series to its own min/max on a shared grid. Every crossing and convergence it draws is an artifact.coach_render.dart:234Delete. Two stacked charts, or index to a common base.
P1Journey — the richest view in the app at 1,300+ lines — is reachable through exactly one card that hides itself for the first 24 hours. New users never learn it exists.today_screen.dart:416Promote to a tab.
P223 Semantics against 236 onTap:. Charts are blanket-ExcludeSemantics. Confidence is a 7 px hue-only dot with no label.kit.dart:498
charts.dart:437		Semantic labels on every interactive surface; colour + word + shape for status.
P2Sub-44 pt tap targets on primary controls (34 pt segmented controls, 40 pt back button); 68 candidate sites.controls.dart:70
app_scaffold.dart:43		Enforce the floor inside Pressable so no call site can opt out.
P2Unbounded workout history built into a non-lazy ListView(children:), each row wrapped in its own animation ticker. One ListView.builder exists in the entire UI.workouts_screen.dart
:354,461		ListView.builder; stagger only the first viewport.
P2Hypnogram — the signature visual of any sleep app — has no minimum segment width (a 30 s wake epoch is sub-pixel on an 8 h night), no time axis, no scrub, and shouldRepaint => true.hypnogram.dart:99,123Clamp to ≥1.5 px, add hour ticks and scrub-to-read.
P2Four god-screens: 2,728 / 2,208 / 2,113 / 1,700 lines. Meanwhile ~5,300 bespoke day-view lines sit beside a 489-line generic one that already works.live_session · workouts
profile · today		Split by responsibility; route the rest through MetricScreen.
P2The app calls itself "OpenStrap" during onboarding and "Edge" once you're inside. Jargon leaks as primary labels: MS RMSSD, pNN50, SD1/SD2.welcome_screen.dart:365
spot_check_screen.dart:186		One name. Method notes hold the method names, not stat captions.
+
+
+ +
+

Ship blockers

+

Four things that would fail a store review or an accessibility complaint.

+
+
+ A11Y +

Caption text below AA contrast, app-wide

+

2.20:1 at worst against a 4.5:1 requirement, across 223 call sites. Two hex values in theme.dart.

+
+
+ A11Y +

No reduced-motion path

+

Users with vestibular disorders get a permanently pulsing empty state and a shimmer that never stops. Also a ticker burning battery on an idle screen.

+
+
+ REL +

Debug surface in production

+

A design gallery and a fake-route fixture reachable from user-facing Settings.

+
+
+ PERF +

30,000-point charts

+

Every sleep detail view rebuilds a 30k-segment path per frame and scans it linearly on each drag pixel. This is the jank users will describe as "the app feels cheap".

+
+
+
+ +
+

Target

+

What production grade looks like here.

+

+ Not "more polish". The target is an interface whose structure is the honesty guarantee, + where a fabricated number is unrepresentable rather than merely avoided. +

+ +
+
+

1

+

A dash is a bug. One MetricValue widget: value → note-derived reason ("4 of 14 nights") → device reason ("not synced since 2 pm") → a dash only as an assertion failure. Target: ≤5 literal dashes in lib/ui, down from 102.

+
+
+

2

+

Tier is an encoding, not a badge. ESTIMATE renders in lighter weight with a hatched band; RELATIVE plots against a zero-line that means "your normal"; confidence below 0.4 dashes the stroke. Enforced in the one chart engine so no call site can regress it. No yellow, no warning triangle — the way Apple Weather greys distant forecast days.

+
+
+

3

+

Drivers are the hero, not a tap-through. Three signed attributions under the score, always expanded — "HRV −18% vs baseline · sleep debt 1 h 40 m · late alcohol logged". This is Oura's Contributors panel, except causal, and the engine already computes it.

+
+
+

4

+

Day 0 shows a contract, not a skeleton. No zeros, no shimmer implying a number is imminent: "Readiness needs 3 nights. 1 done. First score Thursday morning" — alongside live raw HR, the only thing that is real on day 0 and the proof the band is talking.

+
+
+

5

+

One hero, not three dials. WHOOP's three-dial home forces the user to arbitrate. Readiness alone above the fold in the morning; strain-so-far against today's ceiling in the evening; everything else earns its way below. The AI briefing is not a co-hero — an LLM paragraph at the top makes a local-first app look like a wrapper.

+
+
+

6

+

The receipt. Every score links to its inputs: which records, which gaps, coverage in real units ("6 h 12 m of 7 h 40 m in bed"), which algo version. When a version change re-scores a past day, say so on the card instead of silently mutating history.

+
+
+

7

+

Lead with what WHOOP cannot do. A smart alarm that wakes you in light sleep, using live per-second staging, with a morning receipt — "woke you at 06:31, 4 minutes into light sleep". Plus a raw 1 Hz viewer and one-tap export. No subscription product can offer either.

+
+
+
+ +
+
+

Sequence

+

Four phases, ordered by perceived quality per unit of effort.

+
+ +
+

1

+

Stop the bleeding

+

Roughly a day. Almost entirely mechanical.

+ +
+ +
+

2

+

Ship the differentiator

+

The phase that changes what the product is.

+ +
+ +
+

3

+

Collapse the duplication

+

Net deletion. The diff should be negative.

+ +
+ +
+

4

+

Earn the feature slot

+

Where it stops looking like a hobby build.

+ +
+
+ +
+
+

+ The uncomfortable summary: the hard part is already built. On-device staging, corrected RR + intervals, personal baselines, per-metric confidence, honest nulls — years of work that no + competitor ships. The interface throws away five of the six fields that work produces, and + then looks like every other recovery app. +

+

+ Phase 1 is a day. Phase 2 is what makes it this app. +

+
+
+ + + diff --git a/ios/Runner/AccessorySetup.swift b/ios/Runner/AccessorySetup.swift index e057538..ee7c0a8 100644 --- a/ios/Runner/AccessorySetup.swift +++ b/ios/Runner/AccessorySetup.swift @@ -26,9 +26,13 @@ import AccessorySetupKit /// - `removeAll` -> nil (deprovision all — used on unpair) enum AccessorySetup { private static let channelName = "openstrap/accessory_setup" - // The WHOOP "Harvard" Gen4 GATT service (matches GattUuids.service in Dart). - // `fileprivate` so the iOS-18 Impl below can read it. - fileprivate static let whoopServiceUUID = "61080001-8d6d-82b8-614a-1c8cb0f8dcc6" + // WHOOP GATT service UUIDs, one per generation (match GattProfile in Dart). + // `fileprivate` so the iOS-18 Impl below can read them. BOTH must also be + // listed in Info.plist under NSAccessorySetupBluetoothServices. + // • gen4 ("Harvard", WHOOP 4) — 6108… + // • gen5 ("fd4b", WHOOP 5) — fd4b… (EXPERIMENTAL) + fileprivate static let whoopServiceUUIDGen4 = "61080001-8d6d-82b8-614a-1c8cb0f8dcc6" + fileprivate static let whoopServiceUUIDGen5 = "fd4b0001-cce1-4033-93ce-002d5875f58a" static func register(messenger: FlutterBinaryMessenger) { let channel = FlutterMethodChannel(name: channelName, binaryMessenger: messenger) @@ -136,30 +140,35 @@ private final class Impl { return } - let descriptor = ASDiscoveryDescriptor() // Match on the WHOOP custom service UUID alone. The foreground scan finds the - // band via startScan(withServices:[thisUUID]) and succeeds, which proves the - // band advertises this service — so it's a reliable, sufficient filter. Every - // descriptor criterion must be declared in Info.plist; the UUID is listed under - // NSAccessorySetupBluetoothServices. (No bluetoothNameSubstring: a single - // descriptor AND-combines its criteria, and a name filter would also require an - // NSAccessorySetupBluetoothNames entry and risk excluding the band on a name - // mismatch.) - descriptor.bluetoothServiceUUID = CBUUID(string: AccessorySetup.whoopServiceUUID) - - // Show the actual strap render in the ASK pairing sheet (asset catalog → - // StrapProduct.imageset). Fall back to an SF Symbol if the asset is missing. + // band via startScan(withServices:[…]) and succeeds, which proves the band + // advertises this service — so it's a reliable, sufficient filter. Every + // descriptor criterion must be declared in Info.plist; the UUIDs are listed + // under NSAccessorySetupBluetoothServices. (No bluetoothNameSubstring: a + // single descriptor AND-combines its criteria, and a name filter would also + // require an NSAccessorySetupBluetoothNames entry and risk excluding the band + // on a name mismatch.) + // + // ASK matches ANY item in the picker list, so we offer one item per WHOOP + // generation: gen4 (WHOOP 4) and gen5 (WHOOP 5, experimental). A band that + // advertises either service can be provisioned; the provisioned identifier is + // the same CoreBluetooth UUID regardless of generation. let productImage = UIImage(named: "StrapProduct") ?? UIImage(systemName: "sensor.tag.radiowave.forward") ?? UIImage() - let item = ASPickerDisplayItem( - name: "WHOOP band", - productImage: productImage, - descriptor: descriptor - ) + func item(_ serviceUUID: String, _ name: String) -> ASPickerDisplayItem { + let descriptor = ASDiscoveryDescriptor() + descriptor.bluetoothServiceUUID = CBUUID(string: serviceUUID) + return ASPickerDisplayItem( + name: name, productImage: productImage, descriptor: descriptor) + } + let items = [ + item(AccessorySetup.whoopServiceUUIDGen4, "WHOOP band"), + item(AccessorySetup.whoopServiceUUIDGen5, "WHOOP 5 band"), + ] pickerResult = completion - session.showPicker(for: [item]) { [weak self] error in + session.showPicker(for: items) { [weak self] error in guard let self = self else { return } if let error = error { if let cb = self.pickerResult { diff --git a/ios/Runner/Info.plist b/ios/Runner/Info.plist index 75b3d35..45d903a 100644 --- a/ios/Runner/Info.plist +++ b/ios/Runner/Info.plist @@ -54,6 +54,7 @@ NSAccessorySetupBluetoothServices 61080001-8D6D-82B8-614A-1C8CB0F8DCC6 + FD4B0001-CCE1-4033-93CE-002D5875F58A NSAccessorySetupKitSupports diff --git a/lib/ble/ble_engine.dart b/lib/ble/ble_engine.dart index e0fa3ef..2bb7caa 100644 --- a/lib/ble/ble_engine.dart +++ b/lib/ble/ble_engine.dart @@ -84,6 +84,52 @@ typedef ArchiveSink = Future Function(ArchiveRecord archive); /// trigger now that listening is continuous and there's no discrete sync end. typedef DataStoredSink = void Function(); + +/// Map a decoded gen5 historical record onto the band-agnostic `Sample` type, +/// or null when this record kind has no `Sample` equivalent (yet). +/// +/// Only `Gen5HistorySample` (v18, the per-second stream) maps today — the +/// deep buffers (`Gen5OpticalBuffer`/`Gen5ImuBuffer`/`Gen5PpgWaveform`, R22 +/// opt-in only) need their own raw-buffer storage, not a 1Hz `Sample`, so +/// they (and a null [g], e.g. an unrecognised version) correctly return null +/// here — the caller archives those, exactly like an undecodable gen4 +/// record. Extracted as a top-level pure function (rather than inlined in +/// `_ingestHistoricalFrame`) so the mapping is unit-testable without a live +/// BLE session — see `gen5_sample_mapping_test.dart`. +@visibleForTesting +Sample? sampleFromGen5Historical(Gen5HistoricalRecord? g) { + if (g is! Gen5HistorySample) return null; + return Sample( + tsEpoch: g.unix, + counter: g.recordIndex, + hr: g.heartRate, + rrIntervalsMs: List.from(g.rrIntervalsMs), + // Gravity vector is float32 g-units on BOTH generations (unlike skin + // temp / SpO2, which use gen5-specific scales/mechanisms — see + // Gen5HistorySample's field docs) — safe to feed straight into the + // shared ax/ay/az fields analytics already reads band-agnostically. + ax: g.gravityG.isNotEmpty ? g.gravityG[0] : null, + ay: g.gravityG.length > 1 ? g.gravityG[1] : null, + az: g.gravityG.length > 2 ? g.gravityG[2] : null, + // The band computes these itself, every second, whether or not a phone is + // listening — unlike our own step estimate, which only runs during a live + // session. gen4 carries none of them, so they stay null there rather than + // being stored as a zero that reads like a real measurement. + stepCount: g.stepMotionCounter, + stepCadence: g.stepCadence, + activityClass: g.activityClassKnown, // null for the unclassified code + skinTempC: g.skinTempC, + onWrist: g.onWristRaw, + hrValid: g.hrRrValidThisSecond, + hrAlt: g.heartRateAlt, + ); +} + +/// Decode a gen5 historical inner frame to a band-agnostic [Sample], or null. +@visibleForTesting +Sample? decodeGen5HistoricalSample(Uint8List inner) => + sampleFromGen5Historical(parseGen5Historical(inner)); + @visibleForTesting int countHistoricalBurstPackets({ required Map dataPacketCountsByRevision, @@ -296,11 +342,27 @@ class _SessionGapSummary { class _Session { final BluetoothDevice device; BluetoothCharacteristic? cmdTo; + + /// Which WHOOP generation this link speaks. Defaults to gen4 (WHOOP 4) and is + /// pinned once during service discovery via [applyBand] — everything that + /// differs by generation (frame header/CRC, GATT UUIDs, command envelope, + /// history ACK, record decode) reads from here. + BandProfile band = BandProfile.gen4; + final Map asm = { 'cmd_from': FrameReassembler(), 'events': FrameReassembler(), 'data': FrameReassembler(), }; + + /// Pin this session's generation and rebuild the reassemblers with the + /// matching header shape. Called once, at discovery, before any frame is fed. + void applyBand(BandProfile b) { + band = b; + asm['cmd_from'] = FrameReassembler(profile: b); + asm['events'] = FrameReassembler(profile: b); + asm['data'] = FrameReassembler(profile: b); + } final List subs = []; Timer? heartbeat; // Session-owned timers; a disconnect cancels them. @@ -381,6 +443,15 @@ class BleEngine { final Duration Function() deriveDataStaleness; final bool Function() isForegroundActive; + /// Opt-in: send the gen5 "R22" 16-flag SET_CONFIG enable sequence + /// (`kGen5R22EnableFlags`) before the historical offload on a gen5 link, + /// unlocking the v20 (optical)/v21 (IMU)/v26 (PPG) deep buffers. Defaults to + /// OFF — the sequence is + /// UNTESTED on physical hardware, and without it a gen5 strap still serves + /// its always-on v18 per-second stream perfectly well. Wire a caller-owned + /// settings read here to make it a real user-facing toggle. + final bool Function() gen5DeepBuffersEnabled; + BleEngine({ required this.onRecord, required this.onState, @@ -397,8 +468,11 @@ class BleEngine { this.isBackgroundDrainer = false, this.deriveDataStaleness = _defaultDeriveDataStaleness, this.isForegroundActive = _defaultIsForegroundActive, + this.gen5DeepBuffersEnabled = _defaultGen5DeepBuffersDisabled, }); + static bool _defaultGen5DeepBuffersDisabled() => false; + /// True for the headless restore-drain engine (runHeadlessSync). It YIELDS the /// band to a foreground engine rather than fighting it — see [_claimBand]. The /// foreground app engine leaves this false and always wins. @@ -714,23 +788,36 @@ class BleEngine { @visibleForTesting void debugInstallFakeLink({ required Future Function(Uint8List frame) onWrite, + BandProfile band = BandProfile.gen4, + ArchiveSink? onArchive, }) { final session = _Session( BluetoothDevice(remoteId: const DeviceIdentifier('AA:BB:CC:DD:EE:FF')), ); session.connected = true; session.sawConnected = true; + session.applyBand(band); _session = session; debugWriteHook = onWrite; _drain = DrainController( onRecord: _storeRecord, onRecordsBatch: null, onCommit: null, - onArchive: null, + onArchive: onArchive, log: _log, ); } + /// Feed one inbound historical frame through the real ingest path (decode → + /// plausibility gate → store or archive). + /// + /// The routing decisions this covers — which record versions decode, and + /// what happens to a record the gate rejects — are the difference between + /// banking a user's data and letting the band trim it away, and they sit + /// behind a radio otherwise. + @visibleForTesting + void debugIngestHistoricalFrame(Frame frame) => _ingestHistoricalFrame(frame); + /// Drive the canonical historical-refresh path. Returns whether /// SEND_HISTORICAL_DATA actually went out. @visibleForTesting @@ -1046,7 +1133,11 @@ class BleEngine { '[SPO2] hist=v$version count=$count base=inner ' 'whoop4_optical(red@64 ir@66 temp@68 amb@70) ' 'ts=${r.tsEpoch} red=${r.spo2RedRaw} ir=${r.spo2IrRaw} ' + // deprecated names, but this is a raw-bytes debug line — logging them + // under the label we've always used is the point. + // ignore: deprecated_member_use 'temp=${r.skinTempRaw} amb=${r.ambientRaw} ' + // ignore: deprecated_member_use 'ppg_green=${r.ppgGreen} ppg_red_ir=${r.ppgRedIr}', ); return; @@ -1197,7 +1288,10 @@ class BleEngine { await FlutterBluePlus.stopScan(); } _setPhase(BleConnState.scanning); - final svc = Guid(GattUuids.service); + // Advertise-filter on BOTH generations' service UUIDs (gen4 6108xxxx + + // gen5 fd4bxxxx); the actual generation is pinned later at discovery. + final gen4Svc = Guid(GattProfile.gen4.service); + final gen5Svc = Guid(GattProfile.gen5.service); BluetoothDevice? found; final sub = FlutterBluePlus.onScanResults.listen((results) { for (final r in results) { @@ -1207,14 +1301,16 @@ class BleEngine { ); if (found == null && (name.contains('whoop') || - advNames.any((s) => s.startsWith('61080001')))) { + advNames.any((s) => + s.startsWith('61080001') || s.startsWith('fd4b0001')))) { found = r.device; FlutterBluePlus.stopScan(); } } }); try { - await FlutterBluePlus.startScan(withServices: [svc], timeout: timeout); + await FlutterBluePlus.startScan( + withServices: [gen4Svc, gen5Svc], timeout: timeout); await FlutterBluePlus.isScanning.where((on) => on == false).first; } catch (e) { _log('scan error: $e'); @@ -1389,15 +1485,33 @@ class BleEngine { final services = await device .discoverServices() .timeout(_serviceDiscoveryTimeout); + // Pin the generation from whichever service the peripheral exposes: + // gen4 "Harvard" 6108xxxx, or gen5 "fd4b" fd4bxxxx. This drives the frame + // header/CRC, command envelope, ACK, and record decode for the session. BluetoothService? svc; + BandProfile band = BandProfile.gen4; for (final s in services) { - if (s.uuid.str.toLowerCase().startsWith('61080001')) svc = s; + final u = s.uuid.str.toLowerCase(); + if (u.startsWith(GattProfile.gen4.servicePrefix)) { + svc = s; + band = BandProfile.gen4; + break; + } + if (u.startsWith(GattProfile.gen5.servicePrefix)) { + svc = s; + band = BandProfile.gen5; + break; + } } if (svc == null) { - _log('Harvard service not found on device.'); + _log('No WHOOP service (gen4 6108xxxx / gen5 fd4bxxxx) found on device.'); await _failConnect(); return false; } + session.applyBand(band); + state.generation = band.isGen5 ? 'gen5' : 'gen4'; + _log('Detected ${band.isGen5 ? "WHOOP 5 (gen5)" : "WHOOP 4 (gen4)"} link.'); + final gatt = band.gatt; BluetoothCharacteristic? find(String prefix) { for (final c in svc!.characteristics) { if (c.uuid.str.toLowerCase().startsWith(prefix)) return c; @@ -1405,15 +1519,15 @@ class BleEngine { return null; } - session.cmdTo = find('61080002'); - final cmdFrom = find('61080003'); - final events = find('61080004'); - final data = find('61080005'); + session.cmdTo = find(gatt.cmdTo.substring(0, 8)); + final cmdFrom = find(gatt.cmdFrom.substring(0, 8)); + final events = find(gatt.events.substring(0, 8)); + final data = find(gatt.data.substring(0, 8)); if (session.cmdTo == null || cmdFrom == null || events == null || data == null) { - _log('Missing one or more Harvard characteristics.'); + _log('Missing one or more ${band.isGen5 ? "fd4b" : "Harvard"} characteristics.'); await _failConnect(); return false; } @@ -1621,8 +1735,14 @@ class BleEngine { // Re-arm ONLY what the current live mode wants: re-sending the high-rate // R10/R11 toggle while in HR-only mode (background downgrade) or under the // marginal-radio fallback would silently undo the downgrade every 30 s. + // gen5: 0x3F is Unknown/Unhandled — re-arm IMU instead when full live. + final isGen5 = _session?.band.isGen5 ?? false; if (!_liveHrOnly && !state.standardHrFallback) { - _send(Cmd.sendR10R11Realtime, const [0x01]); + if (isGen5) { + _sendToggleImu(true); + } else { + _send(Cmd.sendR10R11Realtime, const [0x01]); + } } _send(Cmd.toggleRealtimeHr, const [0x01]); } @@ -1755,7 +1875,7 @@ class BleEngine { _setOffloadActive(true); if (refreshRange) { _log('[SYNC] refresh($reason) — polling GET_DATA_RANGE before 0x16.'); - await _send(Cmd.getDataRange, const [0x00]); + await _sendGetDataRange(); // INIT spaces commands by ~120 ms; keep the same cadence here so the band // has time to emit the range response before we request another drain. await Future.delayed(const Duration(milliseconds: 120)); @@ -1797,7 +1917,7 @@ class BleEngine { // success anyway leaves the strap with no request, `_offloadActive` stuck // true — so later refreshes bounce off the "already transmitting" guard — // and both rate-limit floors spent on a command that never left the phone. - if (!await _send(Cmd.sendHistoricalData, const [0x00])) { + if (!await _sendHistoricalData()) { _setOffloadActive(false); return false; } @@ -2019,11 +2139,22 @@ class BleEngine { } Future _send(int opcode, List payload) async { - if (dangerousCmds.contains(opcode)) { + // `dangerousCmds` is this codebase's own gen4-curated hard-block list + // (FORCE_TRIM/REBOOT/POWER_CYCLE/TOGGLE_PERSISTENT_R21/firmware-load). + // `OpcodeSafety.destructive` is whoop-rs's independently-curated list of + // opcodes with NO legitimate use anywhere in EITHER codebase (142-144 + // have no named meaning at all) — the two don't fully overlap, so both + // apply. Deliberately NOT `OpcodeSafety.forbidden`: that broader list + // also flags opcodes this app sends ON PURPOSE via named, reviewed call + // sites (SET_ADVERTISING_NAME/SELECT_WRIST/SET_CONFIG for the R22 + // sequence/SET_CLOCK_MAVERICK) — see that class's own doc for why a + // blanket block on `forbidden` would be wrong here. + if (dangerousCmds.contains(opcode) || OpcodeSafety.isDestructive(opcode)) { _log('REFUSED dangerous opcode 0x${opcode.toRadixString(16)}'); return false; } - final frame = buildCommand(_seq.nextLive(), opcode, payload); + final frame = buildCommand( + _seq.nextLive(), opcode, payload, _session?.band ?? BandProfile.gen4); final ok = await _write(frame); if (!ok) { _log('WRITE FAILED for opcode 0x${opcode.toRadixString(16)} — ' @@ -2032,11 +2163,34 @@ class BleEngine { return ok; } + // Offload commands whose PAYLOAD (not just the frame envelope) is + // generation-specific: gen4 sends a single 0x00, gen5 sends an EMPTY payload. + // Centralised so every offload trigger — the initial handshake, periodic + // backfill, manual refresh, and retry — emits the correct gen5 format on a + // gen5 link. (_send already frames with the session's BandProfile.) + List get _offloadPayload => + (_session?.band.isGen5 ?? false) ? const [] : const [0x00]; + /// IMU_SET_DATA_STREAM for the session's band. gen5 wants a leading revision + /// byte where gen4 sends a bare on/off byte; protocol's `cmdToggleImu` owns + /// that split. Sent the gen4 body, a gen5 strap reads the state from past the + /// end of the body, the stream never arms, and step calibration stays at 0. + Future _sendToggleImu(bool on) => _write( + cmdToggleImu(_seq.nextLive(), on, + profile: _session?.band ?? BandProfile.gen4), + ); + + Future _sendGetDataRange() => + _send(Cmd.getDataRange, _offloadPayload); + Future _sendHistoricalData() => + _send(Cmd.sendHistoricalData, _offloadPayload); + Future applyHighFreqWakeWindow({ required bool enabled, required DateTime? targetWake, Duration duration = const Duration(minutes: 90), - int intervalSeconds = 60, + // 61, not 60: gen5 refuses an interval of 60 or less outright, so the + // round number is the one value that guarantees the mode never engages. + int intervalSeconds = 61, String reason = 'wake_window', }) async { if (_session?.connected != true) return; @@ -2054,13 +2208,22 @@ class BleEngine { '[SYNC] HighFreq enter ($reason) — interval=${intervalSeconds}s ' 'duration=${duration.inSeconds}s until=${targetWake.toIso8601String()}', ); - await _write( + // Frame for the SESSION'S band. Built gen4-only, a gen5 strap got a header + // length and checksum it cannot parse, so high-frequency sync never + // engaged — while the flags below claimed it had. Only claim the mode when + // the write actually landed. + final ok = await _write( cmdEnterHighFreqSync( _seq.nextLive(), intervalSeconds: intervalSeconds, durationSeconds: duration.inSeconds, + profile: _session?.band ?? BandProfile.gen4, ), ); + if (!ok) { + _log('[SYNC] HighFreq enter ($reason) write FAILED — mode NOT claimed.'); + return; + } _highFreqModeRequested = true; _highFreqReason = reason; _highFreqUntil = targetWake; @@ -2074,7 +2237,8 @@ class BleEngine { return; } _log('[SYNC] HighFreq exit ($reason).'); - await _write(cmdExitHighFreqSync(_seq.nextLive())); + await _write(cmdExitHighFreqSync(_seq.nextLive(), + profile: _session?.band ?? BandProfile.gen4)); _highFreqModeRequested = false; _highFreqReason = null; _highFreqUntil = null; @@ -2195,7 +2359,19 @@ class BleEngine { } else if (pt == PacketType.consoleLogs && _offloadActive) { _drain?.onBurstConsole(); } - final decoded = _maybeAugmentDataRange(frame, decodeFrame(frame)); + final band = _session?.band ?? BandProfile.gen4; + final decoded = _maybeAugmentClockEpoch( + frame, + decodeFrame(frame, profile: band), + ); + // gen5-only, debug-visibility ONLY (never persisted, never gated on): + // log the strap's own console text (now decoded by protocol's + // `parseConsoleLog`, wired into `decodeFrame` above). Genuinely useful + // for diagnosing the untested gen5 handshake/offload on real hardware. + if (band.isGen5 && decoded.kind == 'console_log') { + _log('[CONSOLE gen5] idx=${decoded.fields['record_index']} ' + 'ts=${decoded.fields['ts_epoch']}: ${decoded.fields['text']}'); + } _absorbState(decoded); } @@ -2267,6 +2443,32 @@ class BleEngine { /// path is deliberate: the previous duplicate had drifted, silently losing /// the plausibility gate and freezing the frontier the stuck-strap / /// auto-continue policies read. + /// Set a historical frame aside in `raw_archive` — the never-pruned store for + /// bytes this build could not fully turn into a [Sample]. + /// + /// Routed through the drain when one is active so the write lands inside the + /// SAME transaction as the batch commit (safe-trim invariant: nothing the + /// band is told it may trim has been discarded). + void _archiveHistoricalFrame( + Frame frame, + int counter, { + required String reason, + }) { + final archive = ArchiveRecord( + counter: counter, + hex: _innerHex(frame.inner), + packetType: frame.inner.isNotEmpty ? frame.inner[0] : 0, + capturedAt: DateTime.now().millisecondsSinceEpoch, + reason: reason, + ); + final d = _drain; + if (d != null) { + d.onUndecodableRecord(archive); + } else { + unawaited(onArchiveRecord?.call(archive) ?? Future.value()); + } + } + void _ingestHistoricalFrame(Frame frame) { final pt = frame.packetType; if (pt != PacketType.historicalData) return; @@ -2288,7 +2490,45 @@ class BleEngine { // backfill (all received in one sync) splits into correct per-real-day // buckets instead of collapsing into one "today". Sample? sample; - if (recType == Record.r24 || recType == Record.r12) { + final wallNow = DateTime.now().millisecondsSinceEpoch ~/ 1000; + final isGen5 = _session?.band.isGen5 ?? false; + if (isGen5) { + // gen5 (WHOOP 5): `parseGen5Historical` dispatches across all four real + // gen5 historical-record kinds (v18 per-second summary, v20 optical/ + // v21 IMU/v26 PPG deep buffers — R22 opt-in only). Only v18 maps onto + // the band-agnostic `Sample` type today; the deep buffers need their + // own raw-buffer storage (a future db table), not a 1Hz Sample, so they + // fall through to the undecodable archive below — that is honest + // (correctly-identified-but-not-yet-stored), not a decode failure. + sample = decodeGen5HistoricalSample(frame.inner); + } else if (recType != Record.r25 && + kKnownRecordVersions.contains(recType)) { + // Every gen4 layout version protocol has a field map for — not just + // v24/v12. v7/v9/v18 decode fine through the same chain and used to fall + // through to `undecodable_rec_v*` purely because this branch never routed + // them. gen5 also ships a v18 with a completely different layout; the + // `isGen5` branch above claims it first, so this is gen4 only. + // + // v25 is EXCLUDED on purpose and keeps going to the archive, exactly as + // before. Not because the decode is wrong — it is right, and checked + // against 20k of these records: the timestamp at inner[7] is monotonic + // and steps by exactly 1 s, and the gravity vector reads a mean |g| of + // 0.97 with 19999/20000 inside the plausible window. + // + // The record genuinely has NO heart rate. Every byte and u16 offset was + // scanned for anything in a bpm range with physiological drift across + // 18k consecutive one-second pairs; the only matches are the packet type, + // the timestamp's high bytes and a gravity high byte. What sits between + // is a 24-slot raw waveform buffer spanning the full i16 range — the band + // ships samples here and computes HR into the v24 records instead. + // + // So `_parseV25` reporting `hr: 0` is an honest absence. The collision is + // downstream: `decoded_onehz.hr` is NOT NULL and `hr == 0` is this app's + // off-skin sentinel, so banking v25 would assert "the band was off the + // wrist" for every one of those seconds — ~50k in one real export — while + // the genuine gravity keeps the accel-coverage gate happy with the + // window. The gravity IS worth having; it needs a nullable `hr` column + // first. Until then the bytes are archived and nothing is lost. // Legacy decoder first, firmware-fallback chain second, undecodable // archive last — see FirmwareAwareR24Decoder. var decodeTarget = frame.inner; @@ -2311,6 +2551,9 @@ class BleEngine { az: r.accelG.length > 2 ? r.accelG[2] : 0, spo2RedRaw: r.spo2RedRaw, spo2IrRaw: r.spo2IrRaw, + // stored raw under the column it's always had. nothing reads it as a + // temperature — that's what the deprecation is warning about. + // ignore: deprecated_member_use skinTempRaw: r.skinTempRaw, ); } @@ -2327,35 +2570,33 @@ class BleEngine { // archive rides the SAME commit that runs before the batch-ACK, so nothing the // band trims has been discarded (safe-trim invariant intact). if (sample == null) { - final archive = ArchiveRecord( - counter: counter, - hex: _innerHex(frame.inner), - packetType: frame.inner.isNotEmpty ? frame.inner[0] : 0, - capturedAt: DateTime.now().millisecondsSinceEpoch, + _archiveHistoricalFrame( + frame, + counter, reason: 'undecodable_rec_v$recType', ); - final d = _drain; - if (d != null) { - d.onUndecodableRecord(archive); - } else { - unawaited(onArchiveRecord?.call(archive) ?? Future.value()); - } return; } // PLAUSIBILITY GATE + FRONTIER (RecordGate, shared with the detectors). // Drop records whose unix is implausible vs wall-clock and (when known) the // strap's own GET_DATA_RANGE window — a previous owner's wandering-clock // pollution. Records with no decodable ts are kept (can't gate them). - // Rejected records are neither stored nor counted. Mixed bursts (some - // rows banked) may still ACK; a drop-only empty burst must not — see - // TrimAckVerdict.blockedNoDurableProgress. + // A rejected record is not BANKED and not counted — but its bytes are + // archived (see below). Mixed bursts (some rows banked) may still ACK; a + // drop-only burst must not — see TrimAckVerdict.blockedNoDurableProgress. // Past this point [sample] is non-null — undecodable records returned above. if (!_recordGate.admit( sample.tsEpoch, - wallNow: DateTime.now().millisecondsSinceEpoch ~/ 1000, + wallNow: wallNow, sessionOldestUnix: _sessionOldestUnix, sessionNewestUnix: _sessionNewestUnix, )) { + // ARCHIVE, don't drop. A record we merely MISTRUST used to be written + // nowhere at all, i.e. treated strictly worse than one we cannot parse — + // and the batch-ACK then let the band trim those bytes away for good. + // The archive rides the same pre-ACK transaction, so the bytes survive + // and a later pass can re-time them once the clock correlation is known. + _archiveHistoricalFrame(frame, counter, reason: kGateDroppedReason); return; } final raw = RawRecord( @@ -2380,8 +2621,15 @@ class BleEngine { void _absorbState(Decoded d) { final f = d.fields; if (d.kind == 'cmd_response' && f['opcode'] == Cmd.getDataRange) { - final oldest = (f['history_oldest'] as num?)?.toInt(); - final newest = (f['history_newest'] as num?)?.toInt(); + // `range_oldest`/`range_newest` come from the reply's real field map. + // These used to come from a local byte-scan that took the min/max of every + // 4-byte window that looked like a unix time — which is how a cross-field + // read once landed as "newest" in 2034 and left `backlogRemains` true + // forever, chasing a target the band could never reach. The scanner was + // then given a tighter ceiling instead of being replaced, and it kept + // running alongside the correct value, feeding a different decision. + final oldest = (f['range_oldest'] as num?)?.toInt(); + final newest = (f['range_newest'] as num?)?.toInt(); if (oldest != null) _strapHistoryOldestTs = oldest; if (newest != null) _strapHistoryNewestTs = newest; unawaited( @@ -2442,6 +2690,19 @@ class BleEngine { state.wristOn = f['on_wrist'] as bool; onState(state); } + // A GET_CLOCK reply releases the read gate whether or not a usable epoch + // came out of it — "the read completed" and "the read produced a plausible + // clock" are different questions. A revision byte we do not recognise, or a + // corrupt above-ceiling value, yields no `clock_epoch` at all; leaving the + // gate to time out would then cost 3 s on EVERY clock read, stalling both + // the connect-path SET_CLOCK decision and the drain gate. + if (d.kind == 'cmd_response' && + (f['opcode'] == Cmd.getClock || f['opcode'] == Cmd.getClockGen5)) { + final pendingRead = _clockReadPending; + if (pendingRead != null && !pendingRead.isCompleted) { + pendingRead.complete(); + } + } if (f.containsKey('clock_epoch')) { final dev = f['clock_epoch'] as int; final wall = DateTime.now().millisecondsSinceEpoch ~/ 1000; @@ -2458,21 +2719,15 @@ class BleEngine { } else if (!_phoneClockSuspect) { _phoneClockSuspectSince = null; } - // Release the gate waiting on THIS read (see [_readClock]). Done as soon - // as the verdict above is settled, before the alarm-correlation work - // below, because the verdict is all a gate is waiting for. + // The read gate is released above, on the reply itself, not here. // - // UNCORRELATED: any fresh clock_epoch releases the waiter, including a - // reply to setClock()'s read-back or the keep-alive poll. Telling them - // apart needs the echoed request seq, which the pinned protocol does not - // surface — see the pin note in pubspec.yaml and OpenStrap/protocol#28. - // The reply that lands is still a real strap read from this session, so - // the verdict is fresh; it may just answer a request a few hundred ms - // older than ours. - final pendingRead = _clockReadPending; - if (pendingRead != null && !pendingRead.isCompleted) { - pendingRead.complete(); - } + // UNCORRELATED either way: any GET_CLOCK reply releases the waiter, + // including one answering setClock()'s read-back or the keep-alive poll. + // Telling them apart needs the echoed request seq, which the pinned + // protocol does not surface — see the pin note in pubspec.yaml and + // OpenStrap/protocol#28. The reply that lands is still a real strap read + // from this session, so the verdict is fresh; it may just answer a + // request a few hundred ms older than ours. if (_phoneClockSuspect != wasSuspect) { _log(_phoneClockSuspect ? '[SYNC] Phone clock appears wrong: strap RTC=$dev is > 1 day ahead ' @@ -2488,7 +2743,19 @@ class BleEngine { // correlation the alarm falls back to the raw wall epoch. connect() // already issues an unconditional SET_CLOCK, and the periodic re-verify // re-reads, so a genuinely-wrong RTC still gets corrected. - if (!ClockPolicy.acceptsClockRead(dev, wall)) { + if (dev < kMinPlausibleUnix) { + // UNSET RTC. This read is now surfaced instead of swallowed by the + // decoder (see [_maybeAugmentClockEpoch]) so the SET_CLOCK correction + // below can finally fire for it — but it must NOT become a ClockRef: + // correlating a factory-epoch clock yields a drift of decades, and + // `AlarmPayloads.toStrapFrame` would arm every alarm that far in the + // past. + _log( + '[SYNC] GET_CLOCK clock_epoch=$dev is below the plausible floor — ' + 'the strap RTC was never set. NOT correlating; SET_CLOCK below is ' + 'the fix.', + ); + } else if (!ClockPolicy.acceptsClockRead(dev, wall)) { _corruptClockReadCount++; _log( '[SYNC] GET_CLOCK clock_epoch=$dev is implausibly far in the future ' @@ -2587,6 +2854,16 @@ class BleEngine { state.wristOn = h.wristOn ?? state.wristOn; onState(state); } + // gen5's GET_HELLO (opcode 145) response shape is unrelated to gen4's + // HelloInfo — it carries a device_name + a gated fw_version instead + // (parseCommandResponse's gen5 GET_HELLO branch). No confirmed serial/ + // battery/wrist-on offsets for it yet, so — unlike gen4's HELLO above — + // this is diagnostics-only for now (confirms the untested gen5 handshake + // actually got a byte-parseable reply) rather than wired into `state`. + if (d.kind == 'cmd_response' && f.containsKey('device_name')) { + _log('[HELLO gen5] device_name=${f['device_name']} ' + 'fw_version=${f['fw_version']}'); + } if (d.kind == 'realtime_hr') { final hr = f['hr'] as int; if (hr > 0) { @@ -2969,8 +3246,20 @@ class BleEngine { } final r = d.bufferedRecTsRange; final droppedThisBurstForLog = droppedThisBurst; + // Banked records, plus archives that are NOT plausibility drops. + // + // Counting EVERY archive makes blockedNoDurableProgress unfireable in the + // one case it exists for — a drop-only burst archives too, so the band + // would be cleared to trim flash we never read. Counting NO archive wedges + // a burst that is entirely records we cannot decode (a gen4 R10 + // historical has its own decoder and is not in kKnownRecordVersions) into + // being re-delivered forever. Hence the split, not a bare record count. + // + // This only decides whether the band may TRIM. Archives are committed in + // the same transaction regardless — except on the no-progress path, which + // returns before commit precisely because there is nothing to bank. final hadDurableRows = - d.bufferedRecords > 0 || d.bufferedArchives > 0; + d.bufferedRecords > 0 || d.bufferedProgressArchives > 0; _log( '[SYNC] HistoryEnd batch=${m.batchId} records=${d.records} ' 'expected=${m.expectedPacketCount} actual=${d.currentBurstPacketCount} ' @@ -3040,7 +3329,8 @@ class BleEngine { ); return; } - final ack = buildHistoryResultOk(_seq.nextSync(), m.token!); + final ack = buildHistoryResultOk(_seq.nextSync(), m.token!, + profile: _session?.band ?? BandProfile.gen4); _log( '[SYNC] ACK frame=' '${ack.map((b) => b.toRadixString(16).padLeft(2, '0')).join()}', @@ -3122,6 +3412,11 @@ class BleEngine { 'last_ack_batches': d.batches, 'strap_history_oldest_ts': _strapHistoryOldestTs, 'strap_history_newest_ts': _strapHistoryNewestTs, + // Which WHOOP generation this batch came from — records/sessions + // vary hugely in richness by generation (and, for gen5, by whether + // the R22 deep-buffer opt-in was sent), so downstream diagnostics + // need this without reaching into the transport layer. + 'band_generation': state.generation, }, )); // Same event, but a REAL per-chunk row keyed by the token — closes out @@ -3181,6 +3476,7 @@ class BleEngine { 'history_completions': _historyCompletions, 'strap_history_oldest_ts': _strapHistoryOldestTs, 'strap_history_newest_ts': _strapHistoryNewestTs, + 'band_generation': state.generation, }, )); _log( @@ -3262,8 +3558,48 @@ class BleEngine { int _counterFromInner(Uint8List inner) => inner.length >= 7 ? u32(inner, 3) : 0; - String _innerHex(Uint8List inner) => - inner.map((b) => b.toRadixString(16).padLeft(2, '0')).join(); + static const _hexDigits = '0123456789abcdef'; + // Called once per stored record, once per archived record and once per live + // frame, so it runs ~50k times in an offload on the UI isolate. The obvious + // `map(toRadixString).join()` allocates a String per byte plus a List plus + // the join: ~146 ms/50k at 120 bytes, and ~1.2 s/50k at v21's 1232. + String _innerHex(Uint8List inner) { + final out = Uint8List(inner.length * 2); + for (var i = 0; i < inner.length; i++) { + final b = inner[i]; + out[i * 2] = _hexDigits.codeUnitAt(b >> 4); + out[i * 2 + 1] = _hexDigits.codeUnitAt(b & 0x0F); + } + return String.fromCharCodes(out); + } + + /// Send the gen5 "R22" 16-flag SET_CONFIG enable sequence + /// (protocol's `buildR22EnableSequence`/`kGen5R22EnableFlags`), unlocking + /// the v20 (optical)/v21 (IMU)/v26 (PPG) deep-buffer historical records. + /// Sequential, ~40ms apart (same spacing discipline as the gen4 5-packet + /// INIT). Not sent unless [gen5DeepBuffersEnabled] opts in (see the + /// constructor doc). UNTESTED on physical hardware. No-op on a gen4 link. + /// + /// Written directly via [_write] (not [_send]) because the pre-built + /// frames already carry their own sequence numbers — going through `_send` + /// would double-allocate from [_seq] for no benefit. SET_FF_VALUE (120) is + /// in `OpcodeSafety.forbidden` but NOT `OpcodeSafety.destructive`; per that + /// class's own doc this deliberate, explicitly-opted-in sequence is exactly + /// the kind of call site the broader `forbidden` list is not meant to gate + /// (see `_send`'s doc for the full reasoning) — writing it directly here + /// keeps that intentional exception in ONE place rather than needing an + /// allowlist parameter threaded through the shared chokepoint. + Future enableGen5DeepBuffers() async { + if (!(_session?.band.isGen5 ?? false)) return; + final frames = buildR22EnableSequence(startSeq: _seq.nextLive()); + _log('Sending gen5 R22 deep-buffer enable sequence (${frames.length} ' + 'flags)…'); + for (final frame in frames) { + await _write(frame); + await Future.delayed(const Duration(milliseconds: 40)); + } + _log('gen5 R22 deep-buffer enable sequence sent.'); + } // ── high-level flows ───────────────────────────────────────────────────────────── /// [drain] false sends the first FOUR packets only: seq4 is @@ -3275,6 +3611,64 @@ class BleEngine { /// `_offloadActive` set behind it wedges every later refresh on the /// already-transmitting guard. Future sendInit({bool drain = true}) async { + final band = _session?.band ?? BandProfile.gen4; + if (band.isGen5) { + // gen5 handshake: a single CLIENT_HELLO (GET_HELLO 0x91) written + // with-response opens the just-works bond, then the offload is driven by + // GET_DATA_RANGE + SEND_HISTORICAL_DATA with EMPTY payloads (gen4 sends a + // 0x00). The HISTORY_END ACK is byte-structured identically (handled in + // the metadata path). NOTE: untested on physical hardware — pending a + // WHOOP 5 device; the gen4 path below is unchanged. + // + // [drain] is honoured here for the same reason it exists on gen4: the + // drain must not start while the phone clock is suspect, or the records + // it pulls get stamped against a clock we do not trust. + _log('Sending gen5 CLIENT_HELLO + offload…'); + var ok = false; + // try/finally for the same reason the gen4 loop below has one: other + // paths rely on `_connectSetup` being cleared here, and a throw anywhere + // above the clear leaves the link pinned at setup priority for the whole + // connection with `_applyLinkPriority` early-returning forever. + try { + ok = await _write(gen5ClientHello()); + await Future.delayed(const Duration(milliseconds: 120)); + // Opt-in deep-buffer sequence, BEFORE the offload trigger (SET_CONFIG + // flags must land before SEND_HISTORICAL_DATA to take effect for this + // drain). Default OFF — see [gen5DeepBuffersEnabled]. + if (gen5DeepBuffersEnabled()) { + await enableGen5DeepBuffers(); + } + // Same band-aware helpers the refresh/backfill/retry paths use, so the + // gen5 offload command format is identical everywhere. + // + // Stop at the first failure, exactly as the gen4 loop below does — and + // here for a second reason. The caller pre-arms `_offloadActive` and + // rolls it back when this returns false, which is only sound while + // "false" implies the drain trigger never went out. Firing + // SEND_HISTORICAL_DATA after an earlier write failed breaks that: the + // band floods while the caller clears `_offloadActive`, so link priority + // steps back down mid-drain, maintenance traffic resumes, and the + // offload branches all take the not-offloading path. + if (ok) { + ok = await _sendGetDataRange(); + await Future.delayed(const Duration(milliseconds: 120)); + } + if (!drain) { + _log('gen5 INIT: skipping the drain (phone clock suspect).'); + } else if (ok) { + ok = await _sendHistoricalData(); + } + if (!ok) { + _log('gen5 INIT write failed — abandoning the remaining packets.'); + } + } finally { + if (_connectSetup) { + _connectSetup = false; + unawaited(_applyLinkPriority()); + } + } + return ok; + } final pkts = drain ? initPackets : initPackets.take(initPackets.length - 1).toList(); _log('Sending ${pkts.length}-packet INIT…'); @@ -3386,29 +3780,45 @@ class BleEngine { /// subsecond value is the safe thing. Then read the clock back (GET_CLOCK) so /// the response handler can VERIFY it latched and re-issue on drift. Future setClock() async { - final ms = DateTime.now().millisecondsSinceEpoch; + final now = DateTime.now(); + final ms = now.millisecondsSinceEpoch; final sec = ms ~/ 1000; final subsec = ((ms % 1000) * 32768) ~/ 1000; // 0..32767, 1/32768 s units - final payload = [ - sec & 0xff, - (sec >> 8) & 0xff, - (sec >> 16) & 0xff, - (sec >> 24) & 0xff, - subsec & 0xff, - (subsec >> 8) & 0xff, - 0, - 0, - ]; - await _send(Cmd.setClock, payload); - _log('SET_CLOCK → sec=$sec subsec=$subsec (WHOOP-exact 8B).'); + // gen5 ("Maverick") uses a DIFFERENT opcode for SET_CLOCK than gen4 and a + // body that leads with a revision byte; protocol owns both — see + // `cmdSetClockGen5`. Gen4 keeps the hardware-verified 8-byte body. + final isGen5 = _session?.band.isGen5 ?? false; + if (isGen5) { + await _write(cmdSetClockGen5(_seq.nextLive(), now: now)); + } else { + await _send(Cmd.setClock, [ + sec & 0xff, + (sec >> 8) & 0xff, + (sec >> 16) & 0xff, + (sec >> 24) & 0xff, + subsec & 0xff, + (subsec >> 8) & 0xff, + 0, + 0, + ]); + } + _log('SET_CLOCK${isGen5 ? " (gen5 Maverick)" : ""} → sec=$sec ' + 'subsec=$subsec.'); // Read the RTC back so the GET_CLOCK response handler can confirm it latched // (and re-issue SET_CLOCK if the strap clock is still off — see _onDecoded). await getClock(); } /// Read the strap RTC. The response carries `clock_epoch`, handled where we - /// verify drift and re-correlate the strap-RTC ↔ wall clock. - Future getClock() => _send(Cmd.getClock, const []); + /// verify drift and re-correlate the strap-RTC ↔ wall clock. gen5 uses its + /// own GET_CLOCK opcode and needs a leading revision byte — protocol's + /// `cmdGetClockGen5` owns both. + Future getClock() { + if (_session?.band.isGen5 ?? false) { + return _write(cmdGetClockGen5(_seq.nextLive())); + } + return _send(Cmd.getClock, const []); + } /// GET_CLOCK, awaited to the *response* rather than to the write. /// @@ -3429,7 +3839,7 @@ class BleEngine { /// signal that the read never landed. Future _readClock() async { final pending = _clockReadPending = Completer(); - await _send(Cmd.getClock, const []); + await getClock(); // band-correct opcode + body; gen4 sent to a gen5 strap is silence try { await pending.future.timeout(_clockReadTimeout); return true; @@ -3451,47 +3861,60 @@ class BleEngine { static const Duration _clockReadTimeout = Duration(seconds: 3); /// On-device wake alarm (SET_ALARM_TIME = 0x42) — the RICH 20-byte form that - /// actually FIRES on WHOOP 4.0: + /// actually FIRES: /// ``` /// [0] 0x04 rich-form marker - /// [1] u8 index alarm slot (default 0) + /// [1] u8 index alarm slot (gen4: 0; gen5: 1) /// [2..6] u32 epoch-sec LE the wake time /// [6..8] u16 subsec LE (millis % 1000) * 32768 ~/ 1000 (1/32768 s units) /// [8..20] 12-byte haptic pattern (see [AlarmPayloads.defaultHaptics]) /// ``` - /// The short 7-byte time-only form ([setAlarmSimple]) is accepted and ACKed by - /// the band but carries no waveform, so the strap never buzzes it — our earlier - /// short-form attempts silently failed for exactly this reason. The strap - /// confirms the alarm latched via event 56 (STRAP_DRIVEN_ALARM_SET) and reports - /// firing via events 57/58 + 60. Byte layout lives in the pure [AlarmPayloads]. - /// Returns whether the arm write actually reached the band, so the caller can - /// avoid persisting / confirming a phantom alarm on a failed write. - Future setAlarm( + /// WHOOP 5 requires slot index 1: index 0 is + /// rejected with `arm info is invalid, error 0xb`. The short 7-byte + /// time-only form ([setAlarmSimple]) is ACKed but never buzzes. The strap + /// confirms via event 56 and reports firing via 57/58 + 60. + /// + /// Returns the wall-clock instant armed, or null if the write failed (so the + /// caller does not persist a phantom alarm). + Future setAlarm( DateTime when, { int index = 0, List? haptics, }) async { + final isGen5 = _session?.band.isGen5 ?? false; + if (isGen5) { + // Official WHOOP app SET_CLOCKs before SET_ALARM; refresh RTC drift first. + await setClock(); + await Future.delayed(const Duration(milliseconds: 120)); + } // Arm in the STRAP's RTC frame. The strap fires the wake alarm autonomously // on its OWN clock, so if that clock is offset from wall time (SET_CLOCK not // latched / drift) the raw wall epoch fires at the wrong strap-time — or // never (a raw wall epoch is decades ahead of a strap clock still near its - // factory epoch, which is exactly why an immediate RUN_ALARM buzz works but a - // scheduled alarm never fires). Shift the target by the GET_CLOCK drift; fall - // back to the raw epoch when we have no correlation yet (e.g. just after a - // reconnect, before this session's GET_CLOCK reply). Byte layout + the frame - // conversion both live in the pure [AlarmPayloads]. + // factory epoch, which is exactly why an immediate RUN_ALARM / Maverick buzz + // works but a scheduled alarm never fires). Shift the target by the + // GET_CLOCK drift; fall back to the raw epoch when we have no correlation + // yet (e.g. just after a reconnect, before this session's GET_CLOCK reply). + // Byte layout + the frame conversion both live in the pure [AlarmPayloads]. final ref = _clockRef; final driftSec = ref?.driftSec ?? 0; final armWhen = AlarmPayloads.toStrapFrame(when, driftSec); - final ok = await _send( - Cmd.setAlarmTime, - AlarmPayloads.rich(armWhen, index: index, haptics: haptics), + final payload = AlarmPayloads.setPayloadForBand( + armWhen, + isGen5: isGen5, + index: index, + haptics: haptics, ); - _log('SET_ALARM_TIME (rich 20B) → wallSec=${when.millisecondsSinceEpoch ~/ 1000} ' - 'strapSec=${armWhen.millisecondsSinceEpoch ~/ 1000} drift=${driftSec}s ' - 'correlated=${ref != null} subsec=${AlarmPayloads.subsecOf(armWhen)} ' - 'write=${ok ? 'ok' : 'FAILED'}'); - return ok; + final ok = await _send(Cmd.setAlarmTime, payload); + _log( + 'SET_ALARM_TIME (${isGen5 ? "gen5 rich index1" : "rich"} ${payload.length}B) ' + '→ wallSec=${when.millisecondsSinceEpoch ~/ 1000} ' + 'strapSec=${armWhen.millisecondsSinceEpoch ~/ 1000} drift=${driftSec}s ' + 'correlated=${ref != null} subsec=${AlarmPayloads.subsecOf(armWhen)} ' + 'idx=${payload.length >= 2 ? payload[1] : -1} ' + 'write=${ok ? 'ok' : 'FAILED'}', + ); + return ok ? when : null; } /// Time-only alarm (SET_ALARM_TIME = 0x42), SHORT 7-byte form: @@ -3503,21 +3926,61 @@ class BleEngine { '(ACKs but will not fire)'); } - Future getAlarm() => _send(Cmd.getAlarmTime, const [revision1]); + /// Read the armed alarm back. Body is band-specific (see + /// [AlarmPayloads.getPayloadForBand]) — gen5 rejects gen4's operand-less + /// revision-1 body. + Future getAlarm({int? id}) { + final isGen5 = _session?.band.isGen5 ?? false; + return _send( + Cmd.getAlarmTime, + AlarmPayloads.getPayloadForBand( + isGen5: isGen5, + id: id ?? AlarmPayloads.gen5Slot, + ), + ); + } - /// Fire the alarm haptics IMMEDIATELY (RUN_ALARM = 0x44), payload `[0x01]`. - /// A "test buzz" so the user can confirm the strap actually fires before - /// trusting the scheduled wake. - Future runAlarm() => _send(Cmd.runAlarm, AlarmPayloads.runNow); + /// Fire the alarm haptics IMMEDIATELY — a "test buzz" so the user can confirm + /// the strap actually fires before trusting the scheduled wake. + /// + /// WHOOP 4: RUN_ALARM (0x44) `[0x01]`. + /// WHOOP 5: RUN_ALARM does not buzz on hardware we tested; use the same + /// Maverick `0x13` short pulse as Find-band. Do NOT STOP_HAPTICS first — + /// on gen5 that can race and swallow the buzz. + Future runAlarm() async { + if (_session?.band.isGen5 ?? false) { + await _send( + Cmd.runHapticPatternMaverick, + AlarmPayloads.gen5MaverickBuzz(), + ); + return; + } + await _send(Cmd.runAlarm, AlarmPayloads.runNow); + } - /// Cancel the on-device alarm (DISABLE_ALARM = 0x45), payload `[0x01]`. - /// (The earlier `[0x00]` body was ACKed but did not clear the alarm.) - Future disableAlarm() => _send(Cmd.disableAlarm, AlarmPayloads.disable); + /// Cancel the on-device alarm (DISABLE_ALARM = 0x45). gen4 body `[0x01]` + /// (the earlier `[0x00]` body was ACKed but did not clear the alarm); gen5 + /// needs revision 2 plus the alarm id, defaulting to "all slots" — see + /// [AlarmPayloads.disableForBand]. + Future disableAlarm({int? id}) { + final isGen5 = _session?.band.isGen5 ?? false; + return _send( + Cmd.disableAlarm, + AlarmPayloads.disableForBand( + isGen5: isGen5, + id: id ?? AlarmPayloads.gen5AllSlots, + ), + ); + } - Future getStrapName() => - _send(Cmd.getAdvertisingNameHarvard, const [0x00]); + /// Read the strap's advertising name. gen5 does not implement gen4's + /// advertising-name opcodes at all — it has its own pair. + Future getStrapName() => (_session?.band.isGen5 ?? false) + ? _send(Cmd.getCustomAdvertisingName, const [revision1]) + : _send(Cmd.getAdvertisingNameHarvard, const [0x00]); /// Rename the strap. Payload: [0x01][name length u8][ASCII name bytes][u32 0]. + /// Same body on both generations; only the opcode differs. Future setStrapName(String name) async { // Cap at 20 ASCII chars (matches the reference + the GET decoder's length // assumption); the length byte then always stays < 0x20. @@ -3526,16 +3989,36 @@ class BleEngine { .take(20) .toList(); final payload = [0x01, ascii.length, ...ascii, 0, 0, 0, 0]; - await _send(Cmd.setAdvertisingNameHarvard, payload); + final isGen5 = _session?.band.isGen5 ?? false; + await _send( + isGen5 ? Cmd.setCustomAdvertisingName : Cmd.setAdvertisingNameHarvard, + payload, + ); _log('SET_ADVERTISING_NAME → "$name"'); } + // main's throttled poll (a raw send here was 2,880 round-trips a day), and + // the branch's gen5 HELLO, which is a different opcode on Maverick. Future getBattery() => _pollBatteryIfDue(force: true); - Future getHello() => _send(Cmd.getHelloHarvard, const [0x00]); + Future getHello() => (_session?.band.isGen5 ?? false) + ? _send(Cmd.getHello, const [0x01]) + : _send(Cmd.getHelloHarvard, const [0x00]); Future buzz() => buzzPattern(hapticShortPulse); - Future buzzPattern(int pattern) => - _send(Cmd.runHapticsPattern, [pattern, 0, 0, 0, 0]); + /// Play a haptic buzz. gen5 ("Maverick") has a DIFFERENT buzz opcode and + /// payload shape than gen4 (`Cmd.runHapticPatternMaverick`, 12-byte body — + /// see `cmdBuzzGen5Maverick` in protocol/commands.dart) — [pattern] is + /// honoured only on gen4; a gen5 link always plays the strap's fixed + /// `[47, 152]` waveform pair (the only Maverick buzz byte-verified so far). + Future buzzPattern(int pattern) { + if (_session?.band.isGen5 ?? false) { + return _send( + Cmd.runHapticPatternMaverick, + AlarmPayloads.gen5MaverickBuzz(), + ); + } + return _send(Cmd.runHapticsPattern, [pattern, 0, 0, 0, 0]); + } /// Signal strength of the live link, in dBm (negative; closer to zero is /// stronger). Null whenever there is nothing to measure. @@ -3571,6 +4054,7 @@ class BleEngine { unawaited(_applyLinkPriority()); // a live consumer earns the fast interval _armTime = DateTime.now(); // marginal-radio detector measures arm→drop latency + final isGen5 = _session?.band.isGen5 ?? false; await _send(Cmd.toggleRealtimeHr, const [0x01]); // MARGINAL-RADIO FALLBACK: a weak radio can't sustain the high-rate R10/R11 + // IMU + optical flood, so once the detector trips we arm HR only. @@ -3579,12 +4063,28 @@ class BleEngine { return; } await Future.delayed(const Duration(milliseconds: 100)); - await _send(Cmd.sendR10R11Realtime, const [0x01]); - await Future.delayed(const Duration(milliseconds: 100)); - await _send(Cmd.toggleImuMode, const [0x01]); + // gen5 console: 0x3F (R10/R11 realtime) is Unknown/Unhandled — skip it. + // Live steps ride toggleImuMode (gen5: 0x2B rec 0x15; gen4: 0x33). + if (!isGen5) { + await _send(Cmd.sendR10R11Realtime, const [0x01]); + await Future.delayed(const Duration(milliseconds: 100)); + } + await _sendToggleImu(true); await Future.delayed(const Duration(milliseconds: 100)); - await _send(Cmd.enableOpticalData, const [revision1, 0x01]); - _log('Live streams enabled (optical: wrist-gated).'); + // gen4 only. On gen5 this opcode is the SAVE-to-history toggle, not the + // realtime stream — the realtime one is the next opcode up — so arming it + // here would write a save enable on every live-stream start, next door to + // the persistent-optical footgun that leaves the LEDs on and drains the + // battery. gen5's own 1 Hz stream already carries per-second HR, so there + // is nothing to gain until the roles are confirmed on hardware. The OFF + // writes in disableLiveStreams stay unconditional. + if (!isGen5) { + await _send(Cmd.enableOpticalData, const [revision1, 0x01]); + } + _log( + 'Live streams enabled (' + '${isGen5 ? "gen5 IMU rev1; optical skipped" : "optical: wrist-gated"}).', + ); } /// Clear the sticky standard-HR fallback and give the full live set another @@ -3616,28 +4116,17 @@ class BleEngine { if (_session?.connected != true) return; _liveEnabled = true; _liveHrOnly = true; + final isGen5 = _session?.band.isGen5 ?? false; unawaited(_applyLinkPriority()); // downgraded to HR-only ⇒ step the link down await _send(Cmd.toggleRealtimeHr, const [0x01]); - final offOps = >[ - [ - Cmd.toggleOpticalMode, - [revision1, 0x00], - ], - [ - Cmd.enableOpticalData, - [revision1, 0x00], - ], - [ - Cmd.sendR10R11Realtime, - [0x00], - ], - [ - Cmd.toggleImuMode, - [0x00], - ], + final offOps = Function()>[ + () => _send(Cmd.toggleOpticalMode, const [revision1, 0x00]), + () => _send(Cmd.enableOpticalData, const [revision1, 0x00]), + if (!isGen5) () => _send(Cmd.sendR10R11Realtime, const [0x00]), + () => _sendToggleImu(false), ]; for (final op in offOps) { - await _send(op[0] as int, (op[1] as List).cast()); + await op(); await Future.delayed(const Duration(milliseconds: 60)); } _log('Live streams: HR-only (background downgrade — raw flood off).'); @@ -3645,30 +4134,16 @@ class BleEngine { /// Turn everything off. Safe + idempotent. Clears flags back to wrist-gated. Future disableLiveStreams() async { - final ops = >[ - [ - Cmd.toggleOpticalMode, - [revision1, 0x00], - ], - [ - Cmd.enableOpticalData, - [revision1, 0x00], - ], - [ - Cmd.sendR10R11Realtime, - [0x00], - ], - [ - Cmd.toggleImuMode, - [0x00], - ], - [ - Cmd.toggleRealtimeHr, - [0x00], - ], + final isGen5 = _session?.band.isGen5 ?? false; + final ops = Function()>[ + () => _send(Cmd.toggleOpticalMode, const [revision1, 0x00]), + () => _send(Cmd.enableOpticalData, const [revision1, 0x00]), + if (!isGen5) () => _send(Cmd.sendR10R11Realtime, const [0x00]), + () => _sendToggleImu(false), + () => _send(Cmd.toggleRealtimeHr, const [0x00]), ]; for (final op in ops) { - await _send(op[0] as int, (op[1] as List).cast()); + await op(); await Future.delayed(const Duration(milliseconds: 60)); } _liveEnabled = false; @@ -3834,34 +4309,52 @@ class BleEngine { ); } - Decoded _maybeAugmentDataRange(Frame frame, Decoded decoded) { + /// Make sure a GET_CLOCK reply always carries a `clock_epoch` — INCLUDING an + /// implausible one. + /// + /// `parseCommandResponse` decodes the field for both generations now, but it + /// emits it only when the value already looks like a real wall-clock time. + /// That silently disabled the whole point of [ClockPolicy.shouldSetClock], + /// which exists to detect a strap whose RTC was never set (a 1970s value): + /// the one reading that proves the fault produced no field at all, so the + /// policy could never fire and the RTC was never corrected. Read the field at + /// its documented offset and pass it through verbatim; judging it is the + /// policy's job, not the decoder's. + Decoded _maybeAugmentClockEpoch(Frame frame, Decoded decoded) { if (decoded.kind != 'cmd_response') return decoded; - final opcode = decoded.fields['opcode']; - if (opcode != Cmd.getDataRange) return decoded; - final payload = frame.inner.length > 3 - ? Uint8List.sublistView(frame.inner, 3) - : Uint8List(0); - // We scan every byte offset for a plausible unix u32 (the field layout isn't - // fully pinned), so the UPPER bound must be tight: a data-range timestamp can - // never be in the FUTURE. The old ceiling (2100000000 ≈ year 2036) let a - // spurious cross-field read land as "newest" — observed 2020230636 (year - // 2034) — which made `history_newest` garbage, so backlogRemains was - // PERMANENTLY true and the offload never recognized completion (it chased a - // 2034 target forever). Cap at wall-clock + 1 day (clock skew slack). - final maxPlausible = - (DateTime.now().millisecondsSinceEpoch ~/ 1000) + 86400; - final ts = []; - for (var off = 0; off + 4 <= payload.length; off++) { - final v = u32(payload, off); - if (v >= 1600000000 && v <= maxPlausible) { - ts.add(v); - } - } - if (ts.isEmpty) return decoded; - final fields = {...decoded.fields}; - fields['history_oldest'] = ts.reduce((a, b) => a < b ? a : b); - fields['history_newest'] = ts.reduce((a, b) => a > b ? a : b); - return Decoded(decoded.kind, fields); + final op = decoded.fields['opcode']; + final isGen5Clock = op == Cmd.getClockGen5; + if (!isGen5Clock && op != Cmd.getClock) return decoded; + if (decoded.fields.containsKey('clock_epoch')) return decoded; + // The strap answers every command with a status byte. A failure or an + // unimplemented-opcode reply leaves the body unpopulated, so reading an + // epoch out of it hands ClockPolicy a stale value from whatever the buffer + // held last — and this path deliberately forwards implausible clocks so the + // unset-RTC case is reachable, which means nothing downstream would filter + // it back out. + final status = decoded.fields['cmd_status']; + if (status != null && status != 1) return decoded; + final inner = frame.inner; + final payload = + inner.length > 3 ? Uint8List.sublistView(inner, 3) : Uint8List(0); + // Reply body starts at payload[2] (payload[0] = echoed request seq, + // payload[1] = status). gen5 leads the body with a revision byte and puts + // the u32 seconds at payload[3]; gen4 has them at payload[2]. + final at = isGen5Clock ? 3 : 2; + if (payload.length < at + 4) return decoded; + final v = u32(payload, at); + // Only the UNSET-RTC case is worth surfacing. Protocol already emits + // clock_epoch for anything inside the plausible window, so everything that + // reaches here is outside it — below the floor (the low strap counter this + // augmenter exists for) or above the ceiling, which is a corrupt reply and + // ~4% of u32 space. Forwarding the high side sets `_phoneClockSuspect` off + // a garbage reading, and that defers the history refresh AND skips the very + // SET_CLOCK that would repair the strap. + if (v >= kMinPlausibleUnix) return decoded; + return Decoded(decoded.kind, { + ...decoded.fields, + 'clock_epoch': v, + }); } } @@ -3930,6 +4423,12 @@ class DrainController { int get bufferedRecords => _raws.length; int get bufferedArchives => _archives.length; + + /// Archives that represent real forward progress, i.e. everything EXCEPT the + /// plausibility drops. A burst of records we simply cannot decode has still + /// been preserved and may be trimmed; a burst we merely distrusted has not. + int get bufferedProgressArchives => + _archives.where((a) => a.reason != kGateDroppedReason).length; int get lastProgressMs => _lastProgressAt.millisecondsSinceEpoch; /// Min/max real record time (rec_ts) currently buffered for this batch — lets @@ -3998,8 +4497,25 @@ class DrainController { /// that failed). Buffered for archival in the next atomic commit — never dropped, /// never ACKed away before it is durably set aside. void onUndecodableRecord(ArchiveRecord a) { - records++; - recordsThisOffload++; + // A plausibility-gated drop is NOT progress, in either counter. + // + // `recordsThisOffload` feeds `banked` at the HISTORY_COMPLETE terminal: a + // strap with a wandered RTC can drop every record and still reach COMPLETE, + // and counting those resets the empty-sync streak so `syncClockLost` never + // fires and the RTC remedy never surfaces. `records` is not diagnostics + // either — `report.records > 0` schedules a derive pass and sets the sync + // ledger to `partial`, so counting drops there means a 50k-record offload + // that banked NOTHING still triggers a full derive, every backfill, forever. + // + // A record we merely could not DECODE is different: it is archived + // durably and is genuinely ACKable progress, so it counts in both. + // + // `_lastProgressAt` always bumps — frames really are arriving, and it drives + // the 60 s idle watchdog, which must not fire mid-burst. + if (a.reason != kGateDroppedReason) { + records++; + recordsThisOffload++; + } _lastProgressAt = DateTime.now(); if (_buffering) { _archives.add(a); diff --git a/lib/ble/ble_state.dart b/lib/ble/ble_state.dart index 5d166de..54cc99c 100644 --- a/lib/ble/ble_state.dart +++ b/lib/ble/ble_state.dart @@ -357,9 +357,16 @@ class TrimAckPolicy { /// [commitDurable] — the atomic commit completed (pass `true` when asking /// the pre-commit question "should I even commit this /// token?"). - /// [hadDurableRows] — this burst buffered at least one raw/sample or archive - /// row to bank before ACK. Pass `true` when unknown - /// (pre-commit stale/discard checks only). + /// [hadDurableRows] — this burst buffered at least one RECORD to bank, or + /// one archive that is NOT a plausibility drop, before + /// ACK. Plausibility drops are excluded on purpose: they + /// are archived too, so counting them would make this + /// gate unfireable exactly in the drop-only case it + /// exists for. Records we simply cannot decode DO count — + /// they are durably set aside, and excluding them wedges + /// an undecodable-only burst into endless re-delivery. + /// Pass `true` when unknown (pre-commit stale/discard + /// checks only). /// [droppedThisBurst] — RecordGate rejects during this burst. Combined with /// `!hadDurableRows`, refuses trim so gate-only bursts /// cannot delete flash we never stored. @@ -705,9 +712,10 @@ class DeriveDebouncer { /// keeping the exact byte layout here makes it unit-testable without a real band. /// /// Alarm opcodes: SET_ALARM_TIME 0x42, GET_ALARM_TIME 0x43, RUN_ALARM 0x44, -/// DISABLE_ALARM 0x45. The RICH SET form (a haptic waveform + time) is the one -/// that actually FIRES on WHOOP 4.0; the SHORT time-only form is ACKed but never -/// buzzes (no waveform to play). +/// DISABLE_ALARM 0x45. The RICH SET form (haptic waveform + time) is the one +/// that actually FIRES: WHOOP 4 uses alarm slot index 0; WHOOP 5 uses index 1. +/// The SHORT time-only form is ACKed but never +/// buzzes (no waveform to play). Prefer [setPayloadForBand] for arming. class AlarmPayloads { /// The strap's stock 12-byte wake-buzz haptic pattern: /// [0..7] eight waveform-effect slots (two active: 47, 152; six idle) @@ -747,7 +755,7 @@ class AlarmPayloads { } /// SHORT 7-byte time-only SET_ALARM_TIME payload (ACKs but does NOT fire): - /// `[0x01][u32 epoch-sec LE][u16 subsec LE]`. + /// `[0x01][u32 epoch-sec LE][u16 subsec LE]`. Prefer [setPayloadForBand]. static List simple(DateTime when) { final ms = when.millisecondsSinceEpoch; final sec = ms ~/ 1000; @@ -763,12 +771,72 @@ class AlarmPayloads { ]; } + /// Generation-correct SET_ALARM_TIME body — 20 bytes on gen4, 21 on gen5. + /// + /// WHOOP 4: slot index 0 (HW-verified). WHOOP 5: slot **index 1**. Index 0 is + /// rejected with console `arm info is invalid, error 0xb`. On gen5 the [index] + /// argument is ignored so callers cannot accidentally arm slot 0. + static List setPayloadForBand( + DateTime when, { + required bool isGen5, + int index = 0, + List? haptics, + int crescendo = 0, + }) => + [ + ...rich(when, index: isGen5 ? gen5Slot : index, haptics: haptics), + // gen5's body carries one byte more than gen4's: a crescendo flag the + // strap validates as 0 or 1 and rejects otherwise, so a 20-byte body + // is refused there. Keep this in step with protocol's cmdSetAlarm, + // which is the reference layout — gen4 stays at the 20 bytes verified + // on hardware. + if (isGen5) crescendo & 0x01, + ]; + + /// The alarm slot WHOOP 5 accepts (index 0 is rejected). + static const int gen5Slot = 1; + + /// The gen5 "every slot" alarm id, used by [disableForBand]. + static const int gen5AllSlots = 0xFF; + + /// Gen5 Maverick test-buzz body (RUN_HAPTIC_PATTERN_MAVERICK = 0x13). + /// Same `[47, 152]` waveform pair as Find-band. Keep [overallLoop] at 1 for + /// a short pulse — the wake-alarm's loop=7 feels like a stuck vibrate. + static List gen5MaverickBuzz({int overallLoop = 1}) { + // `& 0xff` keeps an int (unlike `clamp`, which widens to num). + final loop = overallLoop.clamp(0, 0xff).toInt(); + return [0x01, 47, 152, 0, 0, 0, 0, 0, 0, 0, 0, loop]; + } + /// RUN_ALARM (0x44) body — fire the haptics immediately ("test buzz"). static const List runNow = [0x01]; - /// DISABLE_ALARM (0x45) body — cancel the on-device alarm. + /// DISABLE_ALARM (0x45) body — cancel the on-device alarm. GEN4 form; do not + /// change (hardware-verified). Use [disableForBand]. static const List disable = [0x01]; + /// Generation-correct DISABLE_ALARM (0x45) body. + /// + /// WHOOP 4 takes revision 1 with no operand. WHOOP 5 takes revision 2 plus + /// the alarm id to clear, where [gen5AllSlots] clears every slot; sent the + /// gen4 body it reads the id from past the end of the body and the alarm + /// stays armed. + static List disableForBand({ + required bool isGen5, + int id = gen5AllSlots, + }) => + isGen5 ? [0x02, id & 0xff] : disable; + + /// Generation-correct GET_ALARM_TIME (0x43) body. + /// + /// WHOOP 4 takes revision 1 with no operand; WHOOP 5 takes revision 4 plus + /// the alarm id to read, defaulting to the slot [setPayloadForBand] arms. + static List getPayloadForBand({ + required bool isGen5, + int id = gen5Slot, + }) => + isGen5 ? [0x04, id & 0xff] : const [0x01]; + /// Convert a WALL-CLOCK alarm target into the strap's own RTC frame. /// /// The strap runs the wake alarm autonomously and fires when ITS RTC reaches diff --git a/lib/compute/derivation_engine.dart b/lib/compute/derivation_engine.dart index af384ae..e5c3fd9 100644 --- a/lib/compute/derivation_engine.dart +++ b/lib/compute/derivation_engine.dart @@ -711,7 +711,32 @@ import 'substrate.dart'; // Days already finalized at v62 hold those RR-less results permanently — they // are never revisited at the same version — so this needs the bump to be // re-derived onto real beats. -const int kAlgoVersion = 63; +// v64 - THREE CHANGES TO WHAT REACHES THE 1 Hz SUBSTRATE. +// 1. R-R beats are no longer read for record versions whose field map is +// unconfirmed. v7 carries hr at offset 27 and v18 at 14, so those layouts +// are demonstrably not v24's, yet the beats were being read off v24's map +// and the 200..2500 ms filter passed enough of them to hand RMSSD a full +// set of invented intervals. RMSSD/SDNN/HRV move for any day built from +// those versions. +// 2. A record only decodes if its packet type says it is one. The dispatch +// keyed on inner[1], which is the version on a data frame but the sequence +// byte on a control frame, so roughly 2 in 256 control frames decoded as a +// trusted record — hr and an accel vector read out of log text. The +// substrate re-decodes stored hex, so any such row is gone now. +// 3. Gen4 v25 records stay archived instead of being banked. They carry a +// timestamp and a gravity vector and no heart rate at all, and hr is NOT +// NULL with 0 meaning off-skin — so banking them would have asserted the +// band was off the wrist for every one of those seconds. +// +// day_result is keyed (day_id, algo_version), so days finalized at v63 keep +// results built on the old substrate unless the version moves. +// +// NOT in this bump, though both were candidates: the accel-coverage gate and +// the R10-lite exclusion both landed at v63 already. Check the pinned SHA +// before citing a sibling-package change here — a bump whose stated cause is +// not in the pinned code is how a fix was believed shipped for three releases +// while the pin never carried it. +const int kAlgoVersion = 64; // Fold idempotency, the minimum-nights warm-up, and legacy-payload handling // all live in SleepProfilePolicy (pure, unit-tested) — see diff --git a/lib/compute/substrate.dart b/lib/compute/substrate.dart index 40df505..1d0342b 100644 --- a/lib/compute/substrate.dart +++ b/lib/compute/substrate.dart @@ -16,6 +16,16 @@ import 'dart:math' as math; import 'package:openstrap_analytics/onehz.dart' as ana; import 'package:openstrap_protocol/openstrap_protocol.dart' as proto; +/// Minimum fraction of a nocturnal search window that must carry a REAL +/// gravity vector before accel-led (van Hees) sleep detection is trusted. +/// +/// Below this we do not run it at all and fall through to the HR-led window, +/// which is already the honest low-confidence degraded mode. Set at a half +/// rather than something tiny on purpose: van Hees picks the LONGEST immobile +/// block, and absent seconds are maximally "immobile", so a window that is +/// mostly absent would reliably hand the answer to the missing data. +const double kMinAccelCoverageForVanHees = 0.5; + /// The decoded 1 Hz substrate — the only decoded form (ARCHITECTURE_V2). /// /// All HR/accel/ADC arrays are parallel and 1:1 with [tsSec] (one sample per @@ -82,6 +92,36 @@ class Substrate { ana.AccelSample(tsSec[i] * 1000.0, ax[i], ay[i], az[i]) ]; + /// Whether second [i] carries a REAL gravity vector. + /// + /// `decoded_onehz.ax/ay/az` are `REAL NOT NULL`, so a record decoded without + /// a usable gravity vector — gen5 v18 keeps HR/RR and reports the accel as + /// absent rather than discarding the second — is stored as exact `(0, 0, 0)`. + /// That is not a reading a real device can produce — a gravity vector always + /// has magnitude ~1 g, and every decoder that emits one gates on + /// `magSq >= 0.25` — so exact zero is an unambiguous ABSENT marker rather + /// than a measurement. + /// + /// This matters because absent accel does not merely go unused: a run of + /// `(0, 0, 0)` has a constant z-angle of exactly 0.0°, which the van Hees + /// rule reads as PERFECT IMMOBILITY. Eight hours of missing accel scores + /// 28 501 immobile seconds and yields a fabricated ~7.9 h sleep window, + /// fully staged. Absent input must produce no claim, never a confident one. + bool accelPresentAt(int i) => !(ax[i] == 0 && ay[i] == 0 && az[i] == 0); + + /// Fraction of [lo, hi) seconds carrying a real gravity vector (0..1). + /// Returns 0 for an empty range — no evidence, not "all present". + double accelPresentFraction(int lo, int hi) { + final a = lo < 0 ? 0 : lo; + final b = hi > tsSec.length ? tsSec.length : hi; + if (b <= a) return 0; + var present = 0; + for (var i = a; i < b; i++) { + if (accelPresentAt(i)) present++; + } + return present / (b - a); + } + /// 1 Hz HR as doubles (0 = off-skin). Parallel to [tsSec] / [accelSamples]. List hr1hz() => [for (final h in hr) h.toDouble()]; @@ -276,7 +316,13 @@ Substrate decodeSubstrate(List hexes) { } spo2Red[i] = r.spo2RedRaw; spo2Ir[i] = r.spo2IrRaw; + // both deprecated: neither field is what its name says. still filled here so + // the substrate keeps round-tripping, but _daySkinTempCurve and the + // fit_quality diagnostic both derive from them and shouldn't — separate fix, + // it moves user-facing output and needs an algo version bump. + // ignore: deprecated_member_use skinTemp[i] = r.skinTempRaw; + // ignore: deprecated_member_use skinContact[i] = r.skinContact; // RR beats: anchored at the record second (epoch ms). Beats within a record // share its second; time order is preserved by the record sort above. @@ -531,14 +577,33 @@ List calendarDays( ); src = ov.source; // 'manual' | 'confirmed' } else { - s = ana.segmentSleep( - accelSlice, - hrSlice, - hrBaseline: hrBaseline, - rrMs: rrMsSeg, - rrTsMs: rrTsSeg, - habitualMidsleepSec: habitualMidsleepSec, - ); + // Accel-led detection is only meaningful if we actually HAVE accel. + // Absent gravity is stored as exact (0,0,0) (see `accelPresentAt`) and + // van Hees scores a run of it as perfect immobility, so a night whose + // records all decoded without a gravity vector would otherwise produce + // a confident, fully-staged sleep window built entirely out of missing + // data. `immobilityMask` has no validity input to tell it otherwise — + // it is a pure index-wise angle rule, so neither a NaN sentinel (NaN + // comparisons are false, so the "angle changed" test never trips and + // it reads as immobile) nor omitting the seconds (no gap awareness) + // reaches it. The only honest move at this layer is not to let it + // anchor the window in the first place. + final accelCoverage = sub.accelPresentFraction(loS, hiS); + if (accelCoverage >= kMinAccelCoverageForVanHees) { + s = ana.segmentSleep( + accelSlice, + hrSlice, + hrBaseline: hrBaseline, + rrMs: rrMsSeg, + rrTsMs: rrTsSeg, + habitualMidsleepSec: habitualMidsleepSec, + ); + } else { + // Not an error and not "no sleep" — just no accel evidence. Fall + // through to the HR-led path below, which is exactly the degraded + // mode for this and is already marked low-confidence. + s = ana.SleepSegmentation.absent; + } src = 'auto'; if (!s.present) { // Approach 2: accel-led detection found nothing → HR-led fallback. diff --git a/lib/data/db.dart b/lib/data/db.dart index f3d7a8b..aae9628 100644 --- a/lib/data/db.dart +++ b/lib/data/db.dart @@ -12,6 +12,7 @@ import 'dart:convert'; import 'dart:io'; import 'dart:isolate'; +import 'dart:typed_data'; import 'package:flutter/foundation.dart' show visibleForTesting; import 'package:openstrap_protocol/openstrap_protocol.dart' as proto; @@ -95,7 +96,7 @@ class LocalDb { /// pass it: sqflite throws `ArgumentError('onCreate must be null if no /// version is specified')` BEFORE opening anything when `onCreate` is given /// without `version` (sqflite_common database_mixin.dart). - static const int schemaVersion = 33; + static const int schemaVersion = 34; /// SQLite caps host parameters per statement (`SQLITE_MAX_VARIABLE_NUMBER` — /// only 999 on the builds shipped with older Android/iOS). Any `IN (?, ?, …)` @@ -475,6 +476,13 @@ class LocalDb { // unrecoverably deleting a 1 Hz row (raw_records is dropped). await _rekeyDecodedStoreByRecTs(db); } + if (oldV < 34) { + // Keep the per-second fields the band computes itself instead of + // decoding and discarding them. Additive columns only — see + // _ensureDecodedOneHzBandFields. MUST run after the v33 re-key, which + // rebuilds decoded_onehz from an explicit column list. + await _ensureDecodedOneHzBandFields(db); + } }, onOpen: (db) async { await _repairOpenSchema(db); @@ -571,6 +579,35 @@ class LocalDb { } } + /// v34: the per-second fields a gen5 band computes on its own and reports in + /// every record — its pedometer's cumulative step count and cadence, its + /// activity class, a calibrated skin temperature in °C, its on-wrist + /// determination, and the HR-validity flag plus the second HR byte that + /// corroborates the primary one. They used to be decoded and dropped. + /// + /// Additive and safe on a populated DB: seven nullable columns, no rewrite of + /// the (million-row) table, no backfill. Existing rows read NULL — which is + /// the truth for them, since the values were never stored. `skin_temp_raw`, + /// `skin_contact` and the `spo2_*` columns are deliberately left alone; they + /// hold real historical data regardless of what the names now suggest. + static Future _ensureDecodedOneHzBandFields(Database db) async { + const cols = { + 'step_count': 'INTEGER', + 'step_cadence': 'INTEGER', + 'activity_class': 'INTEGER', + 'skin_temp_c': 'REAL', + 'on_wrist': 'INTEGER', + 'hr_valid': 'INTEGER', + 'hr_alt': 'INTEGER', + }; + final have = await _columnsOf(db, 'decoded_onehz'); + if (have.isEmpty) return; // table not created yet — the DDL carries them + for (final e in cols.entries) { + if (have.contains(e.key)) continue; + await _addColumnIfMissing(db, 'decoded_onehz', e.key, e.value); + } + } + static Future _ensureDayResultSkippedColumn(Database db) => _addColumnIfMissing( db, @@ -2205,9 +2242,20 @@ class LocalDb { az REAL NOT NULL, spo2_red_raw INTEGER NOT NULL, spo2_ir_raw INTEGER NOT NULL, - skin_temp_raw INTEGER NOT NULL + skin_temp_raw INTEGER NOT NULL, + step_count INTEGER, + step_cadence INTEGER, + activity_class INTEGER, + skin_temp_c REAL, + on_wrist INTEGER, + hr_valid INTEGER, + hr_alt INTEGER ) '''); + // Every band-computed column above is NULLABLE ON PURPOSE: only a gen5 band + // sends them, and a gen4 row must read back as "not reported", not as zero + // steps / 0 °C / "off wrist". No DEFAULT, ever. + await _ensureDecodedOneHzBandFields(db); // Forensic-only lookup by the raw counter; not on any read path. await db.execute( 'CREATE INDEX IF NOT EXISTS idx_decoded_onehz_counter ON decoded_onehz(counter)', @@ -2532,32 +2580,65 @@ class LocalDb { static String _localDayLabelFromEpoch(int epochSec) => _localDayLabel(DateTime.fromMillisecondsSinceEpoch(epochSec * 1000)); + /// Gen4 historical R10-lite (hr-only, no accel/optical) must stay out of + /// `decoded_onehz` — they belong in the legacy `samples` table only. + /// + /// Matched on the hex prefix rather than decoded bytes: this runs once per + /// record inside the offload transaction, and parsing 240 hex chars to read + /// two of them costs ~13 ms per 50k records plus a Uint8List of garbage each. + static final _gen4R10LitePrefix = + '${proto.PacketType.historicalData.toRadixString(16).padLeft(2, '0')}' + '${proto.Record.r10.toRadixString(16).padLeft(2, '0')}'; + static bool _isGen4R10LiteHistorical(String hex) => + hex.length >= 4 && hex.substring(0, 4).toLowerCase() == _gen4R10LitePrefix; + static Sample? _decodeOneHzSample(RawRecord raw, {Sample? preferred}) { + // Reject Gen4 R10-lite even when a complete preferred Sample is supplied. + // Invalid/placeholder hex (test fixtures, corrupt imports) must NOT abort + // before the preferred paths — commit 1f85b10 returned null on hexToBytes + // failure and zeroed decoded_onehz for every insertRecord that used non-hex + // placeholders. The bytes are now parsed lazily, below, for that reason too. + if (_isGen4R10LiteHistorical(raw.hex)) return null; if (preferred != null && preferred.hasDecodedOneHz) return preferred; + // The band's OWN decode outranks the other generation's decoder. A gen5 v18 + // has no gen4 optics so hasDecodedOneHz is false, and feeding it to the v24 + // map is not harmless: HR sits at byte 14 in both layouts and gravity is + // exactly one byte off, so whenever an axis lands in |g| ∈ [0.746, 0.75) + // the misread collapses into a believable vector instead of failing the + // plausibility gate — about 0.6% of records, ~550 rows a day. Those rows + // keep the right hr and ts (so they look fine) while carrying a fabricated + // gravity vector, fabricated optics, and none of the gen5 RR beats. + if (preferred != null && preferred.tsEpoch > 0) return preferred; + Uint8List? bytes; try { - // Legacy decoder first, firmware-fallback chain second — see - // FirmwareAwareR24Decoder. This path only runs when no pre-decoded - // `preferred` sample was supplied (e.g. a raw-hex import/merge), so a - // fresh per-call instance is fine — no session state to preserve. - final r = proto.FirmwareAwareR24Decoder().decode( - proto.hexToBytes(raw.hex), - ); - if (r == null || r.tsEpoch <= 0) return null; - return Sample( - tsEpoch: r.tsEpoch, - counter: r.counter, - hr: r.hr, - rrIntervalsMs: List.from(r.rrIntervalsMs), - ax: r.accelG.isNotEmpty ? r.accelG[0] : 0, - ay: r.accelG.length > 1 ? r.accelG[1] : 0, - az: r.accelG.length > 2 ? r.accelG[2] : 0, - spo2RedRaw: r.spo2RedRaw, - spo2IrRaw: r.spo2IrRaw, - skinTempRaw: r.skinTempRaw, - ); - } catch (_) { - return null; + bytes = proto.hexToBytes(raw.hex); + } catch (_) {} + if (bytes != null) { + try { + // Legacy decoder first, firmware-fallback chain second — see + // FirmwareAwareR24Decoder. This path only runs when no pre-decoded + // `preferred` sample was supplied (e.g. a raw-hex import/merge), so a + // fresh per-call instance is fine — no session state to preserve. + final r = proto.FirmwareAwareR24Decoder().decode(bytes); + if (r != null && r.tsEpoch > 0) { + return Sample( + tsEpoch: r.tsEpoch, + counter: r.counter, + hr: r.hr, + rrIntervalsMs: List.from(r.rrIntervalsMs), + ax: r.accelG.isNotEmpty ? r.accelG[0] : 0, + ay: r.accelG.length > 1 ? r.accelG[1] : 0, + az: r.accelG.length > 2 ? r.accelG[2] : 0, + spo2RedRaw: r.spo2RedRaw, + spo2IrRaw: r.spo2IrRaw, + // raw column passthrough, same as the ble path. not read as a temp. + // ignore: deprecated_member_use + skinTempRaw: r.skinTempRaw, + ); + } + } catch (_) {} } + return null; } /// Queues the decoded_onehz + decoded_rr writes for one raw onto [batch]. @@ -2598,6 +2679,16 @@ class LocalDb { 'spo2_red_raw': decoded.spo2RedRaw ?? 0, 'spo2_ir_raw': decoded.spo2IrRaw ?? 0, 'skin_temp_raw': decoded.skinTempRaw ?? 0, + // NO `?? 0` here, unlike the columns above: these are only reported by a + // gen5 band, so a null must land in the DB as NULL. Zeroing them would + // invent a 0-step second / a 0 °C skin temperature for every gen4 record. + 'step_count': decoded.stepCount, + 'step_cadence': decoded.stepCadence, + 'activity_class': decoded.activityClass, + 'skin_temp_c': decoded.skinTempC, + 'on_wrist': decoded.onWrist, + 'hr_valid': decoded.hrValid == null ? null : (decoded.hrValid! ? 1 : 0), + 'hr_alt': decoded.hrAlt, }, conflictAlgorithm: ConflictAlgorithm.replace); var ops = 1; // the decoded_onehz insert batch.rawDelete('DELETE FROM decoded_rr WHERE rec_ts = ?', [recTs]); @@ -3096,25 +3187,33 @@ class LocalDb { return db.query('sync_quarantine', orderBy: 'created_at DESC'); } + /// Columns [Sample.fromDecodedRow] reads. Deliberately NOT `*`: the accel / + /// spo2 / raw-skin-temp columns are bulk and nothing on these two paths uses + /// them (the derive path has its own wider query). + static const List _decodedSampleColumns = [ + 'counter', + 'rec_ts', + 'hr', + 'step_count', + 'step_cadence', + 'activity_class', + 'skin_temp_c', + 'on_wrist', + 'hr_valid', + 'hr_alt', + ]; + static Future> samplesInRange(int fromTs, int toTs) async { final db = await instance; final decodedRows = await db.query( 'decoded_onehz', - columns: ['counter', 'rec_ts', 'hr'], + columns: _decodedSampleColumns, where: 'rec_ts >= ? AND rec_ts <= ?', whereArgs: [fromTs, toTs], orderBy: 'rec_ts ASC, counter ASC', ); if (decodedRows.isNotEmpty) { - return decodedRows - .map( - (m) => Sample( - tsEpoch: (m['rec_ts'] as num).toInt(), - counter: (m['counter'] as num).toInt(), - hr: (m['hr'] as num?)?.toInt() ?? 0, - ), - ) - .toList(); + return decodedRows.map(Sample.fromDecodedRow).toList(); } final rows = await db.query( 'samples', @@ -3129,17 +3228,12 @@ class LocalDb { final db = await instance; final decodedRows = await db.query( 'decoded_onehz', - columns: ['counter', 'rec_ts', 'hr'], + columns: _decodedSampleColumns, orderBy: 'rec_ts DESC, counter DESC', limit: 1, ); if (decodedRows.isNotEmpty) { - final row = decodedRows.first; - return Sample( - tsEpoch: (row['rec_ts'] as num).toInt(), - counter: (row['counter'] as num).toInt(), - hr: (row['hr'] as num?)?.toInt() ?? 0, - ); + return Sample.fromDecodedRow(decodedRows.first); } final rows = await db.query('samples', orderBy: 'ts DESC', limit: 1); return rows.isEmpty ? null : Sample.fromDbMap(rows.first); @@ -3215,7 +3309,9 @@ class LocalDb { if (afterRecTs == null || afterCounter == null) { return db.rawQuery( 'SELECT counter, rec_ts, hr, ax, ay, az, ' - 'spo2_red_raw, spo2_ir_raw, skin_temp_raw ' + 'spo2_red_raw, spo2_ir_raw, skin_temp_raw, ' + 'step_count, step_cadence, activity_class, skin_temp_c, ' + 'on_wrist, hr_valid, hr_alt ' 'FROM decoded_onehz ' 'WHERE rec_ts >= ? AND rec_ts <= ? ' 'ORDER BY rec_ts ASC, counter ASC LIMIT ?', @@ -3224,7 +3320,9 @@ class LocalDb { } return db.rawQuery( 'SELECT counter, rec_ts, hr, ax, ay, az, ' - 'spo2_red_raw, spo2_ir_raw, skin_temp_raw ' + 'spo2_red_raw, spo2_ir_raw, skin_temp_raw, ' + 'step_count, step_cadence, activity_class, skin_temp_c, ' + 'on_wrist, hr_valid, hr_alt ' 'FROM decoded_onehz ' 'WHERE rec_ts >= ? AND rec_ts <= ? ' 'AND (rec_ts > ? OR (rec_ts = ? AND counter > ?)) ' diff --git a/lib/data/models.dart b/lib/data/models.dart index 9429626..b7cb6ff 100644 --- a/lib/data/models.dart +++ b/lib/data/models.dart @@ -16,6 +16,41 @@ class Sample { final int? spo2IrRaw; final int? skinTempRaw; + // ── Fields only a gen5 band sends (all null on gen4) ─────────────────────── + // NULL means "this band never reported it", and that is NOT the same as zero: + // a fabricated 0-step day or a 0 °C skin temperature reads as real data + // downstream. Every one of these stays nullable end to end, into the DB. + + /// Cumulative step count from the band's own pedometer, which runs whether or + /// not the app is connected. Monotonic — it does NOT reset at midnight, so a + /// day's steps are a difference between two records, never the value itself. + final int? stepCount; + + /// The band's own cadence for this second (steps/min). + final int? stepCadence; + + /// The band's own activity class: 0 = not committed to a class yet (NOT + /// "still" — do not count it as sedentary), 1 = walk, 2 = run. Null when the + /// band reported no valid class at all; a class is never invented from an + /// out-of-range code. + final int? activityClass; + + /// Calibrated skin temperature in °C, as computed by the band. Unlike + /// [skinTempRaw] (an uncalibrated ADC count that needs days of personal + /// baseline before it means anything) this is usable on its first second. + final double? skinTempC; + + /// The band's own on-wrist determination for this second (2-bit code). + final int? onWrist; + + /// The band's own "HR and RR are valid this second" flag. + final bool? hrValid; + + /// A second heart-rate byte the band reports alongside [hr]. It CORROBORATES + /// [hr] (agreement runs ~58-75%, best when [hrValid]); it is not a substitute + /// heart rate and must never be displayed as one. + final int? hrAlt; + Sample({ required this.tsEpoch, required this.counter, @@ -27,6 +62,13 @@ class Sample { this.spo2RedRaw, this.spo2IrRaw, this.skinTempRaw, + this.stepCount, + this.stepCadence, + this.activityClass, + this.skinTempC, + this.onWrist, + this.hrValid, + this.hrAlt, }); /// Copy with an overridden [tsEpoch] — used by the clock-offset salvage path @@ -44,6 +86,13 @@ class Sample { spo2RedRaw: spo2RedRaw, spo2IrRaw: spo2IrRaw, skinTempRaw: skinTempRaw, + stepCount: stepCount, + stepCadence: stepCadence, + activityClass: activityClass, + skinTempC: skinTempC, + onWrist: onWrist, + hrValid: hrValid, + hrAlt: hrAlt, ); bool get wristOn => hr > 0; @@ -66,6 +115,24 @@ class Sample { counter: m['counter'] as int, hr: m['hr'] as int, ); + + /// One `decoded_onehz` row → [Sample]. Missing/NULL columns stay null (a gen4 + /// row carries none of the band-computed fields), never 0. + factory Sample.fromDecodedRow(Map m) { + final valid = (m['hr_valid'] as num?)?.toInt(); + return Sample( + tsEpoch: (m['rec_ts'] as num).toInt(), + counter: (m['counter'] as num?)?.toInt() ?? 0, + hr: (m['hr'] as num?)?.toInt() ?? 0, + stepCount: (m['step_count'] as num?)?.toInt(), + stepCadence: (m['step_cadence'] as num?)?.toInt(), + activityClass: (m['activity_class'] as num?)?.toInt(), + skinTempC: (m['skin_temp_c'] as num?)?.toDouble(), + onWrist: (m['on_wrist'] as num?)?.toInt(), + hrValid: valid == null ? null : valid != 0, + hrAlt: (m['hr_alt'] as num?)?.toInt(), + ); + } } /// A raw historical record exactly as it came off the band — the source of truth. @@ -102,6 +169,15 @@ class RawRecord { /// archive the raw bytes durably (never pruned) so they can be re-decoded once /// the format is understood. Archived as part of the SAME durable commit that /// runs BEFORE the HISTORY_END ACK, so the safe-trim invariant holds. +/// [ArchiveRecord.reason] for a record the plausibility gate refused. +/// +/// Load-bearing, not a label: three separate decisions branch on it — whether +/// the burst counts as offload progress, whether the band may be told to trim, +/// and whether a derive pass is scheduled. A typo in any one of them either +/// wedges the offload or lets the band erase data we distrusted, so the string +/// exists once. +const String kGateDroppedReason = 'gate_dropped'; + class ArchiveRecord { final int counter; final String hex; // full inner bytes, hex @@ -174,6 +250,13 @@ class DeviceState { /// session-relative plausibility gate + the UI's "history available" readout. int? dataRangeOldest; int? dataRangeNewest; + /// Which WHOOP generation this connection is speaking — `'gen4'` or + /// `'gen5'`, set once at service discovery (see `BleEngine._doConnect`'s + /// `session.applyBand`). Null until a link has been established at least + /// once this process. Lets the UI show "WHOOP 5 connected" and gate any + /// gen5-only controls (e.g. a deep-buffer opt-in toggle) without reaching + /// into the transport layer. + String? generation; DeviceState({this.connection = 'disconnected'}); } diff --git a/lib/state/app_state.dart b/lib/state/app_state.dart index 288ff2b..489f10d 100644 --- a/lib/state/app_state.dart +++ b/lib/state/app_state.dart @@ -2087,12 +2087,11 @@ class AppState extends ChangeNotifier { if (breathingActive && (pt == 0x28 || pt == 0x2B)) { if (_breathingFrames.length < 8000) _breathingFrames.add(hex); } - // LIVE STEP COUNTER. The dedicated 0x33 IMU stream is the high-rate live - // accel — it arrives ~10 frames/s (10 samples each), so it drives a smooth, - // responsive count. Full R10 (0x2B) is only a fallback when the IMU stream - // isn't flowing (and live 0x2B is often R10-LITE, which carries no accel). - // `frameAccel` returns |a|(g) samples for both; once 0x33 is seen we ignore - // 0x2B to avoid double-counting the same motion from two stream formats. + // LIVE STEP COUNTER. Gen4: dedicated 0x33 IMU (~10 frames/s × 10 samples) + // is preferred; full R10 (0x2B) is only a fallback when 0x33 isn't flowing. + // Gen5 Maverick: live IMU is 0x2B (rec 0x15, 100 Hz planar) — see + // protocol's frameAccelForBand. Once gen4 0x33 is seen we ignore 0x2B to avoid + // double-counting the same motion from two stream formats. if (pt == 0x33) { _imuStreamSeen = true; final f = _safeFrameAccel(hex); @@ -2101,6 +2100,7 @@ class AppState extends ChangeNotifier { _trackCoverage(recTs); } } else if (pt == 0x2B && !_imuStreamSeen) { + // Gen5 Maverick live IMU is 0x2B (100 Hz planar), not top-level 0x33. final f = _safeFrameAccel(hex); if (f != null) { _ingestLiveMags(f); @@ -2111,7 +2111,9 @@ class AppState extends ChangeNotifier { proto.ImuFrame? _safeFrameAccel(String hex) { try { - return proto.frameAccel(hex); + // Gen5 Maverick live IMU is 0x2B; gen4 stays on frameAccel (0x33 / R10). + // protocol's gen5 path abstains unless the record is the IMU buffer. + return proto.frameAccelForBand(hex); } catch (_) { return null; } @@ -3031,20 +3033,18 @@ class AppState extends ChangeNotifier { Future setAlarm(DateTime when) async { if (!isConnected) throw Exception('Connect to your strap first'); - final epoch = - when.millisecondsSinceEpoch ~/ 1000; // local wall-clock → unix // Pass the DateTime through so the engine computes REAL sub-seconds for the // rich 20-byte firing form (a hardcoded 0 subsec would still fire, but the - // engine owns the exact on-wire layout). - final ok = await engine.setAlarm(when); - if (!ok) { - // The arm write never reached the band — do NOT persist or start the - // confirmation machine, or we'd strand a phantom alarm "waiting for the - // strap to confirm" that can never fire. Surface it so the UI reflects - // "couldn't send" (the coach/profile callers snackbar on a throw). + // engine owns the exact on-wire layout). Persist the wall instant the + // engine reports armed (null = write never reached the band). + final armed = await engine.setAlarm(when); + if (armed == null) { + // Do NOT persist or start the confirmation machine, or we'd strand a + // phantom alarm "waiting for the strap to confirm" that can never fire. _log('[alarm] arm write FAILED — not persisting; alarm not set.'); throw Exception('Alarm not sent — the strap did not accept the write'); } + final epoch = armed.millisecondsSinceEpoch ~/ 1000; _savedAlarm = epoch; device.alarmEpoch = epoch; // optimistic display _alarm.set(epoch, DateTime.now().millisecondsSinceEpoch); // await event 56 @@ -3084,7 +3084,10 @@ class AppState extends ChangeNotifier { _alarmAutoRetried = true; var rearmed = false; try { - rearmed = await engine.setAlarm(when); + // gen5 made setAlarm return the armed instant (null = the write never + // reached the band) where it used to return a bool. Same signal, so the + // retry bookkeeping below is unchanged. + rearmed = await engine.setAlarm(when) != null; } catch (e) { _log('[alarm] auto-retry re-arm failed: $e'); } @@ -3612,7 +3615,8 @@ class AppState extends ChangeNotifier { enabled: plan.shouldEnable, targetWake: plan.targetWake, duration: HighFreqWakeWindow.lease, - intervalSeconds: 60, + intervalSeconds: 61, // gen5 rejects <= 60 + reason: plan.source, ); _log( diff --git a/lib/sync/background_sync.dart b/lib/sync/background_sync.dart index d9847ee..6894e42 100644 --- a/lib/sync/background_sync.dart +++ b/lib/sync/background_sync.dart @@ -99,7 +99,8 @@ Future runHeadlessSync({BandLease? lease}) async { enabled: plan.shouldEnable, targetWake: plan.targetWake, duration: HighFreqWakeWindow.lease, - intervalSeconds: 60, + intervalSeconds: 61, // gen5 rejects <= 60 + reason: plan.source, ); debugPrint( diff --git a/lib/ui/profile/profile_screen.dart b/lib/ui/profile/profile_screen.dart index 03a2a1d..67118e1 100644 --- a/lib/ui/profile/profile_screen.dart +++ b/lib/ui/profile/profile_screen.dart @@ -790,6 +790,11 @@ class ProfileScreen extends StatelessWidget { : '${d.batteryPct!.round()}%${d.charging == true ? ' ⚡' : ''}', wrist: d.wristOn == null ? '—' : (d.wristOn! ? 'On wrist' : 'Off wrist'), serial: d.serial ?? app.paired?.serial ?? '—', + generation: switch (d.generation) { + 'gen5' => 'WHOOP 5 (experimental)', + 'gen4' => 'WHOOP 4', + _ => null, + }, // Manual pull: anything the strap flashed that we don't hold yet, over // the CURRENT connection (no reconnect). Only offered while connected. onSyncNow: conn == 'connected' ? () => app.forceResync() : null, @@ -1050,6 +1055,11 @@ class DeviceTile extends StatefulWidget { final VoidCallback? onTap; final Future Function()? onSyncNow; + /// Human label for [DeviceState.generation] ('WHOOP 4' / 'WHOOP 5 + /// (experimental)'), or null before a link has been established this + /// process. Purely informational — never gates any behavior here. + final String? generation; + const DeviceTile({ super.key, required this.name, @@ -1060,6 +1070,7 @@ class DeviceTile extends StatefulWidget { required this.serial, this.onTap, this.onSyncNow, + this.generation, }); @override @@ -1114,7 +1125,22 @@ class _DeviceTileState extends State { overflow: TextOverflow.ellipsis, ), const SizedBox(height: Sp.x2), - StatusChip(widget.statusText, tone: widget.statusTone), + // Wrap, not Row: these are two intrinsically-sized chips + // with no flex, and the second one carries a long label + // ("WHOOP 5 (experimental)"). At large text scales, or on + // a narrow device, their combined width exceeds the + // Expanded column and a Row overflows. Wrapping degrades + // to a second line instead. + Wrap( + spacing: Sp.x2, + runSpacing: Sp.x2, + children: [ + StatusChip(widget.statusText, tone: widget.statusTone), + if (widget.generation != null) + StatusChip(widget.generation!, + tone: ChipTone.neutral), + ], + ), ], ), ), @@ -1617,9 +1643,26 @@ class _DeviceSheet extends StatelessWidget { // blanket watch() before; select the fields actually used instead. (Prior // pass here missed `device`/`paired` — re-audited against every `live.` // touchpoint in this class after finding the same gap cost a real bug in - // the main ProfileScreen build above.) - context.select( - (a) => (a.isConnected, a.alarmEpoch, a.strapName, a.device, a.paired), + // the main ProfileScreen build above.) Also select confirmation flags: + // omitting them left the caption stuck on "Setting alarm…" after grace. + // + // Select the SERIAL VALUE, not the `device`/`paired` OBJECTS. `select` + // compares with `==`, `DeviceState` declares no `==`/`hashCode` (so it is + // identity equality), and `BleEngine` mutates `state.serial` IN PLACE — + // the selector therefore returns the same reference before and after, no + // change is detected, and this row can sit on a stale serial indefinitely. + // Selecting the string the row actually renders makes the dependency real. + context.select( + (a) => ( + a.isConnected, + a.alarmEpoch, + a.strapName, + a.device.serial ?? a.paired?.serial, + a.alarmConfirmed, + a.alarmPending, + a.alarmUnconfirmed, + ), ); final live = context.read(); final connected = live.isConnected; diff --git a/pubspec.lock b/pubspec.lock index 566a91a..14663ca 100644 --- a/pubspec.lock +++ b/pubspec.lock @@ -973,8 +973,8 @@ packages: dependency: "direct main" description: path: "." - ref: "7edcb3e377329968118c62cb03a81d95e2f6db8e" - resolved-ref: "7edcb3e377329968118c62cb03a81d95e2f6db8e" + ref: "3ea0290c924cec1a1877bec1cc50db2e7e93a0f2" + resolved-ref: "3ea0290c924cec1a1877bec1cc50db2e7e93a0f2" url: "https://github.com/OpenStrap/protocol.git" source: git version: "1.0.0" diff --git a/pubspec.yaml b/pubspec.yaml index 14b6efa..662fe4c 100644 --- a/pubspec.yaml +++ b/pubspec.yaml @@ -27,37 +27,28 @@ dependencies: # 0.9.13/0.9.14 with no edge change and shipped the main-thread staging ANRs). # Bump these SHAs deliberately, as part of a reviewed edge change. Local dev # still builds against ../analytics and ../protocol via pubspec_overrides.yaml. - # NOTE: this branch is intentionally WHOOP-4-only — protocol stays on `main`, - # NOT the gen5/multiband branch. - # Both are now MERGE COMMITS ON `main`, not PR-branch heads — the PR-branch - # SHAs these briefly pointed at are no longer the canonical location of the - # change, and a branch deletion could orphan them. + # NOTE: this is the multiband branch, so protocol is pinned at the gen5 work + # rather than plain `main`. Prefer merge commits on `main` over PR-branch + # heads where possible — a deleted branch can orphan a PR-branch SHA. openstrap_protocol: git: url: https://github.com/OpenStrap/protocol.git - # Tip of protocol main — OpenStrap/protocol#20 merged: the reassembler now - # checks the length-field crc8 before trusting it (a corrupted length byte - # used to consume up to 4092 bytes of good stream before this), and - # hexToBytes rejects odd-length hex instead of silently flooring it. This - # PR sat unpinned for a full day after merging — see fix/issues #22 for - # why (edge's own release pipeline shipped the pre-fix decoder). - # protocol main @ #21 merge. Picks up the realtimeRr RR-bound (live.dart - # accepted ANY positive int16 as an interval, unlike parseRealtimeHr and - # parseR24 which both gate 200-2500ms, so a misaligned 0x28 frame could - # hand a 5ms "beat" to live HRV/coherence) plus the historical-family - # activity/steps_inc null-instead-of-0 fix. Edge does not read the latter - # two fields, so only the RR bound is behaviour-visible here. - # Verified present: `git show :lib/src/live.dart | grep -c kMinRrMs`. - # DELIBERATELY NOT MOVED for the clock-gate work in this PR. Correlating - # a GET_CLOCK reply to its own request needs the echoed request seq - # surfaced (OpenStrap/protocol#28), and the only in-convention way to pin - # that is a merge commit on protocol main — which now also carries the - # gen5/multiband surface (gen5_records.dart, the frame-revision changes - # in framing.dart, new dangerousCmds entries). Adopting ~1850 lines of - # that as a side effect of a one-field addition is the pin drift that - # shipped the v42 ANRs. _readClock degrades to accepting any fresh - # clock_epoch, which is what it did before the correlation existed. - ref: 7edcb3e377329968118c62cb03a81d95e2f6db8e + # protocol PR #27 head. main deliberately stays on a pre-gen5 SHA so the + # multiband surface doesn't ride in as a side effect of unrelated work — + # that is the right call THERE. This is the branch that wants it. + # + # Carries the gen5 decoders plus the fixes from #27: command responses + # have two more header bytes than we assumed (body location was reading + # the status byte as the location), GET_CLOCK/GET_DATA_RANGE read their + # fields instead of scanning for them, the R22 config values were + # inverted, and the gen4 optical fields are deprecated with what they + # actually are. + # protocol main @ #27 merge. Carries the gen5 command/decoder fixes plus + # the decode-integrity work: a record only decodes if its packet type says + # it is one, r-r beats are read only where the field map is confirmed, and + # a frame whose payload crc32 fails no longer lets its declared length + # swallow the frames behind it. + ref: 3ea0290c924cec1a1877bec1cc50db2e7e93a0f2 openstrap_analytics: git: url: https://github.com/OpenStrap/analytics.git diff --git a/test/alarm_test.dart b/test/alarm_test.dart index 215ca97..b5acee9 100644 --- a/test/alarm_test.dart +++ b/test/alarm_test.dart @@ -59,6 +59,30 @@ void main() { expect(p, [0x01, 0x04, 0x03, 0x02, 0x01, 0x00, 0x40]); }); + test('setPayloadForBand: gen4 index0 rich, gen5 index1 rich', () { + final g4 = AlarmPayloads.setPayloadForBand(when, isGen5: false); + final g5 = AlarmPayloads.setPayloadForBand(when, isGen5: true); + expect(g4.length, 20); + expect(g4[0], 0x04); + expect(g4[1], 0x00); + expect(g5.length, 21); // gen5 adds the crescendo byte + expect(g5[0], 0x04); + expect(g5[1], 0x01); // gen5 arms slot 1 + expect(g5.sublist(8, 20), AlarmPayloads.defaultHaptics); + expect(g5[20], 0, reason: 'crescendo flag, off by default'); + // Gen5 ignores a caller-supplied index so slot 0 cannot be armed by accident. + expect( + AlarmPayloads.setPayloadForBand(when, isGen5: true, index: 0)[1], + 0x01, + ); + }); + + test('gen5 Maverick buzz is a short Find-band-style pulse', () { + expect(AlarmPayloads.gen5MaverickBuzz(), + [0x01, 47, 152, 0, 0, 0, 0, 0, 0, 0, 0, 1]); + expect(AlarmPayloads.gen5MaverickBuzz(overallLoop: 7).last, 7); + }); + test('RUN_ALARM + DISABLE_ALARM bodies are both [0x01]', () { expect(AlarmPayloads.runNow, [0x01]); expect(AlarmPayloads.disable, [0x01]); diff --git a/test/gen5_decoded_onehz_persistence_test.dart b/test/gen5_decoded_onehz_persistence_test.dart new file mode 100644 index 0000000..f54f448 --- /dev/null +++ b/test/gen5_decoded_onehz_persistence_test.dart @@ -0,0 +1,295 @@ +// Gen5 v18 samples must land in `decoded_onehz` via the preferred-sample +// fallback in LocalDb._decodeOneHzSample — they lack gen4 optics so R24 +// decode fails, but they are honest 1 Hz substrate rows. R10-lite hr-only +// records must stay excluded. + +import 'dart:typed_data'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:openstrap_edge/ble/ble_engine.dart'; +import 'package:openstrap_edge/data/db.dart'; +import 'package:openstrap_edge/data/models.dart'; +import 'package:openstrap_protocol/openstrap_protocol.dart'; +import 'package:path/path.dart' as p; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; + +String _bytesToHex(Uint8List bytes) => + bytes.map((b) => b.toRadixString(16).padLeft(2, '0')).join(); + +Uint8List _buildR10LiteInner({required int ts, required int counter, required int hr}) { + final inner = Uint8List(18); + inner[0] = PacketType.historicalData; + inner[1] = Record.r10; + inner.buffer.asByteData().setUint32(3, counter, Endian.little); + inner.buffer.asByteData().setUint32(7, ts, Endian.little); + inner[17] = hr; + return inner; +} + +/// Synthetic gen5 v18 lenient inner: valid unix@7 + HR, gravity fails gate. +Uint8List _buildGen5V18LenientInner({ + required int unix, + required int counter, + required int hr, + List rrMs = const [], +}) { + final inner = Uint8List(112); + inner[0] = PacketType.historicalData; + inner[1] = 18; + inner[2] = 0x80; + inner[3] = counter & 0xff; + inner[4] = (counter >> 8) & 0xff; + inner[5] = (counter >> 16) & 0xff; + inner[6] = (counter >> 24) & 0xff; + inner.buffer.asByteData().setUint32(7, unix, Endian.little); + inner[14] = hr; + inner[15] = rrMs.length.clamp(0, 4).toInt(); + final view = inner.buffer.asByteData(); + for (var i = 0; i < rrMs.length && i < 4; i++) { + view.setInt16(16 + 2 * i, rrMs[i], Endian.little); + } + view.setFloat32(33, 0.5, Endian.little); + view.setFloat32(37, 0.05, Endian.little); + view.setFloat32(41, 0.05, Endian.little); + view.setFloat32(45, 0.05, Endian.little); + return inner; +} + +void main() { + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + LocalDb.dbName = 'openstrap_gen5_onehz_test.db'; + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + tearDownAll(() async { + await LocalDb.close(); + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + group('gen5 → decoded_onehz persistence', () { + test('gen5 v18-shaped sample persists via preferred fallback (+ RR)', () async { + // Real fixture inner — same bytes as gen5_sample_mapping_test.dart. + final frameHex = + 'aa01740001003fb12f1280733d8401b69f266a66460066025a0265020000000' + '000007b0a8d656463ff0012163cf6a439bf2924fd3ed763fe3e3200aa000000' + '000000000000f7000901f10b0007010c020c000000000000000000000000000' + '00000000000000000000100656f1e1e0000009d61a7c00000003e862817'; + final frame = Uint8List.fromList( + List.generate(frameHex.length ~/ 2, (i) { + return int.parse(frameHex.substring(i * 2, i * 2 + 2), radix: 16); + }), + ); + final parsed = parseFrame(frame, profile: BandProfile.gen5)!; + final inner = parsed.inner; + final sample = sampleFromGen5Historical(parseGen5Historical(inner)); + expect(sample, isNotNull); + + const recTs = 1780916150; + final raw = RawRecord( + counter: sample!.counter, + packetType: PacketType.historicalData, + hex: _bytesToHex(inner), + capturedAt: recTs * 1000, + recTs: recTs, + ); + + await LocalDb.commitSyncBatch([raw], [sample]); + + final db = await LocalDb.instance; + final rows = await db.query( + 'decoded_onehz', + where: 'rec_ts = ?', + whereArgs: [recTs], + ); + expect(rows.length, 1); + expect(rows.first['hr'], 102); + expect(rows.first['counter'], sample.counter); + + final rr = await db.query( + 'decoded_rr', + where: 'rec_ts = ?', + whereArgs: [recTs], + ); + expect(rr.length, 2); + expect([for (final r in rr) r['rr_ms']], containsAll([602, 613])); + }); + + // NOTE what this pins, and what it does NOT. `decoded_onehz.ax/ay/az` are + // REAL NOT NULL, so absent gravity has to be STORED as 0 — that is a schema + // constraint, not a claim about the wrist. Exact (0,0,0) is therefore the + // ABSENT marker (no real gravity vector has zero magnitude, and every decoder + // that emits one gates on magSq >= 0.25); `Substrate.accelPresentAt` is what + // stops it being read back as a measurement. See + // substrate_accel_absence_test.dart — without that, a night of these scores + // as perfect immobility and fabricates a fully-staged sleep window. + test('v18 sample with null accel still persists (stored as 0)', () async { + const unix = 1785801600; + const counter = 42; + final inner = _buildGen5V18LenientInner(unix: unix, counter: counter, hr: 72); + // Built directly rather than decoded: what this test pins is the + // PERSISTENCE of a null-accel sample, independent of which decoder produced + // it. Protocol emits exactly this shape for a v18 whose gravity vector + // fails the magnitude gate — the rest of the second is kept. + final sample = Sample(tsEpoch: unix, counter: counter, hr: 72); + expect(sample.ax, isNull); + + final raw = RawRecord( + counter: counter, + packetType: PacketType.historicalData, + hex: _bytesToHex(inner), + capturedAt: unix * 1000, + recTs: unix, + ); + await LocalDb.commitSyncBatch([raw], [sample]); + + final db = await LocalDb.instance; + final rows = await db.query( + 'decoded_onehz', + where: 'rec_ts = ?', + whereArgs: [unix], + ); + expect(rows.length, 1); + expect(rows.first['hr'], 72); + expect(rows.first['ax'], 0); + expect(rows.first['ay'], 0); + expect(rows.first['az'], 0); + }); + + test('R10-lite + complete preferred → no decoded_onehz row', () async { + const ts = 1780000100; + const counter = 99; + final inner = _buildR10LiteInner(ts: ts, counter: counter, hr: 65); + final preferred = Sample( + tsEpoch: ts, + counter: counter, + hr: 65, + ax: 0.1, + ay: -0.2, + az: 0.95, + spo2RedRaw: 100, + spo2IrRaw: 200, + skinTempRaw: 300, + ); + expect(preferred.hasDecodedOneHz, isTrue); + final raw = RawRecord( + counter: counter, + packetType: PacketType.historicalData, + hex: _bytesToHex(inner), + capturedAt: ts * 1000, + recTs: ts, + ); + + await LocalDb.commitSyncBatch([raw], [preferred]); + + final db = await LocalDb.instance; + final rows = await db.query( + 'decoded_onehz', + where: 'rec_ts = ?', + whereArgs: [ts], + ); + expect(rows, isEmpty); + }); + + test('full gen4 R24 sample still persists', () async { + const ts = 1780000200; + const counter = 5001; + final sample = Sample( + tsEpoch: ts, + counter: counter, + hr: 70, + rrIntervalsMs: [800], + ax: 0.1, + ay: -0.2, + az: 0.95, + spo2RedRaw: 100, + spo2IrRaw: 200, + skinTempRaw: 300, + ); + // Minimal non-R10 historical hex — R24 decode won't match, but preferred + // has full gen4 optics so _decodeOneHzSample returns it immediately. + final raw = RawRecord( + counter: counter, + packetType: PacketType.historicalData, + hex: '2f18' '00' * 20, + capturedAt: ts * 1000, + recTs: ts, + ); + + await LocalDb.commitSyncBatch([raw], [sample]); + + final db = await LocalDb.instance; + final rows = await db.query( + 'decoded_onehz', + where: 'rec_ts = ?', + whereArgs: [ts], + ); + expect(rows.length, 1); + expect(rows.first['hr'], 70); + expect(rows.first['spo2_red_raw'], 100); + }); + }); + + // CodeRabbit: the R10-lite case asserted only ABSENCE from `decoded_onehz`, + // which would also pass if the record were dropped entirely. Retention in + // `samples` is the other half of that contract. + test('an R10-lite record is excluded from decoded_onehz but RETAINED in samples', + () async { + const ts = 1780000300; + const counter = 4242; + final inner = _buildR10LiteInner(ts: ts, counter: counter, hr: 71); + final sample = Sample(tsEpoch: ts, counter: counter, hr: 71); + final raw = RawRecord( + counter: counter, + packetType: PacketType.historicalData, + hex: _bytesToHex(inner), + capturedAt: ts * 1000, + recTs: ts, + ); + await LocalDb.commitSyncBatch([raw], [sample]); + + final db = await LocalDb.instance; + expect( + await db.query('decoded_onehz', where: 'rec_ts = ?', whereArgs: [ts]), + isEmpty, + reason: 'hr-only R10-lite is not 1 Hz substrate', + ); + expect( + await db.query('samples', where: 'counter = ?', whereArgs: [counter]), + hasLength(1), + reason: 'excluded from the substrate is NOT the same as discarded', + ); + }); + + // Protects the hex-conversion fallback in `LocalDb._decodeOneHzSample`: when + // the raw hex cannot be parsed, a timestamp-valid preferred Sample must still + // reach `decoded_onehz` rather than the record being lost. + test('unparseable raw hex still persists a timestamp-valid preferred Sample', + () async { + const ts = 1780000400; + const counter = 5150; + final raw = RawRecord( + counter: counter, + packetType: PacketType.historicalData, + hex: 'zzzz-not-hex', + capturedAt: ts * 1000, + recTs: ts, + ); + final sample = Sample( + tsEpoch: ts, + counter: counter, + hr: 66, + rrIntervalsMs: const [910], + ); + await LocalDb.commitSyncBatch([raw], [sample]); + + final db = await LocalDb.instance; + final rows = + await db.query('decoded_onehz', where: 'rec_ts = ?', whereArgs: [ts]); + expect(rows, hasLength(1)); + expect(rows.first['hr'], 66); + }); +} diff --git a/test/gen5_sample_fields_test.dart b/test/gen5_sample_fields_test.dart new file mode 100644 index 0000000..e0031f1 --- /dev/null +++ b/test/gen5_sample_fields_test.dart @@ -0,0 +1,317 @@ +// The per-second fields a gen5 band computes ITSELF — its pedometer's +// cumulative step count and cadence, its activity class, a calibrated skin +// temperature in °C, its on-wrist determination, and the HR-validity flag plus +// the corroborating second HR byte — are decoded off every record and now +// PERSISTED (schema v34) instead of being dropped on the floor. +// +// The invariant these tests exist to protect is ABSENCE, not presence: a gen4 +// band sends none of this, so a gen4 second must store NULL. Zeroing them would +// mint a 0-step second and a 0 °C skin temperature for every gen4 record in the +// ledger — indistinguishable from a real reading downstream, and exactly the +// class of fabrication this codebase keeps having to undo. +// +// Runs the REAL LocalDb over sqflite_ffi, so the DDL, the migration ladder and +// the read paths are the shipping ones. + +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; +import 'package:openstrap_edge/data/db.dart'; +import 'package:openstrap_edge/data/models.dart'; + +/// The v33 `decoded_onehz` / `decoded_rr` shape — rec_ts-keyed, WITHOUT any of +/// the band-computed columns. This is what an existing install's (million-row) +/// table looks like before the upgrade. +const _v33DecodedDdl = [ + ''' + CREATE TABLE decoded_onehz ( + rec_ts INTEGER PRIMARY KEY, + counter INTEGER NOT NULL, + hr INTEGER NOT NULL, + ax REAL NOT NULL, ay REAL NOT NULL, az REAL NOT NULL, + spo2_red_raw INTEGER NOT NULL, + spo2_ir_raw INTEGER NOT NULL, + skin_temp_raw INTEGER NOT NULL) +''', + 'CREATE INDEX idx_decoded_onehz_counter ON decoded_onehz(counter)', + ''' + CREATE TABLE decoded_rr ( + rec_ts INTEGER NOT NULL, beat_index INTEGER NOT NULL, + rr_ts_ms INTEGER NOT NULL, rr_ms INTEGER NOT NULL, + PRIMARY KEY (rec_ts, beat_index)) +''', +]; + +Future _dbPath(String name) async => + p.join(await databaseFactory.getDatabasesPath(), name); + +/// Point LocalDb at a fresh, empty database file. +Future _useFreshDb(String name) async { + await LocalDb.close(); + LocalDb.dbName = name; + await databaseFactory.deleteDatabase(await _dbPath(name)); +} + +RawRecord _raw(int recTs, int counter) => RawRecord( + counter: counter, + packetType: 47, + // Not hex: forces the decode helper onto the supplied (already decoded) + // Sample, which is how the BLE engine hands records over anyway. + hex: 'decoded-by-the-supplied-sample', + capturedAt: recTs * 1000, + recTs: recTs, +); + +/// A row straight out of `decoded_onehz`, so NULL can be told from 0. +Future> _rowAt(int recTs) async { + final db = await LocalDb.instance; + final rows = await db.query( + 'decoded_onehz', + where: 'rec_ts = ?', + whereArgs: [recTs], + ); + return rows.single; +} + +void main() { + final created = []; + + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + }); + + tearDownAll(() async { + await LocalDb.close(); + for (final n in created) { + await databaseFactory.deleteDatabase(await _dbPath(n)); + } + }); + + test('a gen5 second round-trips every band-computed field', () async { + const name = 'openstrap_gen5_fields_test.db'; + created.add(name); + await _useFreshDb(name); + + const recTs = 1785000000; + await LocalDb.insertRecord( + _raw(recTs, 9001), + Sample( + tsEpoch: recTs, + counter: 9001, + hr: 58, + rrIntervalsMs: const [1010], + ax: 0.02, + ay: -0.05, + az: 0.99, + spo2RedRaw: 1111, + spo2IrRaw: 2222, + skinTempRaw: 3333, + stepCount: 12345, + stepCadence: 94, + activityClass: 2, + skinTempC: 30.62, + onWrist: 1, + hrValid: true, + hrAlt: 59, + ), + ); + + // The derive read path must actually SELECT the new columns — a column that + // is written but never read back is the same as not storing it. + final frames = await LocalDb.decodedOneHzBatchByRecTsRange( + limit: 10, + fromRecTs: recTs - 1, + toRecTs: recTs + 1, + ); + expect(frames, hasLength(1)); + final f = frames.single; + expect(f['step_count'], 12345); + expect(f['step_cadence'], 94); + expect(f['activity_class'], 2); + expect((f['skin_temp_c'] as num).toDouble(), closeTo(30.62, 1e-9)); + expect(f['on_wrist'], 1); + expect(f['hr_valid'], 1); + expect(f['hr_alt'], 59); + // The pre-existing columns are untouched by the widening. + expect(f['hr'], 58); + expect(f['skin_temp_raw'], 3333); + + // …and the Sample-returning paths carry them back as typed fields. + final ranged = await LocalDb.samplesInRange(recTs - 1, recTs + 1); + expect(ranged, hasLength(1)); + expect(ranged.single.stepCount, 12345); + expect(ranged.single.stepCadence, 94); + expect(ranged.single.activityClass, 2); + expect(ranged.single.skinTempC, closeTo(30.62, 1e-9)); + expect(ranged.single.onWrist, 1); + expect(ranged.single.hrValid, isTrue); + expect(ranged.single.hrAlt, 59); + + final latest = await LocalDb.latestSample(); + expect(latest!.stepCount, 12345); + expect(latest.hrValid, isTrue); + + // hrValid FALSE must survive as false, not collapse to null/absent. + await LocalDb.insertRecord( + _raw(recTs + 1, 9002), + Sample( + tsEpoch: recTs + 1, + counter: 9002, + hr: 0, + ax: 0, + ay: 0, + az: 1, + spo2RedRaw: 0, + spo2IrRaw: 0, + skinTempRaw: 0, + stepCount: 12345, + hrValid: false, + ), + ); + expect((await _rowAt(recTs + 1))['hr_valid'], 0); + expect((await LocalDb.latestSample())!.hrValid, isFalse); + }); + + test('a gen4 second stores NULL — never a fabricated 0 — for all of them', + () async { + const name = 'openstrap_gen4_null_fields_test.db'; + created.add(name); + await _useFreshDb(name); + + const recTs = 1785100000; + await LocalDb.insertRecord( + _raw(recTs, 4001), + // Exactly what a gen4 record decodes to: no step count, no cadence, no + // class, no calibrated temperature, no wear bits, no HR-quality flag. + Sample( + tsEpoch: recTs, + counter: 4001, + hr: 62, + rrIntervalsMs: const [960], + ax: 0.1, + ay: -0.2, + az: 0.97, + spo2RedRaw: 1234, + spo2IrRaw: 2345, + skinTempRaw: 3456, + ), + ); + + final row = await _rowAt(recTs); + for (final c in const [ + 'step_count', + 'step_cadence', + 'activity_class', + 'skin_temp_c', + 'on_wrist', + 'hr_valid', + 'hr_alt', + ]) { + expect(row[c], isNull, reason: '$c must be NULL on a gen4 record, not 0'); + } + // The gen4 fields it DOES carry are stored as before. + expect(row['hr'], 62); + expect(row['skin_temp_raw'], 3456); + + final s = await LocalDb.latestSample(); + expect(s!.stepCount, isNull); + expect(s.skinTempC, isNull); + expect(s.hrValid, isNull); + expect(s.activityClass, isNull); + }); + + test('upgrading a populated v33 database keeps its rows readable', () async { + const name = 'openstrap_v33_upgrade_fields_test.db'; + created.add(name); + final path = await _dbPath(name); + await LocalDb.close(); + await databaseFactory.deleteDatabase(path); + + const recTs = 1780000000; + final old = await databaseFactory.openDatabase( + path, + options: OpenDatabaseOptions( + version: 33, + onCreate: (db, _) async { + for (final s in _v33DecodedDdl) { + await db.execute(s); + } + }, + ), + ); + await old.insert('decoded_onehz', { + 'rec_ts': recTs, + 'counter': 777, + 'hr': 55, + 'ax': 0.01, + 'ay': 0.02, + 'az': 0.99, + 'spo2_red_raw': 10, + 'spo2_ir_raw': 20, + 'skin_temp_raw': 30, + }); + await old.insert('decoded_rr', { + 'rec_ts': recTs, + 'beat_index': 0, + 'rr_ts_ms': recTs * 1000, + 'rr_ms': 1090, + }); + await old.close(); + + LocalDb.dbName = name; + final db = await LocalDb.instance; + expect( + ((await db.rawQuery('PRAGMA user_version')).first.values.first as num) + .toInt(), + LocalDb.schemaVersion, + ); + + // The pre-migration row survived intact, and reads back through the same + // widened query the derive path uses. + final frames = await LocalDb.decodedOneHzBatchByRecTsRange( + limit: 10, + fromRecTs: recTs - 1, + toRecTs: recTs + 1, + ); + expect(frames, hasLength(1)); + expect(frames.single['hr'], 55); + expect(frames.single['skin_temp_raw'], 30); + // It predates the columns, so it reports nothing for them — not zero. + expect(frames.single['step_count'], isNull); + expect(frames.single['skin_temp_c'], isNull); + expect((await LocalDb.latestSample())!.stepCount, isNull); + + final rr = await LocalDb.decodedRrByRecTsRange( + fromRecTs: recTs, + toRecTs: recTs, + ); + expect(rr.single['rr_ms'], 1090); + + // A NEW gen5 second lands beside the migrated one in the upgraded table. + await LocalDb.insertRecord( + _raw(recTs + 60, 778), + Sample( + tsEpoch: recTs + 60, + counter: 778, + hr: 57, + ax: 0, + ay: 0, + az: 1, + spo2RedRaw: 0, + spo2IrRaw: 0, + skinTempRaw: 0, + stepCount: 4242, + skinTempC: 22.5, + onWrist: 0, + ), + ); + final fresh = await _rowAt(recTs + 60); + expect(fresh['step_count'], 4242); + expect((fresh['skin_temp_c'] as num).toDouble(), closeTo(22.5, 1e-9)); + expect(fresh['on_wrist'], 0); + expect(fresh['hr_alt'], isNull); + + expect((await LocalDb.schemaHealth())['ok'], isTrue); + }); +} diff --git a/test/gen5_sample_mapping_test.dart b/test/gen5_sample_mapping_test.dart new file mode 100644 index 0000000..37c611a --- /dev/null +++ b/test/gen5_sample_mapping_test.dart @@ -0,0 +1,100 @@ +// Tests for BleEngine's gen5 -> band-agnostic Sample mapping +// (sampleFromGen5Historical) — the seam that turns protocol's typed gen5 +// historical-record decode into the same `Sample` shape gen4 records +// produce, so the derivation pipeline / analytics stay band-agnostic. +// +// The v18 fixture is the same real, independently byte-verified capture used +// by protocol's own gen5_historical_test.dart (CRC16-modbus header + CRC32 +// payload both check out; see that file's header comment for provenance) — +// reused here rather than re-typed, so a transcription slip can't silently +// diverge the two test suites' expectations. + +import 'dart:typed_data'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:openstrap_edge/ble/ble_engine.dart'; +import 'package:openstrap_edge/data/models.dart'; +import 'package:openstrap_protocol/openstrap_protocol.dart'; + +Uint8List hex(String s) { + final clean = s.replaceAll(' ', ''); + final out = Uint8List(clean.length ~/ 2); + for (int i = 0; i < out.length; i++) { + out[i] = int.parse(clean.substring(i * 2, i * 2 + 2), radix: 16); + } + return out; +} + +void main() { + group('sampleFromGen5Historical — v18 (real fixture)', () { + // "worn" capture, unix=1780916150 — CRC16+CRC32 both verified. Same + // bytes as protocol/test/gen5_historical_test.dart's v18 real fixture. + final frame = hex( + 'aa01740001003fb12f1280733d8401b69f266a66460066025a0265020000000' + '000007b0a8d656463ff0012163cf6a439bf2924fd3ed763fe3e3200aa000000' + '000000000000f7000901f10b0007010c020c000000000000000000000000000' + '00000000000000000000100656f1e1e0000009d61a7c00000003e862817', + ); + + late Sample? sample; + + setUp(() { + final parsed = parseFrame(frame, profile: BandProfile.gen5)!; + expect(parsed.valid, isTrue, reason: 'both gen5 CRCs must check out'); + sample = sampleFromGen5Historical(parseGen5Historical(parsed.inner)); + }); + + test('maps ts/counter/hr straight through', () { + expect(sample, isNotNull); + expect(sample!.tsEpoch, 1780916150); + expect(sample!.counter, 25443699); + expect(sample!.hr, 102); + }); + + test('maps RR intervals straight through (band-agnostic HRV kernel)', () { + expect(sample!.rrIntervalsMs, [602, 613]); + }); + + test('maps the gravity vector onto ax/ay/az (shared g-units)', () { + expect(sample!.ax, closeTo(-0.7252, 1e-3)); + expect(sample!.ay, closeTo(0.4944, 1e-3)); + expect(sample!.az, closeTo(0.4969, 1e-3)); + }); + + test( + 'does NOT populate skinTempRaw/spo2 — gen5-specific scale/absence', + () { + // See gen5_v18_decode's (now removed, folded into protocol) original + // caution and Gen5HistorySample's field docs: gen5's skin_temp is + // already °C-scaled (raw/100), a DIFFERENT transfer function from + // gen4's per-device affine ADC calibration that `skinTempRaw` feeds — + // reusing that field here would silently corrupt the skin-temp-z + // metric. gen5 v18 has no real dual-wavelength SpO2 at all. + expect(sample!.skinTempRaw, isNull); + expect(sample!.spo2RedRaw, isNull); + expect(sample!.spo2IrRaw, isNull); + }, + ); + }); + + group('sampleFromGen5Historical — non-Sample record kinds', () { + test('a null decode (unrecognised version/garbage) maps to null', () { + expect(sampleFromGen5Historical(null), isNull); + }); + + test('a v21 IMU deep buffer (no Sample equivalent) maps to null', () { + // Synthetic-but-shape-correct v21 buffer: countA/countB both 100 (the + // buffer's actual identity gate, per Gen5V21Decoder — hist_version is + // not trusted for this kind at all). + final inner = Uint8List(kGen5V21InnerLen); + inner[0] = 0x2F; + inner[1] = 21; + final view = inner.buffer.asByteData(); + view.setUint16(16, 100, Endian.little); // countA offset + view.setUint16(622, 100, Endian.little); // countB offset + final decoded = parseGen5Historical(inner); + expect(decoded, isA()); + expect(sampleFromGen5Historical(decoded), isNull); + }); + }); +} diff --git a/test/gen5_wiring_test.dart b/test/gen5_wiring_test.dart new file mode 100644 index 0000000..de80f29 --- /dev/null +++ b/test/gen5_wiring_test.dart @@ -0,0 +1,375 @@ +// Wiring the band generation actually reaches the wire, plus the two +// historical-ingest routing bugs that quietly threw a user's records away. +// +// What each group here is standing in for: +// - the plausibility-drop path used to write the record NOWHERE, so a record +// we merely mistrusted fared worse than one we could not parse at all — +// and the batch-ACK then let the band trim those bytes for good; +// - only v24/v12/v10 were routed to the R24 decoder, so every other version +// the decoder can read (v7/v9/v18/v25) was archived as undecodable — a real +// export carried ~50k readable v25 records filed that way; +// - the alarm bodies are generation-specific, and the gen4 forms are +// hardware-verified, so the gen5 additions must not disturb them. + +import 'dart:typed_data'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:openstrap_edge/ble/ble_engine.dart'; +import 'package:openstrap_edge/ble/ble_state.dart'; +import 'package:openstrap_edge/data/models.dart'; +import 'package:openstrap_edge/sync/sync_policy.dart'; +import 'package:openstrap_protocol/openstrap_protocol.dart'; + +int _wallNow() => DateTime.now().millisecondsSinceEpoch ~/ 1000; + +/// A gen4 historical record inner: `[0x2f][version][…][counter@3][ts@7]`. +/// Zero-filled elsewhere, which the v24 field map reads as a (0,0,0) accel — +/// finite, and v24/v12 are the trusted path so no plausibility gate applies. +Uint8List _gen4Inner({ + required int version, + required int ts, + required int counter, + int length = 89, + // Only v24/v12 are decoded verbatim; every other version is gated on + // physiological plausibility, so those need a real HR byte and a ~1 g accel + // vector (float32 LE at 36/40/44). + int? hrOffset, + int hr = 0, +}) { + final inner = Uint8List(length); + inner[0] = PacketType.historicalData; + inner[1] = version; + final view = ByteData.sublistView(inner); + view.setUint32(3, counter, Endian.little); + view.setUint32(7, ts, Endian.little); + if (hrOffset != null) { + inner[hrOffset] = hr; + view.setFloat32(36, 0.0, Endian.little); + view.setFloat32(40, 0.0, Endian.little); + view.setFloat32(44, 1.0, Endian.little); + } + return inner; +} + +/// v25 has its own layout: time + gravity only, gated on a gravity magnitude of +/// roughly 1 g at inner[69/71/73] (i16 / 16384). +Uint8List _v25Inner({required int ts, required int counter}) { + final inner = _gen4Inner(version: 25, ts: ts, counter: counter, length: 80); + final view = ByteData.sublistView(inner); + view.setInt16(69, 0, Endian.little); // gx = 0.0 + view.setInt16(71, 0, Endian.little); // gy = 0.0 + view.setInt16(73, 16384, Endian.little); // gz = 1.0 → |g| = 1.0 + return inner; +} + +class _Ingest { + final samples = []; + final archives = []; + late final BleEngine engine; + + _Ingest({BandProfile band = BandProfile.gen4}) { + engine = BleEngine( + onRecord: (sample, raw) async => samples.add(sample), + onState: (_) {}, + ); + engine.debugInstallFakeLink( + onWrite: (_) async => true, + band: band, + onArchive: (a) async => archives.add(a), + ); + } + + void feed(Uint8List inner) => + engine.debugIngestHistoricalFrame(Frame(inner, true, true)); +} + +/// Captures every outgoing frame and decodes it back to `[opcode, ...body]`. +class _Wire { + final frames = []; + final BandProfile band; + late final BleEngine engine; + + _Wire({required this.band}) { + engine = BleEngine(onRecord: (_, _) async {}, onState: (_) {}); + engine.debugInstallFakeLink( + onWrite: (f) async { + frames.add(f); + return true; + }, + band: band, + ); + } + + /// The inner of the last frame written, minus the packet-type and seq bytes: + /// `[opcode, ...body]`. Asserts the frame really was built for [band] — a + /// gen4-framed frame does not parse against the gen5 envelope at all, which + /// is precisely how high-frequency sync failed silently. + List get lastCommand { + final parsed = parseFrame(frames.last, profile: band); + expect(parsed, isNotNull, reason: 'frame must parse under $band'); + expect(parsed!.valid, isTrue, reason: 'header + payload CRCs must pass'); + return parsed.inner.sublist(2); + } + + /// [lastCommand] truncated to [length]: `buildFrame` pads the inner out to a + /// 4-byte boundary, so trailing zeros are envelope, not body. + List lastCommandOf(int length) => lastCommand.sublist(0, length); +} + +void main() { + TestWidgetsFlutterBinding.ensureInitialized(); + setUp(BleEngine.resetBandClaimForTest); + tearDown(BleEngine.resetBandClaimForTest); + + group('P0 — a plausibility-dropped record is archived, never discarded', () { + test('an implausibly-old record lands in the archive as gate_dropped', () { + final h = _Ingest(); + // 2001: decodable, but far below the plausible-epoch floor, so the + // RecordGate refuses it. + h.feed(_gen4Inner(version: 24, ts: 1000000000, counter: 7)); + + expect(h.samples, isEmpty, reason: 'the gate still refuses to bank it'); + expect(h.archives, hasLength(1), + reason: 'OLD BEHAVIOUR: a bare return — the bytes went nowhere at ' + 'all, and the batch-ACK then let the band trim them'); + expect(h.archives.single.reason, 'gate_dropped'); + expect(h.archives.single.counter, 7); + }); + + test('an admitted record is banked and NOT archived', () { + final h = _Ingest(); + h.feed(_gen4Inner(version: 24, ts: _wallNow() - 3600, counter: 8)); + + expect(h.samples, hasLength(1)); + expect(h.archives, isEmpty); + }); + + test('the no-progress trim gate still fires once drops archive', () { + // The knock-on: `hadDurableRows` counts banked RECORDS, not archives. + // Were it to count archives, a drop-only burst would always look + // "durable" and the band would be told it may trim flash we never + // decoded. + expect( + TrimAckPolicy.evaluate( + sessionCurrent: true, + burstDiscarded: false, + commitDurable: true, + hadDurableRows: false, + droppedThisBurst: 3, + ), + TrimAckVerdict.blockedNoDurableProgress, + ); + expect( + TrimAckPolicy.evaluate( + sessionCurrent: true, + burstDiscarded: false, + commitDurable: true, + hadDurableRows: true, + droppedThisBurst: 3, + ), + TrimAckVerdict.send, + reason: 'a mixed burst that banked something may still ACK', + ); + }); + + test('a gate-dropped record is neither durable progress nor offload progress', + () { + // The two tests above pass `hadDurableRows` in as a literal, so the thing + // that actually COMPUTES it has never been covered — and it has been + // patched twice. Both halves belong to the same rule: a record we dropped + // for implausibility is archived (so it is not lost) but must not look + // like progress, or the band gets told it may trim flash we never read + // and the wandered-RTC remedy never surfaces. + DrainController drain() => DrainController( + onRecord: (_, _) async {}, + onRecordsBatch: null, + onCommit: (_, _, _, {archives}) async {}, + onArchive: (_) async {}, + log: (_) {}, + ); + ArchiveRecord archive(String reason) => ArchiveRecord( + counter: 1, + hex: '2f18${reason.hashCode.toRadixString(16)}', + packetType: 0x2F, + capturedAt: 0, + reason: reason, + ); + + final dropped = drain()..onUndecodableRecord(archive('gate_dropped')); + expect(dropped.bufferedProgressArchives, 0); + expect(dropped.recordsThisOffload, 0, + reason: 'a drop-only burst must leave `banked` false at COMPLETE'); + + final undecodable = drain()..onUndecodableRecord(archive('undecodable_v22')); + expect(undecodable.bufferedProgressArchives, 1, + reason: 'a version we cannot read IS progress once it is set aside'); + expect(undecodable.recordsThisOffload, 1); + }); + }); + + group('P0 — every gen4 version the decoder can read is decoded', () { + test('v25 is still archived — it has no heart rate to bank', () { + // The other versions here decode into 1 Hz rows; v25 deliberately does + // not. It carries a timestamp and a gravity vector and nothing else, and + // the decoder reports hr 0 because the record has no HR field. hr is NOT + // NULL in decoded_onehz and 0 is the off-skin sentinel, so banking v25 + // would claim the band was off the wrist for every one of those seconds + // — while the real gravity makes the accel-coverage gate accept the + // window. Archiving keeps the bytes for a re-decode once hr is nullable. + final ts = _wallNow() - 7200; + final h = _Ingest(); + h.feed(_v25Inner(ts: ts, counter: 4242)); + + expect(h.samples, isEmpty, reason: 'no fabricated hr 0 row'); + expect(h.archives, hasLength(1), reason: 'bytes kept, nothing lost'); + expect(h.archives.single.reason, 'undecodable_rec_v25'); + expect(h.archives.single.counter, 4242); + }); + + test('v9 and v7 decode too', () { + final ts = _wallNow() - 600; + // HR offset moves per version; the rest of the field map is shared. + for (final entry in {7: 27, 9: 17}.entries) { + final h = _Ingest(); + h.feed(_gen4Inner( + version: entry.key, + ts: ts, + counter: 11, + hrOffset: entry.value, + hr: 61, + )); + expect(h.samples, hasLength(1), reason: 'v${entry.key} must decode'); + expect(h.samples.single!.hr, 61); + expect(h.archives, isEmpty, + reason: 'v${entry.key} must not be archived'); + } + }); + + test('a genuinely unknown version is still archived', () { + final h = _Ingest(); + h.feed(_gen4Inner(version: 99, ts: _wallNow() - 60, counter: 12)); + + expect(h.samples, isEmpty); + expect(h.archives.single.reason, 'undecodable_rec_v99'); + }); + + test('a gen5 v18 is not routed through the gen4 field map', () { + // Same version byte, completely different layout. The gen5 branch claims + // it first; these zeroed bytes are not a valid gen5 v18, so it archives + // rather than yielding a fabricated gen4-shaped sample. + final h = _Ingest(band: BandProfile.gen5); + h.feed(_gen4Inner(version: 18, ts: _wallNow() - 60, counter: 13)); + + expect(h.samples, isEmpty); + expect(h.archives.single.reason, 'undecodable_rec_v18'); + }); + }); + + group('P1 — alarm bodies are generation-correct', () { + test('gen4 forms are byte-identical (hardware-verified — do not change)', + () { + expect(AlarmPayloads.disableForBand(isGen5: false), [0x01]); + expect(AlarmPayloads.getPayloadForBand(isGen5: false), [0x01]); + expect(AlarmPayloads.disable, [0x01]); + // gen4 keeps alarm slot 0. + expect(AlarmPayloads.setPayloadForBand(DateTime.now(), isGen5: false)[1], + 0); + }); + + test('gen5 disable is revision 2 + an alarm id', () { + expect(AlarmPayloads.disableForBand(isGen5: true), [0x02, 0xFF]); + expect(AlarmPayloads.disableForBand(isGen5: true, id: 1), [0x02, 1]); + }); + + test('gen5 get-alarm is revision 4 + the alarm id', () { + expect(AlarmPayloads.getPayloadForBand(isGen5: true), [0x04, 1]); + expect(AlarmPayloads.getPayloadForBand(isGen5: true, id: 0), + [0x04, 0]); + // The default id is the slot the gen5 SET arms. + expect(AlarmPayloads.setPayloadForBand(DateTime.now(), isGen5: true)[1], + AlarmPayloads.gen5Slot); + }); + + test('the engine writes the gen4 alarm bodies unchanged', () async { + final w = _Wire(band: BandProfile.gen4); + await w.engine.disableAlarm(); + expect(w.lastCommandOf(2), [Cmd.disableAlarm, 0x01]); + await w.engine.getAlarm(); + expect(w.lastCommandOf(2), [Cmd.getAlarmTime, 0x01]); + }); + + test('the engine writes the gen5 alarm bodies', () async { + final w = _Wire(band: BandProfile.gen5); + await w.engine.disableAlarm(); + expect(w.lastCommandOf(3), [Cmd.disableAlarm, 0x02, 0xFF]); + await w.engine.getAlarm(); + expect(w.lastCommandOf(3), [Cmd.getAlarmTime, 0x04, 1]); + }); + }); + + group('P1 — band-specific opcodes and framing', () { + test('strap rename uses the opcodes each generation implements', () async { + final g4 = _Wire(band: BandProfile.gen4); + await g4.engine.setStrapName('band'); + expect(g4.lastCommand.first, Cmd.setAdvertisingNameHarvard); + await g4.engine.getStrapName(); + expect(g4.lastCommand.first, Cmd.getAdvertisingNameHarvard); + + final g5 = _Wire(band: BandProfile.gen5); + await g5.engine.setStrapName('band'); + expect(g5.lastCommand.first, Cmd.setCustomAdvertisingName, + reason: 'gen5 does not implement the gen4 advertising-name opcodes'); + await g5.engine.getStrapName(); + expect(g5.lastCommand.first, Cmd.getCustomAdvertisingName); + }); + + test('high-frequency sync is framed for the session band', () async { + final w = _Wire(band: BandProfile.gen5); + await w.engine.applyHighFreqWakeWindow( + enabled: true, + targetWake: DateTime.now().add(const Duration(hours: 1)), + ); + // The assertion that matters is inside lastCommand: a gen4-framed frame + // does not parse under the gen5 envelope, so the strap could never act + // on it — while the engine claimed the mode had engaged. + expect(w.lastCommand.first, Cmd.enterHighFreqSync); + }); + + test('the mode is not claimed when the write fails', () async { + final engine = BleEngine(onRecord: (_, _) async {}, onState: (_) {}); + engine.debugInstallFakeLink(onWrite: (_) async => false); + await engine.applyHighFreqWakeWindow( + enabled: true, + targetWake: DateTime.now().add(const Duration(hours: 1)), + ); + expect(engine.offloadSnapshot['high_freq_requested'], isFalse); + }); + + test('the gen5 clock commands carry the revision byte', () async { + final w = _Wire(band: BandProfile.gen5); + await w.engine.getClock(); + expect(w.lastCommandOf(2), [Cmd.getClockGen5, 0x01]); + }); + }); + + group('P0 — an unset strap RTC is visible to the clock policy', () { + test('an implausibly-low clock_epoch reaches shouldSetClock', () { + final logs = []; + final engine = BleEngine( + onRecord: (_, _) async {}, + onState: (_) {}, + log: logs.add, + ); + // A strap whose RTC was never set. This is the exact reading + // ClockPolicy.shouldSetClock was written for, and it used to be dropped + // by the decoder before the policy ever saw it. + engine.debugAbsorbDecoded(Decoded('cmd_response', {'clock_epoch': 100})); + + expect(ClockPolicy.shouldSetClock(100, _wallNow()), isTrue); + expect(engine.clockRef, isNull, + reason: 'a factory-epoch clock must not become the alarm ' + 'correlation — the drift would be decades'); + expect(logs.where((l) => l.contains('never set')), isNotEmpty); + }); + }); +} diff --git a/test/substrate_accel_absence_test.dart b/test/substrate_accel_absence_test.dart new file mode 100644 index 0000000..3559813 --- /dev/null +++ b/test/substrate_accel_absence_test.dart @@ -0,0 +1,290 @@ +// P0 REGRESSION — absent gravity must not be read back as perfect stillness. +// +// A gen5 v18 record whose gravity vector fails the magnitude gate keeps its +// HR/RR and reports the accel as ABSENT (ax/ay/az stay null) rather than +// discarding the whole second. But `decoded_onehz.ax/ay/az` are REAL NOT NULL, +// so the persistence layer writes +// `decoded.ax ?? 0` and the substrate loader reads `?? 0` back — turning +// "we did not measure this" into "the wrist was at exactly (0,0,0)". +// +// That is not an inert default. zAngle(0,0,0) is exactly 0.0 in Dart (atan2 +// (0,0) == 0.0, NOT NaN), so a run of absent seconds has a PERFECTLY CONSTANT +// z-angle, which is the van Hees immobility criterion satisfied maximally. +// Measured against the pinned analytics: 8 h of (0,0,0) yields 28 501 immobile +// seconds and `vanHeesSleepWindow.present == true` — a fabricated ~7.9 h night, +// fully staged, from data that does not exist. The strict gravity gate rejected +// EVERY v18 record we have looked at, so this is the ordinary case, not a +// corner. +// +// Two sentinels were tried and REJECTED, both verified against the pinned +// analytics rather than assumed: +// * NaN — fails OPEN. The rule tests "did the angle change by >= thr", and +// every comparison against NaN is false, so it never trips: NaN scores the +// SAME 28 501 immobile seconds as zeros. +// * dropping the seconds — `immobilityMask` is a pure index-wise angle rule +// with no timestamp/gap awareness (unlike `nap.dart`'s `stillAt`), so it +// simply joins across the hole. +// Hence the gate below: absent accel must not be allowed to ANCHOR a window. + +import 'dart:math' as math; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:openstrap_analytics/onehz.dart' as ana; +import 'package:openstrap_edge/compute/substrate.dart'; + +Substrate _sub({ + required int n, + required bool accelPresent, + int startSec = 1750000000, +}) { + final ts = []; + final hr = []; + final ax = []; + final ay = []; + final az = []; + for (var i = 0; i < n; i++) { + ts.add(startSec + i); + hr.add(58); + if (accelPresent) { + // A real, still-ish wrist: unit-magnitude gravity with a slight drift. + final rad = (i % 3) * 0.5 * math.pi / 180.0; + ax.add(math.sin(rad)); + ay.add(0.0); + az.add(math.cos(rad)); + } else { + // What the NOT NULL column forces for an abstaining decoder. + ax.add(0.0); + ay.add(0.0); + az.add(0.0); + } + } + return Substrate( + tsSec: ts, + hr: hr, + rrTsMs: const [], + rrMs: const [], + ax: ax, + ay: ay, + az: az, + spo2Red: List.filled(n, 0), + spo2Ir: List.filled(n, 0), + skinTemp: List.filled(n, 0), + skinContact: List.filled(n, 0), + ); +} + +void main() { + group('the hazard this guards against is real', () { + test( + 'zAngle(0,0,0) is 0.0, not NaN — so absent accel is maximally "still"', + () { + expect(ana.zAngle(0.0, 0.0, 0.0), 0.0); + }, + ); + + test( + '8 h of absent accel scores as an almost entirely immobile night', + () { + const n = 8 * 3600; + final s = _sub(n: n, accelPresent: false); + final m = ana.vanHeesSleepWindow(s.accelSamples()); + final win = m.value!; + final immobile = win.immobile.where((b) => b).length; + expect( + immobile, + greaterThan((n * 0.95).round()), + reason: 'this is why the gate exists — missing data reads as sleep', + ); + expect(m.present, isTrue); + }, + ); + + test( + 'a NaN sentinel would NOT have fixed it — comparisons against NaN are ' + 'false, so the "angle changed" test never trips', + () { + const n = 8 * 3600; + final base = 1750000000000.0; + final nan = [ + for (var i = 0; i < n; i++) + ana.AccelSample( + base + i * 1000, double.nan, double.nan, double.nan), + ]; + final immobile = + ana.vanHeesSleepWindow(nan).value!.immobile.where((b) => b).length; + expect( + immobile, + greaterThan((n * 0.95).round()), + reason: 'NaN fails OPEN here; documented so nobody "fixes" it that way', + ); + }, + ); + }); + + group('Substrate.accelPresentAt / accelPresentFraction', () { + test('exact (0,0,0) is absent; a real vector is present', () { + final absent = _sub(n: 10, accelPresent: false); + final present = _sub(n: 10, accelPresent: true); + expect(absent.accelPresentAt(0), isFalse); + expect(present.accelPresentAt(0), isTrue); + expect(absent.accelPresentFraction(0, 10), 0.0); + expect(present.accelPresentFraction(0, 10), 1.0); + }); + + test('an empty range reports 0 — no evidence, not "all present"', () { + final s = _sub(n: 10, accelPresent: true); + expect(s.accelPresentFraction(5, 5), 0.0); + expect(Substrate.empty.accelPresentFraction(0, 100), 0.0); + }); + + test('a mixed window reports the real fraction', () { + final s = _sub(n: 100, accelPresent: true); + for (var i = 0; i < 40; i++) { + s.ax[i] = 0.0; + s.ay[i] = 0.0; + s.az[i] = 0.0; + } + expect(s.accelPresentFraction(0, 100), closeTo(0.60, 1e-9)); + }); + + test( + 'the van Hees coverage floor rejects an all-absent night and accepts a ' + 'fully-measured one', + () { + final absent = _sub(n: 8 * 3600, accelPresent: false); + final present = _sub(n: 8 * 3600, accelPresent: true); + expect( + absent.accelPresentFraction(0, absent.length), + lessThan(kMinAccelCoverageForVanHees), + ); + expect( + present.accelPresentFraction(0, present.length), + greaterThanOrEqualTo(kMinAccelCoverageForVanHees), + ); + }, + ); + }); + + // CodeRabbit, correctly: everything above tests the PREDICATE + // (`accelPresentFraction`) and never executes the gate it feeds. That made + // this a vacuous guard for a P0 -- the coverage floor could have been deleted + // and every assertion here would still pass. These drive `calendarDays`, the + // real entry point, and assert on the SLEEP RESULT. + group('the coverage floor actually gates accel-led detection', () { + /// A full night of the given accel presence, with a plausible nocturnal HR + /// dip so the HR-led fallback has something to find. + Substrate night({required bool accelPresent, double presentFraction = 1.0}) { + // 22:00 -> 08:00 local, 1 Hz. + final start = _localMidnightOf(1750000000) + 22 * 3600; + const n = 10 * 3600; + final ts = []; + final hr = []; + final ax = []; + final ay = []; + final az = []; + for (var i = 0; i < n; i++) { + ts.add(start + i); + // Awake ~70, asleep ~50 between 23:00 and 07:00. + final inNight = i > 3600 && i < 9 * 3600; + hr.add(inNight ? 50 : 70); + final present = accelPresent && (i / n) < presentFraction; + if (present) { + if (inNight) { + // Asleep: a constant gravity vector — a genuine immobile block for + // van Hees to find. + ax.add(0.0); + ay.add(0.0); + az.add(1.0); + } else { + // Awake: a 10 deg/s sweep. It has to be a RAMP, not an + // alternation — the mask smooths the z-angle with a 5-second + // rolling MEDIAN, which erases a 1 Hz square wave entirely and + // would make the whole record read immobile. + final deg = (i % 9) * 10.0; + final rad = deg * math.pi / 180.0; + ax.add(math.cos(rad)); + ay.add(0.0); + az.add(math.sin(rad)); + } + } else { + ax.add(0.0); + ay.add(0.0); + az.add(0.0); + } + } + return Substrate( + tsSec: ts, + hr: hr, + rrTsMs: const [], + rrMs: const [], + ax: ax, + ay: ay, + az: az, + spo2Red: List.filled(n, 0), + spo2Ir: List.filled(n, 0), + skinTemp: List.filled(n, 0), + skinContact: List.filled(n, 0), + ); + } + + test( + 'when the NIGHT\'s own records carry no gravity, no accel-led `auto` ' + 'sleep is produced — the fabricated window never reaches a PhysioDay', + () { + // The discriminating shape, and the realistic one: the evening has + // real accel, the night's records decoded without a gravity vector. + // Coverage lands below the floor, and the zeros would otherwise form a + // clean multi-hour "immobile" block for van Hees to anchor on. With + // the gate removed this test FAILS — verified by mutation. + final days = calendarDays(night(accelPresent: true, presentFraction: 0.15)); + for (final d in days) { + expect( + d.sleepSource, + isNot('auto'), + reason: 'accel-led detection ran on data that does not exist', + ); + } + }, + ); + + test( + 'a night with NO gravity at all is likewise never accel-led', + () { + // The pure gen5-lenient case: every record decoded without gravity. + // Kept as documentation of the headline scenario — note it is not + // independently discriminating, because a record that is immobile end + // to end is also rejected downstream. The test above is the one that + // pins the gate. + final days = calendarDays(night(accelPresent: false)); + for (final d in days) { + expect(d.sleepSource, isNot('auto')); + } + }, + ); + + test('a fully-measured night still detects accel-led sleep normally', () { + final days = calendarDays(night(accelPresent: true)); + expect( + days.any((d) => d.sleepSource == 'auto'), + isTrue, + reason: 'the gate must not suppress a night we CAN measure', + ); + }); + + test('the boundary: just below the floor gates, at/above it does not', () { + final below = calendarDays( + night(accelPresent: true, presentFraction: kMinAccelCoverageForVanHees - 0.1)); + expect(below.any((d) => d.sleepSource == 'auto'), isFalse); + + final atOrAbove = calendarDays( + night(accelPresent: true, presentFraction: kMinAccelCoverageForVanHees + 0.1)); + expect(atOrAbove.any((d) => d.sleepSource == 'auto'), isTrue); + }); + }); +} + +/// Local midnight for [epochSec], matching `substrate.dart`'s own day anchor. +int _localMidnightOf(int epochSec) { + final d = DateTime.fromMillisecondsSinceEpoch(epochSec * 1000); + return DateTime(d.year, d.month, d.day).millisecondsSinceEpoch ~/ 1000; +}