[failproofaid] Capture sessions from more than one location per agent CLI, and stop upgrades emptying the policy set

CHANGELOG.md

@@ -4,11 +4,11 @@
 
 ### Features
 - Add `failproofai flush` — deliver what is already spooled, now. The collector is unhurried on purpose (a batch is swept once it is older than two minutes, at most 64 per pass, on a 60-second cadence), which is right for a backlog and exactly wrong for somebody standing at a dashboard waiting to see their own events: from there "not delivered yet" and "not working" look identical. The command asks the daemon for a pass with no minimum age and no cap, and `--wait` blocks until the spool drains so a script can flush and then assert. It re-sends nothing — for history the collector already read past, that is still `backfill`. (#PR)
-
-### Features
 - Rename the cloud-policy vocabulary: a **generation** is now a **deployment**, and a policy **revision** is a **version**. Both were words the product used nowhere else — a customer reads "deployment 7" and "version 3" without a glossary. The rename goes through the wire format and the on-disk manifest, not just the labels, and lands with the matching AgentEye change: a server and a daemon that disagree on these names means the fleet stops reconciling, silently. (#PR)
+- Capture sessions from more than one location per agent CLI. Every source watched exactly the place its own installer puts it — `~/.claude/projects`, `~/.hermes/state.db` — which is right for one machine and wrong for every other arrangement: a second profile, a mounted team share, a container's home beside the host's, an agent an operator relocated. Those hold real sessions and nothing collected them. `failproofai harness add-path <harness> [<label>=]<path>` (plus `list` / `remove-path`) writes `[collector.sources.<harness>] extra_paths` in `config.toml`; `FAILPROOFAI_<HARNESS>_EXTRA_PATHS` overrides it for containers. Covers all twelve harnesses — thirteen sources, since `claude` carries its subagent transcripts, which share a root and would otherwise be silently dropped under an added path. Grammar, folder-name label fallback and `<label>-<agentId>` namespacing match AgentEye's collector (`--openclaw-extra-path`, `--hermes-extra-path`) rather than inventing a second answer to what an extra path is. Three things are structural rather than incidental: **each labelled path is its own task**, because `roots` is walked by one task with one `Params` and the label lives there — sharing would capture both under one id; each gets **its own cursor directory**, because the store writes its whole map atomically and two instances sharing one clobber each other's watermark and both re-read from zero after every restart; and each gets **its own health key**, or they overwrite each other's record and `root_present` alternates, destroying the "absent root versus idle source" distinction that file exists to draw — the exact failure Hermes hit with its per-profile databases. The label is applied in `SpoolWriter::push`, the one point every event from every source passes through (where `machine_id` and `user` are already stamped for the same reason), and **not** in either engine: a SQLite format is handed `params.agent_id` only as a fallback and derives the real id from the row (`devin::agent_id` returns `devin-<project>` and never consults the fallback for a session that has a working directory, which is all of them), so prefixing there would namespace only the sessions that failed to derive an id. A path overlapping one already captured is refused at startup rather than collected twice under two ids, and an unrecognised `[collector.sources.*]` table is reported instead of silently capturing nothing. A machine with nothing configured is unchanged, byte for byte, including its `config.toml`. Adding or removing a path needs no restart and no sudo: the daemon's collector manager already re-reads `config.toml` on an interval and cycles the collector whenever the resolved `CollectorConfig` changes, and `sources` rides that by living inside the compared value — verified live (task count 21 → 23 within one interval, and the first transcript under the new path reached the server) and pinned by an e2e test that drives the real binary against a hand-edited config. (#PR)
 
 ### Fixes
+- Stop an upgrade from layout 1 silently emptying the user's policy selection. Layout 1 kept it at `~/.failproofai/policies-config.json` and layout 2 keeps it at `policies/local-policies/policies-config.json` — and BOTH were on the reset list, so moving between them discarded every builtin the user had enabled, every explicit `customPoliciesPaths` entry and every `policyParams` value. The reset runs unattended from `checkLayoutForCli` on the first command after an upgrade, and the machine still read as configured afterwards (`isConfigured()` is a union that sees the agent CLIs' untouched settings files) — so the wizard never re-asked, and hooks kept firing against a policy set that had quietly become the default one. That is the same silent enforcement gap `migrateConventionPolicies()` and `migrateHookActivity()` already exist to close, by a third route. The carry is deliberately narrow, and stays inside the standing wipe-and-re-setup decision: everything derived (cursors, spool, health, audit cache) still goes and is rebuilt, and what moves is only what a person typed and nothing regenerates. It is an ALLOWLIST rather than a copy, because layout 1's file also carried a `collector` block in camelCase and layout 2 moved those settings to `[collector]` in `config.toml` in snake_case — carrying it would put a key into the new file that nothing reads, looking preserved and behaving absent, which is the exact bug `fpai-collect`'s `Settings` documents. It runs in two phases (read before the deletions, write after) because its source AND its destination's parent are both on the reset list; writing first would have the reset delete the carry moments after it happened, which the note on `hookActivityDir()` records happening once already. A reset FROM the current layout no longer clears `local-policies/` at all — that is not a layout migration, and there it would discard a current, valid selection. An existing layout-2 config is never overwritten by the layout-1 file beside it. (#PR)
 - Make `failproofai config` refuse setup on an unsupported platform (Windows, today) instead of completing it unenforced. The wizard used to skip the daemon requirement and finish anyway, leaving the machine reading as configured while enforcing in-process with no fail-closed guarantee — now it prints why and exits 1 before drawing a single prompt, writing nothing. (#664)
 - Cut roughly a second of dead wait out of installing the daemon. `waitForDaemonRunning` slept a flat 750ms and then read the service status once, which was wrong in both directions: the socket is up in ~13ms and answers a real hook in ~125ms, so a healthy machine sat there with the answer already in hand — twice, because the repair path uninstalls and reinstalls — while a daemon that died at 100ms went unnoticed until the sleep was over. The window is now polled: leaving `running` fails immediately, and the wait ends as soon as the daemon accepts a connection, which is stronger evidence it did not die at startup than "still active after an arbitrary sleep". (#PR)
 - Let the setup wizard go back a step. Cancelling and going back were both `null`, so the only way to change an earlier answer was to abandon setup and start over; `←` on the harness step now re-asks the policy step with the previous selection still ticked. The policy step itself offers no `←` — the step before it is frequently not asked at all (a single scope choice is stated, not prompted), so it would sometimes go nowhere. (#PR)

CLAUDE.md

@@ -1152,13 +1152,43 @@ src/hooks/
                               all 11 CLIs, recoverable only by hand-editing
                               ~/.failproofai/policies-config.json
   manager.ts                 policies --install / --uninstall / list
+  harness-cli.ts             `failproofai harness add-path/remove-path/list` —
+                              edits [collector.sources.<harness>] extra_paths in
+                              config.toml and NOTHING else. The grammar, `~`
+                              expansion, overlap rejection and <label>-<agentId>
+                              namespacing all live in the daemon
+                              (crates/fpai-collect/src/extra_paths.rs); this side
+                              validates only the HARNESS NAME, because that is
+                              the one failure with no other detector — a typo'd
+                              table is valid TOML that captures nothing.
+                              HARNESS_KEYS here and in main.rs are two
+                              hand-maintained copies of one list; a test reads
+                              the Rust source to keep them identical
 src/index.ts                 Public API entry point → compiled to dist/index.js
 dist/index.js                CJS bundle (built by `bun run build`; shipped in npm pkg)
 dist/cli.mjs                 Bundled bin/failproofai.mjs (bun run build:cli)
 dist/worker.mjs              Bundled bin/failproofai-worker.mjs (bun run build:worker) —
                               plain Node can't resolve raw .ts specifiers, so the warm
                               worker needs this bundle just like the CLI does
 Cargo.toml                   Rust workspace root (resolver "3", shared [workspace.package])
+crates/fpai-collect/src/extra_paths.rs
+                              Per-source extra capture paths: `label=path`
+                              grammar, folder-name label fallback, and the
+                              rejections that each prevent a SILENT failure — a
+                              path overlapping a default root is collected twice
+                              under two agent ids, and two entries sharing a
+                              label share a cursor directory whose whole map is
+                              written atomically, so both re-read from zero after
+                              every restart. Each accepted path becomes its OWN
+                              task (file_source / sqlite_harness in main.rs) with
+                              its own cursor dir and health key — Hermes reached
+                              this shape first, via per-profile databases. The
+                              <label>-<agentId> prefix is applied in
+                              SpoolWriter::push, never in an engine: a SQLite
+                              format derives its real id from the row and takes
+                              params.agent_id only as a fallback, so prefixing
+                              there namespaces just the sessions that failed to
+                              derive one
 crates/fpai-ipc/              Wire protocol shared by the daemon and its tests: length-
                               prefixed JSON framing, protocolVersion envelope, peer-
                               credential checks (see crates/PROTOCOL.md)

__tests__/hooks/fp-reset.test.ts

@@ -313,3 +313,132 @@ describe("daemon flag self-heal", () => {
     spyStat.mockRestore();
   });
 });
+
+// ── the policy selection a person typed ──────────────────────────────────
+//
+// Layout 1 kept the user policy config at `~/.failproofai/policies-config.json`;
+// layout 2 keeps it at `policies/local-policies/policies-config.json`. BOTH were
+// on the reset list, so an upgrade silently emptied `enabledPolicies` — and the
+// machine still read as configured afterwards (`isConfigured()` is a union that
+// sees the agent CLIs' untouched settings files), so hooks kept firing against a
+// policy set that had quietly become the default one.
+describe("resetHome carries the layout-1 policy selection", () => {
+  const layoutOneConfig = {
+    enabledPolicies: ["block-sudo", "block-env-files", "require-tests-before-stop"],
+    customPoliciesPaths: ["/home/u/team/policies.mjs"],
+    disabledCustomPolicies: ["team/noisy-rule"],
+    policyParams: { "block-sudo": { allowlist: ["sudo -n true"] } },
+    llm: { baseUrl: "https://llm.internal/v1", model: "gpt-4o-mini" },
+    // Layout 1 also kept collector settings HERE, in camelCase. Layout 2 moved
+    // them to config.toml in snake_case, where `fpai-collect`'s `Settings`
+    // deserializes them — so carrying this block forward would put a key into
+    // the new file that nothing reads, looking preserved and behaving absent.
+    collector: { hooksVerbosity: "all", machineId: "m-legacy" },
+  };
+
+  function seedLayoutOnePolicyConfig() {
+    writeFileSync(legacy.policyConfig(), JSON.stringify(layoutOneConfig, null, 2));
+  }
+
+  it("carries the fields layout 2 still means", () => {
+    seedLayoutOne();
+    seedLayoutOnePolicyConfig();
+
+    const out = resetHome(1);
+
+    const carried = JSON.parse(
+      readFileSync(resolve(localPoliciesDir(), "policies-config.json"), "utf8"),
+    );
+    expect(carried.enabledPolicies).toEqual([
+      "block-sudo",
+      "block-env-files",
+      "require-tests-before-stop",
+    ]);
+    expect(carried.customPoliciesPaths).toEqual(["/home/u/team/policies.mjs"]);
+    expect(carried.disabledCustomPolicies).toEqual(["team/noisy-rule"]);
+    expect(carried.policyParams).toEqual({ "block-sudo": { allowlist: ["sudo -n true"] } });
+    expect(carried.llm).toEqual({ baseUrl: "https://llm.internal/v1", model: "gpt-4o-mini" });
+    expect(out.policyConfig).toContain("enabledPolicies");
+  });
+
+  // The one exclusion, and the reason the carry is an allowlist rather than a
+  // copy. A camelCase `collector` block in the new file reads as a preserved
+  // setting and does nothing.
+  it("does NOT carry the collector block, which moved to config.toml", () => {
+    seedLayoutOne();
+    seedLayoutOnePolicyConfig();
+
+    const out = resetHome(1);
+
+    const carried = JSON.parse(
+      readFileSync(resolve(localPoliciesDir(), "policies-config.json"), "utf8"),
+    );
+    expect(carried.collector).toBeUndefined();
+    expect(out.policyConfig).not.toContain("collector");
+  });
+
+  // A home already set up on layout 2 has a NEWER answer than the layout-1 file
+  // beside it, and a stale file winning would UNDO configuration rather than
+  // preserve it. The reset still CLEARS that layout-2 config on a layout
+  // migration (see "still clears the machine-owned children of policies/") —
+  // what must never happen is layout 1's values taking its place.
+  it("never lets the layout-1 file overwrite an existing layout-2 config", () => {
+    seedLayoutOne();
+    seedLayoutOnePolicyConfig();
+    mkdirSync(localPoliciesDir(), { recursive: true });
+    writeFileSync(
+      resolve(localPoliciesDir(), "policies-config.json"),
+      JSON.stringify({ enabledPolicies: ["current-choice"] }),
+    );
+
+    const out = resetHome(1);
+
+    expect(out.policyConfig).toEqual([]);
+    const path = resolve(localPoliciesDir(), "policies-config.json");
+    if (existsSync(path)) {
+      const after = JSON.parse(readFileSync(path, "utf8"));
+      expect(after.enabledPolicies).not.toContain("block-sudo");
+    }
+  });
+
+  it("is a no-op when there is no layout-1 config", () => {
+    seedLayoutOne();
+    // `seedLayoutOne()` writes one; this case is the home that never had it.
+    rmSync(legacy.policyConfig(), { force: true });
+    const out = resetHome(1);
+    expect(out.policyConfig).toEqual([]);
+    expect(existsSync(resolve(localPoliciesDir(), "policies-config.json"))).toBe(false);
+  });
+
+  // Unparseable is not worth aborting a reset over, and there is nothing to
+  // carry — which is exactly what happened before this function existed.
+  it("survives an unparseable layout-1 config without aborting the reset", () => {
+    seedLayoutOne();
+    writeFileSync(legacy.policyConfig(), "{ this is not json");
+    const out = resetHome(1);
+    expect(out.policyConfig).toEqual([]);
+    expect(readVersionFile()?.layout).toBe(LAYOUT_VERSION);
+  });
+
+  // A file carrying only a collector block has nothing to carry, and must not
+  // produce an empty layout-2 file that looks like a real one.
+  it("writes nothing when only excluded fields are present", () => {
+    seedLayoutOne();
+    writeFileSync(legacy.policyConfig(), JSON.stringify({ collector: { hooksVerbosity: "all" } }));
+    const out = resetHome(1);
+    expect(out.policyConfig).toEqual([]);
+    expect(existsSync(resolve(localPoliciesDir(), "policies-config.json"))).toBe(false);
+  });
+
+  // The whole point: after the reset, the carried file is what the loader reads.
+  it("leaves the carried config where the layout-2 reader looks", () => {
+    seedLayoutOne();
+    seedLayoutOnePolicyConfig();
+    resetHome(1);
+    // The layout-1 original is gone...
+    expect(existsSync(legacy.policyConfig())).toBe(false);
+    // ...and the layout-2 path, which `globalPolicyConfigFile()` resolves to,
+    // holds the selection.
+    expect(existsSync(resolve(localPoliciesDir(), "policies-config.json"))).toBe(true);
+  });
+});