Agent workspace: personas, and the reason verification never ran

docs/agent-harness-progress.html

@@ -182,5 +182,119 @@ <h2>Residual risks</h2>
 <li><strong>Server <code>typecheck</code> excludes test files</strong> (<code>tsconfig.json</code> <code>exclude</code>), so a fake that drifts from an interface — e.g. a <code>Db</code> stub missing <code>updateModel</code> — is caught only at runtime. Two fakes needed manual updating this loop for exactly that reason.</li>
 </ul>
 
+<h2>Agent workspace — first slice (personas)</h2>
+<p>One file: <code>.build/agents/hyper/user.md</code>, global scope, injected at the top of every
+turn ahead of the skills manifest. It carries the things a user would otherwise retype — "I use
+Tailwind", "keep copy lowercase".</p>
+<p><strong>The design point is the asymmetry.</strong> A persona is injected as <em>trusted
+guidance</em>; the skills manifest, three feet away in the same prompt, is injected as
+<em>untrusted data</em>. That is defensible for exactly one reason: <strong>the agent cannot write
+a persona.</strong> <code>.build/agents/**</code> is refused by <code>fs_write</code>,
+<code>fs_batch_write</code> and <code>fs_delete</code> while staying open to <code>fs_read</code>.
+A skill file <em>is</em> agent-writable, which is why a prompt injection reaching one tool call
+could plant one and have it re-read with authority every turn afterwards — a one-shot compromise
+made durable. If that write ban is ever lifted, the trusted framing must go with it; the two facts
+move together and there are tests on both halves.</p>
+<table>
+<tr><th>Check</th><th>Evidence</th></tr>
+<tr><td>Persona reaches the prompt, ahead of skills</td><td>Browser probe: <code>personaInPrompt</code>, <code>personaBeforeSkills</code> both true; parity test pins the order on both client and server builders</td></tr>
+<tr><td>Write ban holds through the real executor stack</td><td>Browser probe: <code>fs_write</code> and <code>fs_delete</code> refused, <code>fs_read</code> allowed, file intact after the attempt</td></tr>
+<tr><td>Persona actually steers the model</td><td>Live run, <code>PERSONA_MARKER</code>: <strong>4/5</strong> files the agent wrote carried the required header line. The miss is <code>style.css</code>, where <code>//</code> is not a valid CSS comment — correct judgment, not disobedience.</td></tr>
+</table>
+
+<h2>The bug the persona run found: verification was never running</h2>
+<p>Seeding a persona was incidental to the real find. The new refusal log surfaced
+<code>exec: [exit 1]</code> followed by tsc's help text — and reproducing it outside the browser was
+decisive:</p>
+<ul>
+<li>The starter template shipped <strong>no <code>tsconfig.json</code></strong>. With no config and
+no file arguments, <code>tsc</code> does not typecheck at all: it prints its usage banner and exits
+1. <code>npx tsc --noEmit</code> is the headline command of the built-in <code>verify</code> skill,
+so <em>every turn that tried to verify burned a step on a command that could not succeed.</em></li>
+<li>It also shipped no <code>@types/react</code> / <code>@types/react-dom</code>, so even with a
+config a real error drowns under TS7016/TS7026 noise about React having no declarations.</li>
+<li><code>typescript: "latest"</code> had silently resolved to <strong>TypeScript 7.0.2</strong>, the
+native rewrite — a major version change under existing projects, in the one dependency this template
+left unpinned while carefully pinning everything else.</li>
+</ul>
+<p>Fixed on both duplicated template surfaces (the drift guard caught the Gleam side, as designed):
+a <code>tsconfig.json</code> with <code>types: ["vite/client"]</code> so side-effect CSS imports are
+not reported as TS2882, React type declarations, and <code>typescript</code> pinned to
+<code>^5.7.2</code>. <code>tsconfig.json</code> joins <code>UNDELETABLE</code> — deleting it does not
+break the app, it breaks the agent's ability to <em>check</em> the app, which is worse because the
+loop keeps reporting that it verified.</p>
+<p><strong>Measured before/after on the same live turn:</strong></p>
+<table>
+<tr><th></th><th>Steps</th><th>Provider calls</th><th>Wall</th><th>Trail says</th></tr>
+<tr><td>Before</td><td>8</td><td>7</td><td>100s</td><td>"Checked the code — found problems"</td></tr>
+<tr><td>After</td><td>5</td><td>4</td><td>58s</td><td>"Checked the code — no problems"</td></tr>
+</table>
+<p>Verification now genuinely runs, and the turn is shorter because the agent is no longer fighting a
+command that structurally could not work. Three regression tests in
+<code>templates.test.ts</code> pin the reason each of the three pieces exists, so a future cleanup
+does not read them as boilerplate.</p>
+
+<h2>Refusals are now visible</h2>
+<p>The trail showed <code>Tried to write files it may not change</code> and nothing else — no file,
+no reason — so a live run could only guess at what a guard had blocked. <code>executeTool</code> now
+ring-buffers failed tool results (failures only, reason truncated to 300 chars, capped at 50) and the
+live suite prints them. That is what turned an unexplained <code>exit 1</code> into the tsconfig
+finding above.</p>
+
+<h2>Review round on PR #38 — one blocker, and two guards that did not guard</h2>
+<p>A judge pass against the branch confirmed the write ban held (24 adversarial paths: no-slash,
+trailing slash, whitespace, <code>..</code> re-entry, double slash, backslash, leading <code>/</code>,
+leading <code>./</code>, sibling-prefix escape, and the batch form hiding the path among legitimate
+files — all refused, and the batch stayed all-or-nothing). It also found real problems.</p>
+<table>
+<tr><th>Finding</th><th>Resolution</th></tr>
+<tr><td><strong>Blocker: nothing could author a persona.</strong> The agent was banned from writing
+<code>.build/agents/</code> and no other writer existed — no editor, no settings surface. In
+production <code>readPersona()</code> always returned <code>''</code>. The whole slice was prompt
+plumbing for a file with no author.</td>
+<td>Built the editor: a Standing-instructions field in the settings panel, in <em>both</em> the BYOK
+and managed panels, with its own save (the managed panel has no Save button). It is deliberately the
+only writer, which is precisely what makes the trusted framing legitimate.</td></tr>
+<tr><td><strong>Two new guard tests passed vacuously.</strong> <code>indexOf</code> returns
+<code>-1</code> when the injection is deleted, and <code>String.slice(-1)</code> then yields one
+character — so the "stays trusted" test passed with the injection removed. It also matched lowercase
+<code>untrusted</code> only, so a capitalized re-framing slipped through. The ordering test could be
+defeated by hoisting <code>const p = body.persona</code> above the pushes.</td>
+<td>Both rewritten to call the two builders and inspect real output. Mutation-tested: deletion,
+capitalized re-framing, and hoist-and-reorder now each fail.</td></tr>
+<tr><td><strong>The server did not own the block it treats as trusted.</strong> The client sent a
+pre-framed system message; the server validated only <code>typeof === 'string'</code>. A modified
+client could post its own trusted framing, or megabytes of it.</td>
+<td>The client now sends RAW text. Both prompt builders author and cap the block themselves — a third
+duplicated surface, guarded like SHARED_RULES, including a test that the cap applies on both sides.</td></tr>
+<tr><td><strong>The tsconfig fix reached new projects only.</strong> Every project created before it
+kept the broken verify loop.</td>
+<td><code>ensureVerifiable()</code> backfills tsconfig.json and the React types on project load —
+additive, never repinning <code>typescript</code> (that would force a reinstall on open), and
+returning the same array when nothing is missing so an untouched project is not re-saved.</td></tr>
+<tr><td><strong>The Ollama JSON path bypassed every write guard</strong>, applying model-supplied
+paths straight to the project actor. It could not reach the workspace store, but it could put a file
+at the literal persona path into <code>project.files</code>, which publish ships verbatim.</td>
+<td>Routed through the same path policy, with refusals surfaced rather than silent. <code>.build</code>
+added to <code>IGNORED_SYNC_DIRS</code> so a container-side <code>.build/</code> cannot sync back into
+one path with two contents.</td></tr>
+<tr><td>The persona ban was case-sensitive while <code>DENIED_EXACT</code> twenty lines above carries
+a comment about that exact asymmetry having been an oversight once.</td>
+<td>Made case-insensitive. Match the strict half, not the lenient one.</td></tr>
+<tr><td>The failed-tool log survived project switches, so one project's failure text was readable
+while working in another.</td><td>Cleared on project open.</td></tr>
+</table>
+<p><strong>Found while fixing, and missed by the review:</strong> three call sites read
+<code>starterFiles[2].path</code> — "the third starter file", which meant <code>src/main.tsx</code>
+only by accident of ordering. Inserting <code>tsconfig.json</code> silently retargeted all three to
+<code>index.html</code>, changing which file the editor opens on. Replaced with a lookup by name.</p>
+<p>The editor was then verified in a real browser against nine conditions — visible on first run,
+enables on edit, disables after save, the agent reads exactly what was typed, it reaches the prompt
+framed as guidance, the agent still cannot overwrite it, it survives reload, and clearing the box
+removes the file rather than leaving an empty one. The reload check <strong>failed first time</strong>
+and found a real bug: the panel starts open, so no <code>SettingsOpened</code> message is ever sent on
+a fresh load and the load effect never fired — standing instructions would have looked lost on every
+reload.</p>
+
 </body>
 </html>

scripts/live-agent-suite.mjs

@@ -89,6 +89,18 @@ await page.evaluate(([key, model]) => {
   globalThis.__liveKey = key
   globalThis.__liveModel = model
 }, [KEY, MODEL])
+// LIVE_PERSONA="..." seeds the user's standing instructions before the run, so
+// the suite can check the persona actually steered the model rather than just
+// reaching the prompt. Seeded before the first turn; it applies to all of them.
+if (process.env.LIVE_PERSONA) {
+  await page.evaluate(async text => {
+    const agents = await import('/src/agents.ts')
+    const ws = await import('/src/workspace-store.ts')
+    await ws.writeWorkspaceFile(agents.PERSONA_PATH, text)
+  }, process.env.LIVE_PERSONA)
+  console.log(`▸ persona seeded (${process.env.LIVE_PERSONA.length} chars)`)
+}
+
 await bridge(`bridge.dispatchSettingsLoaded({ provider: 'openrouter', apiKey: globalThis.__liveKey, model: globalThis.__liveModel, job: 'standard' })`)
 await page.waitForSelector('.modalBackdrop', { state: 'detached', timeout: 8000 })
 const optOut = await page.$('text=Just describe it instead')
@@ -114,6 +126,12 @@ let shot = 0
 for (const turn of TURNS) {
   const callsBefore = providerCalls
   const errorsBefore = previewErrors.length
+  // Snapshot before the turn so persona adherence is measured against files the
+  // agent actually WROTE. Scoring every file in the project counts untouched
+  // starter files as violations and reports 4/13 for a turn that was 4/4.
+  const filesBefore = await page.evaluate(() =>
+    Object.fromEntries((globalThis.__buildProjectFiles ?? []).map(f => [f.path, f.content])),
+  )
   console.log(`── ${turn.label} ──`)
   console.log(`   "${turn.prompt.slice(0, 78)}${turn.prompt.length > 78 ? '…' : ''}"`)
   console.log(`   watching: ${turn.watch}`)
@@ -167,6 +185,13 @@ for (const turn of TURNS) {
     rows: [...document.querySelectorAll('.trailStep')].map(r => r.textContent.trim()),
     reply: [...document.querySelectorAll('.msg.assistant')].pop()?.textContent?.trim() ?? '',
     files: (globalThis.__buildProjectFiles ?? []).map(f => f.path),
+    refusals: (globalThis.__buildToolLog ?? []).map(r => `${r.name}: ${r.reason}`),
+    // Full first lines, so a persona rule about file headers is checkable.
+    firstLines: (globalThis.__buildProjectFiles ?? []).map(f => ({
+      path: f.path,
+      head: (f.content ?? '').split('\n')[0] ?? '',
+      full: f.content ?? '',
+    })),
   }))
   shot += 1
   await page.screenshot({ path: join(OUT, `suite-${shot}-${turn.label.replace(/\s+/g, '-')}.png`) })
@@ -185,6 +210,9 @@ for (const turn of TURNS) {
     previewErrors: previewErrors.length - errorsBefore,
     batched: info.rows.some(r => /Wrote \d+ files/.test(r)),
     summary: info.summary,
+    firstLines: info.firstLines,
+    refusals: info.refusals,
+    written: info.firstLines.filter(f => filesBefore[f.path] === undefined || filesBefore[f.path] !== f.full),
     rows: info.rows,
     reply: info.reply,
     fileCount: info.files.length,
@@ -202,6 +230,26 @@ for (const r of results) {
 }
 const verifiedCount = results.filter(r => r.verified).length
 console.log(`\nS1 verification rate: ${verifiedCount}/${results.length} turns ran a check`)
+
+// PERSONA_MARKER="// crafted for tom" asserts the seeded persona actually
+// steered the output. Reaching the prompt is not the same as being obeyed, and
+// only the second one is the feature.
+if (process.env.PERSONA_MARKER) {
+  const marker = process.env.PERSONA_MARKER
+  const last = results[results.length - 1]
+  const src = (last?.written ?? []).filter(f => /\.(tsx?|jsx?|css)$/.test(f.path))
+  const hit = src.filter(f => f.head.includes(marker))
+  console.log(
+    `persona adherence: ${hit.length}/${src.length} files the agent WROTE start with ${JSON.stringify(marker)}`,
+  )
+  for (const f of src) console.log(`   ${f.head.includes(marker) ? '\u2713' : '\u2717'} ${f.path}`)
+}
+
+const refusals = results.flatMap(r => r.refusals ?? [])
+if (refusals.length) {
+  console.log(`\nrefusals (${refusals.length}) — what the guards actually blocked`)
+  for (const r of refusals) console.log(`   \u00b7 ${r}`)
+}
 console.log(`final project: ${results.at(-1)?.fileCount} files`)
 
 for (const r of results) {

server/src/app.ts

@@ -263,6 +263,7 @@ function isValidStepBody(body: unknown): body is StepRequestBody {
       m => (m?.role === 'user' || m?.role === 'assistant') && typeof m?.content === 'string',
     ) &&
     (c.skillsManifest === undefined || typeof c.skillsManifest === 'string') &&
+    (c.persona === undefined || typeof c.persona === 'string') &&
     Array.isArray(c.toolResults) &&
     c.toolResults.every(
       r => typeof r?.toolCallId === 'string' && typeof r?.content === 'string',

server/src/prompt.ts

@@ -348,12 +348,51 @@ export type StepRequestBody = {
    * DATA — a skill file is writable by the agent itself, so a persisted
    * injection must never carry authority. Bodies are pulled with fs_read. */
   skillsManifest?: string
+  /** The user's standing instructions, RAW TEXT — never a pre-framed system
+   * message. The server caps and frames it here, so a modified client cannot
+   * post its own trusted block. Trusted because `.build/agents/` is refused by
+   * every write tool, so only the account owner can author it. */
+  persona?: string
   /** True once anything has been read from the web in this turn. Carried by the
    * client across steps because the server holds no turn state; re-checked
    * server-side before any web_post actually sends. */
   webRead?: boolean
 }
 
+/**
+ * Cap on the persona. It rides in EVERY turn, so unlike a skill body — pulled
+ * only when relevant — its cost is unconditional.
+ */
+export const MAX_PERSONA_CHARS = 4000
+
+/**
+ * Frame the user's standing instructions as trusted guidance.
+ *
+ * Deliberately the opposite of `buildSkillsManifest`, and safe for exactly one
+ * reason: `.build/agents/` is refused by every write tool, so only the account
+ * owner can author this. See `src/agents.ts`.
+ *
+ * Duplicated from `src/agent.ts` on purpose, like SHARED_RULES: the server is
+ * the declared source of truth for managed mode and must not accept a
+ * pre-framed system message from a client it does not control. Guarded by
+ * `src/prompt-parity.test.ts`.
+ */
+export function buildPersonaPrompt(source: string): string {
+  const text = source.trim()
+  if (!text) return ''
+  const body =
+    text.length > MAX_PERSONA_CHARS
+      ? `${text.slice(0, MAX_PERSONA_CHARS)}\n\n[...truncated — the rest was over the ${MAX_PERSONA_CHARS}-character limit. Move standing detail into a skill instead.]`
+      : text
+  return [
+    'The person you are building for wrote the following standing instructions.',
+    'They apply to every turn. Follow them as you would the rules above; where they',
+    'conflict with a specific request in this turn, the request wins.',
+    '',
+    body,
+  ].join('\n')
+}
+
 export function buildToolModeMessages(
   body: StepRequestBody,
   opts: { webTools: boolean } = { webTools: false },
@@ -362,6 +401,12 @@ export function buildToolModeMessages(
     { role: 'system', content: buildToolModePrompt(opts) },
   ]
 
+  // Persona before skills: the user's own standing instructions outrank a saved
+  // note, and the ordering says so before either is read.
+  const persona = body.persona ? buildPersonaPrompt(body.persona) : ''
+  if (persona) {
+    messages.push({ role: 'system', content: persona })
+  }
   if (body.skillsManifest) {
     messages.push({ role: 'system', content: body.skillsManifest })
   }