Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,19 @@ After init, add the `factory` label to an open issue and run a dry run below.
| `factory featuremap check [--manifest <path>] [--base <ref>]` | Validate the repository feature/test manifest and optionally report advisory drift for unchanged entries whose locations changed. |
| `factory intake notion <manifest>` | Normalize ready specs from a read-only Notion mount into GitHub lifecycle issues or exact-path fleet work. Honors `--dry-run`. |

`factory status` includes `inFlightDispatches`, grouped by issue with agent
names and the provider-claim state (`pending`, `verified`, or `degraded`). This
view is read from Factory's local in-flight registry, so it remains available
when GitHub lifecycle writeback is the degraded subsystem.

Dispatch lifecycle writes are claim-critical. Factory applies the
`factory:in-progress` label/state before the dispatch comment, confirms the
GitHub label by provider read-back, and retries either write three times. An
exhausted write is logged at error level as dead-lettered, recorded as a
degraded claim in the registry, and fails the dispatch instead of reporting a
clean dispatch with missing GitHub state. Durable lifecycle recovery repeats
the same label-and-comment claim without respawning acknowledged agents.

Global options work anywhere in the args: `--config <path>`, `--dry-run`,
`--backend <internal|relay>`, and `--agent-exit-timeout <ms>`. The internal
backend reuses a relay broker that's already running for your workspace, and
Expand Down
82 changes: 82 additions & 0 deletions src/cli/fleet.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2590,6 +2590,88 @@ describe('fleet CLI runtime', () => {
}
})

it('lists registry-backed in-flight issues, agents, and degraded claims in factory status', async () => {
const root = await mkdtemp(join(tmpdir(), 'fleet-cli-registry-status-'))
try {
const heartbeatPath = join(root, 'heartbeat.json')
const registryPath = join(root, 'registry.json')
const configPath = await writeConfig(root, {
loop: { heartbeatPath, registryPath, heartbeatStaleMs: 10_000 },
})
const output = buffer()
const factory = {
status: vi.fn(() => ({ inFlight: [], queued: [], counters: {} })),
} as unknown as Factory
await writeFile(registryPath, JSON.stringify({
pid: 4242,
heartbeatPath,
updatedAt: '2026-08-14T13:30:00.000Z',
updatedAtMs: Date.parse('2026-08-14T13:30:00.000Z'),
agents: [
{
name: 'ar-242-impl-factory',
role: 'implementer',
issue: { uuid: 'AgentWorkforce/factory#242', key: '242', path: '/github/factory/242.json' },
sessionRef: 'session-impl',
pids: [],
node: 'oslo-mini',
dispatchClaim: {
state: 'degraded',
write: 'GitHub dispatch comment',
attempts: 3,
maxAttempts: 3,
deadLettered: true,
error: 'GitHub comment write unavailable',
updatedAtMs: Date.parse('2026-08-14T13:29:00.000Z'),
},
},
{
name: 'ar-242-review',
role: 'reviewer',
issue: { uuid: 'AgentWorkforce/factory#242', key: '242', path: '/github/factory/242.json' },
pids: [],
dispatchClaim: {
state: 'degraded',
write: 'GitHub dispatch comment',
attempts: 3,
maxAttempts: 3,
deadLettered: true,
error: 'GitHub comment write unavailable',
updatedAtMs: Date.parse('2026-08-14T13:29:00.000Z'),
},
},
],
}))

const code = await runFleetCli(['status', '--config', configPath], {
fleet: new FakeFleetClient(),
mount: new FakeMountClient(),
createFactory: () => factory,
stdout: output,
stderr: buffer(),
})

expect(code).toBe(0)
expect(JSON.parse(output.text())).toMatchObject({
inFlightDispatches: [{
issue: { key: '242' },
agents: [
{ name: 'ar-242-impl-factory', role: 'implementer', sessionRef: 'session-impl', node: 'oslo-mini' },
{ name: 'ar-242-review', role: 'reviewer' },
],
claim: {
state: 'degraded',
write: 'GitHub dispatch comment',
attempts: 3,
deadLettered: true,
},
}],
})
} finally {
await rm(root, { recursive: true, force: true })
}
})

it('surfaces a stale registered workspace mirror in factory status', async () => {
const root = await mkdtemp(join(tmpdir(), 'fleet-cli-stale-status-'))
try {
Expand Down
79 changes: 74 additions & 5 deletions src/cli/fleet.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ import {
reapFactoryOrphansOnce,
reapFactoryEnvironmentsOnce,
readFactoryLoopHeartbeat,
readFactoryInFlightRegistry,
resolveFactoryStates,
stateResolutionFromIds,
standaloneBabysitterAgentName,
Expand All @@ -42,6 +43,8 @@ import {
type Capability,
type Factory,
type FactoryEventReporter,
type FactoryInFlightDispatchStatus,
type FactoryInFlightRegistry,
type FactoryConfig,
type IterationReport,
type FleetBackend,
Expand Down Expand Up @@ -733,7 +736,13 @@ async function runFactoryCommand(
return 0
}
if (command.action === 'status') {
writeJson(out, await factoryStatusWithMountHealth(factory, mount, config.loop.heartbeatPath, config.loop.heartbeatStaleMs))
writeJson(out, await factoryStatusWithMountHealth(
factory,
mount,
config.loop.heartbeatPath,
config.loop.registryPath,
config.loop.heartbeatStaleMs,
))
return 0
}
if (command.action === 'loop-status') {
Expand Down Expand Up @@ -764,7 +773,13 @@ async function runFactoryCommand(
const reports = await factory.runLoop({ dryRun: globals.dryRun })
writeJson(out, {
reports,
status: await factoryStatusWithMountHealth(factory, mount, config.loop.heartbeatPath, config.loop.heartbeatStaleMs),
status: await factoryStatusWithMountHealth(
factory,
mount,
config.loop.heartbeatPath,
config.loop.registryPath,
config.loop.heartbeatStaleMs,
),
})
} finally {
removeSignalHandlers()
Expand Down Expand Up @@ -1051,6 +1066,7 @@ async function factoryStatusWithMountHealth(
factory: Factory,
mount: MountClient,
heartbeatPath: string,
registryPath: string,
heartbeatStaleMs: number,
): Promise<ReturnType<Factory['status']> & {
localMountDegraded?: boolean
Expand All @@ -1065,7 +1081,16 @@ async function factoryStatusWithMountHealth(
}
}> {
const status = factory.status()
const heartbeat = await readFactoryLoopHeartbeat(heartbeatPath)
const [heartbeat, registry] = await Promise.all([
readFactoryLoopHeartbeat(heartbeatPath),
readFactoryInFlightRegistry(registryPath),
])
const registryDispatches = registry?.heartbeatPath && registry.heartbeatPath !== heartbeatPath
? []
: inFlightDispatchesFromRegistry(registry)
const observableStatus = registryDispatches.length > 0
? { ...status, inFlightDispatches: registryDispatches }
: status
const liveness = checkFactoryLoopLiveness(heartbeat, { staleMs: heartbeatStaleMs })
const eventListener = liveness.ok
? heartbeat?.eventListener ?? {
Expand All @@ -1077,9 +1102,9 @@ async function factoryStatusWithMountHealth(
reason: liveness.reason,
}
const health = mount.getLocalMountHealth?.()
if (!health) return { ...status, eventListener }
if (!health) return { ...observableStatus, eventListener }
return {
...status,
...observableStatus,
eventListener,
localMountDegraded: health.degraded,
...(health.reason ? { localMountDegradedReason: health.reason } : {}),
Expand All @@ -1093,6 +1118,49 @@ async function factoryStatusWithMountHealth(
}
}

function inFlightDispatchesFromRegistry(
registry: FactoryInFlightRegistry | undefined,
): FactoryInFlightDispatchStatus[] {
if (!registry) return []
const grouped = new Map<string, FactoryInFlightDispatchStatus>()
const claimPriority = { verified: 0, pending: 1, degraded: 2 } as const

for (const agent of registry.agents) {
if (!agent.issue) continue
const key = `${agent.issue.key}\u0000${agent.issue.uuid}\u0000${agent.issue.path}`
const claim = agent.dispatchClaim ?? {
state: 'pending' as const,
updatedAtMs: registry.updatedAtMs,
}
const existing = grouped.get(key)
const entry = existing ?? {
issue: { ...agent.issue },
agents: [],
claim: { ...claim },
}
if (claimPriority[claim.state] > claimPriority[entry.claim.state]) {
entry.claim = { ...claim }
}
if (!entry.agents.some((candidate) => candidate.name === agent.name)) {
entry.agents.push({
name: agent.name,
...(agent.role ? { role: agent.role } : {}),
...(agent.sessionRef ? { sessionRef: agent.sessionRef } : {}),
...(agent.invocationId ? { invocationId: agent.invocationId } : {}),
...(agent.node ? { node: agent.node } : {}),
})
}
grouped.set(key, entry)
}

return [...grouped.values()]
.map((entry) => ({
...entry,
agents: entry.agents.sort((left, right) => left.name.localeCompare(right.name)),
}))
.sort((left, right) => left.issue.key.localeCompare(right.issue.key))
}

function writeMountRefreshSummary(
refreshedStaleMounts: readonly RefreshedStaleMount[],
stderr: Pick<NodeJS.WriteStream, 'write'>,
Expand Down Expand Up @@ -1872,6 +1940,7 @@ async function issueProjectionStatus(
factory,
mount,
config.loop.heartbeatPath,
config.loop.registryPath,
config.loop.heartbeatStaleMs,
)
const githubConnection = mount.integrationConnections
Expand Down
2 changes: 2 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,8 @@ export type {
DispatchResult,
Factory,
FactoryEventPayload,
FactoryDispatchClaimStatus,
FactoryInFlightDispatchStatus,
FactoryInFlightRegistry,
FactoryInFlightRegistryAgent,
FactoryInFlightRegistryProcess,
Expand Down
3 changes: 2 additions & 1 deletion src/orchestrator/batch-tracker.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import type { AgentSpec, SpawnResult } from '../ports'
import type { DispatchResult, IssueRef, TriageDecision } from '../types'
import type { DispatchResult, FactoryDispatchClaimStatus, IssueRef, TriageDecision } from '../types'
import { githubRepositoriesMatch } from '../github/repo-identity'

export interface TrackedAgent {
Expand All @@ -17,6 +17,7 @@ export interface InFlightIssue {
agents: Map<string, TrackedAgent>
invocationIds: Set<string>
result?: DispatchResult
dispatchClaim?: FactoryDispatchClaimStatus

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: The restore() method rebuilds an InFlightIssue from scratch and drops dispatchClaim, even though the new interface now includes it. During crash recovery the in-flight record is rebuilt via inFlightRecordFromLifecycle(...)batch.restore(...) (factory.ts lines 2634, 3049, 3064, 3949, 13040). dispatchClaim is kept by inFlightRecordFromLifecycle but discarded by restore, so a verified/pending claim never survives durable resume. As a result #writeInFlightRegistry() copies record.dispatchClaim from the restored record only when it is present, so a clean dispatch claim set before a crash no longer appears in factory status after recovery — exactly the registry-kept-available-when-writeback-degraded behavior this PR promises. Copy dispatchClaim in restore() like inFlightRecordFromLifecycle does.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/orchestrator/batch-tracker.ts, line 20:

<comment>The `restore()` method rebuilds an `InFlightIssue` from scratch and drops `dispatchClaim`, even though the new interface now includes it. During crash recovery the in-flight record is rebuilt via `inFlightRecordFromLifecycle(...)` → `batch.restore(...)` (factory.ts lines 2634, 3049, 3064, 3949, 13040). `dispatchClaim` is kept by `inFlightRecordFromLifecycle` but discarded by `restore`, so a verified/pending claim never survives durable resume. As a result `#writeInFlightRegistry()` copies `record.dispatchClaim` from the restored record only when it is present, so a clean dispatch claim set before a crash no longer appears in `factory status` after recovery — exactly the registry-kept-available-when-writeback-degraded behavior this PR promises. Copy `dispatchClaim` in `restore()` like `inFlightRecordFromLifecycle` does.</comment>

<file context>
@@ -17,6 +17,7 @@ export interface InFlightIssue {
   agents: Map<string, TrackedAgent>
   invocationIds: Set<string>
   result?: DispatchResult
+  dispatchClaim?: FactoryDispatchClaimStatus
 }
 
</file context>

}

export interface QueuedIssue {
Expand Down
Loading