Skip to content

SystemNotificationEvent content renders as ordinary assistant text instead of distinct system-notification style #958

Description

@JoshuaRowePhantom

Summary

<system_notification> content from the Copilot SDK appears under an [assistant] label with the XML tags visible. The data model is correct end-to-end; the bug is entirely in the rendering layer and consists of two separate root causes that must both be present to produce the observed symptom.

This bug is CLI-only. The GUI WebView path correctly renders [system] because ChatHistoryItemViewModel uses item.Role.Value for the label. The CLI path ignores item.Role entirely.

Root Cause 1 — CLI OnTurnCompleted Always Labels Items "assistant"

File: Phantom.Workspaces.Agent.Cli/Program.cs, lines 355–376

private void OnTurnCompleted(object? sender, AgentChatHistoryItem item)
{
    var itemText = string.Concat(item.Contents.OfType<TextContent>().Select(static c => c.Text));
    ...
    if (!this.supportsInteractiveRendering)
    {
        Console.WriteLine($"assistant > {itemText}");  // ignores item.Role
        return;
    }
    this.FinalizeAssistantTurn();  // also always the assistant visual context
}

AgentChat.TurnCompleted fires for every AgentChatHistoryItem promoted to history — including {Role=System} items. OnTurnCompleted (subscribed at line 92) never inspects item.Role. A system-role item is rendered identically to an assistant message.

Fix: Branch on item.Role — for ChatRole.System, either render with a system > prefix, route to a separate display path, or suppress CLI output (system notifications are informational metadata, not assistant replies).

Root Cause 2 — StripSystemNotificationTags Exact-Match Failure

File: Phantom.Workspaces.Llm.Core/CopilotSdkStreamAdapter.cs, lines 272–285

private static string StripSystemNotificationTags(string content)
{
    const string openTag = "<system_notification>";
    const string closeTag = "</system_notification>";

    var trimmed = content.Trim();
    if (trimmed.StartsWith(openTag, StringComparison.Ordinal)
        && trimmed.EndsWith(closeTag, StringComparison.Ordinal))
    {
        return trimmed[openTag.Length..^closeTag.Length].Trim();
    }

    return content;   // raw XML returned if pattern doesn't match exactly
}

The SDK documentation states the notification content is "typically" wrapped in <system_notification> tags — not always. If the content has text outside the tags, uses different casing, or has no tags at all, StripSystemNotificationTags returns the content verbatim including the XML markup.

Fix: Use a regex or IndexOf-based approach to extract the inner content regardless of surrounding text, or accept that tag-less content is also valid and strip tags wherever they appear in the string.

Data Pipeline (verified correct)

The SystemNotificationEvent carries the correct ChatRole.System through every layer:

Step Location Role preserved
SystemNotificationEvent received CopilotSdkStreamAdapter.cs:168–178 ChatRole.System set
Routed to root channel CopilotSubAgentRouter.cs:83
Passed through middleware StreamingPersistenceMiddleware.cs
Coalesced to history item AgentChat.cs CoalesceSegmentAsync:1285–1292 AgentChatHistoryItem { Role=System }
CLI rendering Program.cs:369 ❌ Role ignored — labeled assistant >
GUI rendering ChatHistoryItemViewModel.cs:19 ✅ Shows [system] correctly

Affected Files

File Root Cause What to change
Phantom.Workspaces.Agent.Cli/Program.cs #1 Inspect item.Role in OnTurnCompleted; render system items differently
Phantom.Workspaces.Llm.Core/CopilotSdkStreamAdapter.cs #2 Robustify StripSystemNotificationTags to handle non-exact-match content

Expected Tests

Test Name Class What It Verifies
OnTurnCompleted_SystemRoleItem_DoesNotRenderAsAssistant CliRenderingTests (new) A history item with Role=System is not output with the assistant > prefix
OnTurnCompleted_SystemRoleItem_RendersWithSystemPrefix CliRenderingTests (new) A history item with Role=System renders with system > prefix (or is suppressed)
StripSystemNotificationTags_ContentWithSurroundingText_StillStripped CopilotSdkStreamAdapterTests Tags are stripped even when non-tag text precedes or follows them
StripSystemNotificationTags_ContentWithNoTags_ReturnsContentUnchanged CopilotSdkStreamAdapterTests Content with no tags is returned as-is without throwing
SystemNotificationEvent_AppearsInHistory_WithSystemRole CopilotSdkStreamAdapterTests End-to-end: SystemNotificationEvent produces AgentChatHistoryItem with Role=System

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdiagnosedRoot cause identified

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions