Preview persisted image DataContent in agent-chat HTML via hosted image endpoint

[JoshuaRowePhantom]

Preview persisted image DataContent in the agent-chat HTML view via a hosted image endpoint

Note: This bug has been split into sub-items. The design below is the full original design; each sub-item contains the relevant portion.

Implementation Sub-Items

Complete in this order:

1. Persistence: read-by-index for image DataContent by (AgentSessionId, Sequence, ContentIndex) #1217 — Persistence: read-by-index for image DataContent by (AgentSessionId, Sequence, ContentIndex) (no dependencies)
2. Web host: GET /agent/persistence/{sessionId}/message/{sequence}/content/{contentIndex}/image endpoint #1218 — Web host: GET /agent/persistence/{sessionId}/message/{sequence}/content/{contentIndex}/image endpoint (depends on Persistence: read-by-index for image DataContent by (AgentSessionId, Sequence, ContentIndex) #1217)
3. Agent-chat HTML: render image DataContent as <img> pointing at hosted image endpoint #1219 — Agent-chat HTML: render image DataContent as <img> pointing at hosted image endpoint (depends on Web host: GET /agent/persistence/{sessionId}/message/{sequence}/content/{contentIndex}/image endpoint #1218)

Summary

Images pasted into the queue composer and images returned by chat providers are already persisted into MongoDB as inline base64 bytes on DataContent items inside serialized ChatMessage payloads. However, the HTML agent-chat view renders those items as a plain text media-type label (e.g. image/png) — the user never actually sees the image. This enhancement adds a small clickable image preview inside the chat HTML by (1) introducing a new HTTP image-streaming endpoint on the in-app web host that reads the bytes back from the persistence store, and (2) updating the HTML renderer to emit an <img> tag pointing at that endpoint (rather than inlining a giant base64 data URI into the HTML). The same design covers both user-pasted images and provider-generated images because both flow through identical AIContent / persistence / rendering paths.

Cross-references:

• WorkspacesWebHost missing /transport/connect mapping breaks reverse-HTTP hub registration (404 instead of 101) #1209 — recent web-host endpoint mapping work (/, /data/*, agent endpoints on WorkspacesWebHost and Phantom.Workspaces.Web.Server). The new image endpoint is registered at the same seam.
• Auto-scroll disabled when sub-agent panel is appended to agent chat WebView #1202 / Stray assistant word (e.g. 'court') leaks into parent transcript after sub-agent completion #1197 — recent agent-chat HTML pipeline work around RunningSubAgentsHtmlTransformer, ChatOutputHtmlRenderer, and chat-output-shell.html. The rendering change lives in the same pipeline.

Design Context / Current Behavior

Ingest (user paste). QueueComposerControl.axaml.cs:314-318 captures the pasted bitmap as bytes and calls vm.AppendImageAttachment(bytes, "image/png", w, h). QueueComposerViewModel.cs:164 wraps that into new DataContent(imageData, mediaType) and appends it to a List<AIContent> attached to the user's message.

Persistence (write). Phantom.Workspaces.Data.MongoDB/MongoDbAgentPersistenceStore.cs:~99 (inside StoreAsync) serializes the entire ChatMessage (including all AIContent items — text, DataContent, function-call/result, etc.) via JsonSerializer.Serialize(message, AIJsonUtilities.DefaultOptions) and stores it as a BsonDocument on MongoDbPersistedMessageDocument with { AgentSessionId, Sequence, Payload }:

var documents = newMessages.Select((message, index) => new MongoDbPersistedMessageDocument
{
    AgentSessionId = request.Agent.AgentSessionId,
    Sequence       = nextSequence + index,
    Payload        = BsonDocument.Parse(JsonSerializer.Serialize(message, AIJsonUtilities.DefaultOptions)),
}).ToArray();

Persistence (read). MongoDbAgentPersistenceStore.ReadMessagesAsync (around :165-182) selects by AgentSessionId, sorts by Sequence, and deserializes each payload back into a ChatMessage[]. This is the authoritative source of persisted image bytes.

Web host & routing. Phantom.Workspaces/Services/WorkspacesWebHost.cs:84-90 and Phantom.Workspaces.Web.Server/Program.cs:53-59 both compose the same route surface via extension methods: MapWebDataAccessEndpoints, MapAgentEndpoints, MapTransportReverseEndpoints, MapWorkspaceToolEndpoints, MapAgentPersistenceEndpoints (the last of which currently exposes POST /agent/persistence/store, .../restore, .../messages, .../sub-agent-links/add, .../sub-agent-links/read — see AgentPersistenceEndpointRouteBuilderExtensions.cs:106-132).

HTML rendering. The chat WebView loads embedded Phantom.Workspaces.Agent.Gui/Assets/chat-output-shell.html from AgentChatOutputControl.axaml.cs:189-199 and receives per-content HTML fragments through IChatOutputHtmlSink.UpdateContent(path, location, content) (AgentChatOutputControl.axaml.cs:128-130). The switch on AIContent type lives in Phantom.Workspaces.Agent.Gui/ViewModels/DocumentModels/ChatOutputHtmlRenderer.cs in RenderContent(...) around :439-508. Today the DataContent case (:497-500) only renders the media-type as a text label:

case DataContent data:
    return IsImageMediaType(data.MediaType)
        ? TextBlock(contentId, "chat-meta",
            string.IsNullOrWhiteSpace(data.MediaType) ? "image" : data.MediaType,
            SerializeContentJson(data))
        : TextBlock(contentId, "chat-monospace",
            string.IsNullOrWhiteSpace(data.MediaType) ? "[data]" : $"[{data.MediaType}]",
            SerializeContentJson(data));

Content ids are already stable within a rendered session: ChatOutputHtmlRenderer.MessageId(historyIndex) => "history-{i}" and ContentId(messageId, subIndex) => "{messageId}-{subIndex}" (:60, :68).

Answers to the design questions

Q1 — Identity: does DataContent uniquely identify itself?

No. Microsoft.Extensions.AI.DataContent exposes only Data (ReadOnlyMemory<byte>) and MediaType (string?); there is no Id, no Name, no annotation field on the base AIContent, and ChatMessage.MessageId is never set or persisted anywhere in the codebase (grep confirms zero assignments). Confirmed via ChatHistoryItemViewModel.cs:140-141 (reads Data, MediaType), CopilotSdkChatClient.cs:1043-1050 (same), and ChatOutputHtmlRenderer.cs:523 (uses a hash-like tuple "data:{MediaType}\^A{Data.Length}" purely for streaming dedup).

Proposed stable composite key (persistence-anchored): AgentSessionId + Sequence + ContentIndex, where:

• AgentSessionId is the session partition key on MongoDbPersistedMessageDocument,
• Sequence is the monotonically increasing per-session integer already assigned at write time (MongoDbPersistedMessageDocument.Sequence, long),
• ContentIndex is the position of the DataContent inside ChatMessage.Contents.

This tuple is stable under reloads because Mongo owns Sequence and content order within a single stored message is fixed by serialization. Optional hardening: append a short content fingerprint sha256(Data)[0..8] as a query-string parameter (?v=…) so the browser cache is invalidated if a message is ever edited to swap image bytes at the same index; the server MAY validate that the requested fingerprint matches the resolved bytes and 404 otherwise.

Q2 — Retrieval source and endpoint

Recommendation: read from IAgentPersistenceStore (Mongo) rather than AgentChatSessionCache / AgentChatHistoryCollection.

Rationale:

• AgentChatSessionCache (Phantom.Workspaces.Llm.Core/AgentChatSessionCache.cs:20-35, registered as a singleton at Phantom.Workspaces.Web.Server/Program.cs:38) only holds live AgentChat objects for currently-open sessions. After a reload, restart, or when opening a historical session from a different host process, the cache miss would mean image tags in the HTML break.
• The persistence store is the durable source of truth. ReadMessagesAsync(ReadMessagesRequest) already exists (MongoDbAgentPersistenceStore.cs:165-182) and returns ChatMessage[] sorted by Sequence, so ContentIndex on a given Sequence is well-defined.
• The persistence store is already reachable from web endpoints — see AgentPersistenceEndpointRouteBuilderExtensions.cs:110-113 (RequestServices.GetService<IAgentPersistenceStore>()).

Proposed endpoint — add to AgentPersistenceEndpointRouteBuilderExtensions so it is picked up by both WorkspacesWebHost and Phantom.Workspaces.Web.Server (both call MapAgentPersistenceEndpoints()):

GET /agent/persistence/{sessionId}/message/{sequence:long}/content/{contentIndex:int}/image

Response: raw image bytes, Content-Type set from DataContent.MediaType (fallback application/octet-stream), Cache-Control: private, max-age=…, plus Content-Length. 404 if the message/index does not exist or the content at that index is not a DataContent image. Reject on missing/whitespace sessionId.

Optimization note (non-blocking): a first cut can call the existing ReadMessagesAsync and index into the array; a follow-up may introduce ReadMessageBySequenceAsync(sessionId, sequence) on IAgentPersistenceStore to avoid deserializing the whole session for a single <img>.

Q3 — HTML chat view rendering

Change the DataContent image branch in ChatOutputHtmlRenderer.RenderContent (:497-500) so that when IsImageMediaType(data.MediaType) is true and rendering context includes the AgentSessionId + current message Sequence + contentIndex, it emits:

<a class="chat-image-link" href="{ENDPOINT}" target="_blank" rel="noopener"
   data-details-target="{contentId}">
  <img class="chat-image-preview"
       src="{ENDPOINT}"
       alt="{escaped media type}"
       loading="lazy" />
</a>

with CSS in chat-output-shell.html:

.chat-image-preview { max-width: 240px; max-height: 240px; object-fit: contain;
                     border: 1px solid var(--chat-border); border-radius: 4px; cursor: zoom-in; }

Click-to-full is handled by the existing anchor-click interceptor already exercised by AgentChatOutputControlTests.ChatOutputShellHtml_ContainsAnchorClickInterceptor (AgentChatOutputControlTests.cs:23), which routes anchor clicks through the WebView's URL-open path (raising UrlNavigationRequested). No new JavaScript is required for the MVP; a follow-up may add an in-shell lightbox overlay.

Explicitly avoided: inlining the base64 bytes into the HTML fragment. The current wire size for a single 4K-screenshot paste can exceed 5–10 MB; multiplying by every render/replace via IChatOutputHtmlSink.UpdateContent and by DOM parsing time in the WebView is prohibitive. The endpoint keeps HTML fragments small (a <img src="…"> URL) and streams the bytes once, cached by the WebView.

Threading identity into the renderer. RenderContent currently takes only contentId and the AIContent. ChatMessageHtmlModel.Render (ChatOutputHtmlModels.cs:195-210) already knows the message index; we thread AgentSessionId (already available from the owning session) and the message's Sequence (from AgentChatHistoryItem / the persistence write path — new field or lookup) through to RenderContent, and pass the loop subIndex as contentIndex. If Sequence is not yet available on the in-memory history item, add it to AgentChatHistoryItem at message-store time.

Q4 — Generated (assistant) images

Confirmed: ChatMessageHtmlModel.Render (ChatOutputHtmlModels.cs:201-210) uses Role only to pick styling flags (isDiagnostic, isHelp) — it does not filter by content type. Every AIContent on every role runs through the same ChatOutputHtmlRenderer.RenderContent switch, and MongoDbAgentPersistenceStore.StoreAsync serializes assistant ChatMessages the same way as user messages. Therefore provider-generated DataContent images in assistant messages are covered by the same identity scheme and the same endpoint. The only observable difference is the message Role label around the fragment; the image branch itself is role-agnostic.

Affected Files

File	Contribution
features/Phantom.Workspaces.Web.Server/AgentPersistenceEndpointRouteBuilderExtensions.cs	Register new GET /agent/persistence/{sessionId}/message/{sequence}/content/{contentIndex}/image endpoint.
features/Phantom.Workspaces.Llm.Core/IAgentPersistenceStore.cs (or sibling helper)	Optional ReadMessageBySequenceAsync for single-message reads; MVP can compose from existing ReadMessagesAsync.
features/Phantom.Workspaces.Data.MongoDB/MongoDbAgentPersistenceStore.cs	Optional impl of ReadMessageBySequenceAsync (BSON filter on AgentSessionId + Sequence).
features/Phantom.Workspaces.Agent.Gui/ViewModels/DocumentModels/ChatOutputHtmlRenderer.cs	Extend RenderContent signature to accept agentSessionId + messageSequence; change DataContent image branch (:497-500) to emit <a><img src="…"></a>.
features/Phantom.Workspaces.Agent.Gui/ViewModels/DocumentModels/ChatOutputHtmlModels.cs	Thread agentSessionId + message Sequence from ChatMessageHtmlModel.Render (:195-210) into RenderContent.
features/Phantom.Workspaces.Llm.Core/AgentChatHistoryItem.cs	Add long? Sequence { get; init; } so the in-memory item knows its persistence key.
features/Phantom.Workspaces.Agent.Gui/Assets/chat-output-shell.html	Add .chat-image-preview / .chat-image-link CSS (max-width/max-height 240px, cursor: zoom-in).
features/Phantom.Workspaces.Web.Server/Program.cs (existing wiring)	Endpoint is picked up automatically via app.MapAgentPersistenceEndpoints() (:59).
features/Phantom.Workspaces/Services/WorkspacesWebHost.cs	Endpoint is picked up automatically via application.MapAgentPersistenceEndpoints() chain (adjacent to :84-90 composition — extend the same call if not already invoked).

Design / Fix

Content-identity scheme

(AgentSessionId, Sequence, ContentIndex). AgentSessionId is the existing session key; Sequence is MongoDbPersistedMessageDocument.Sequence assigned at write; ContentIndex is the position of the DataContent within ChatMessage.Contents. Optional ?v={sha256(Data)[..16]} for cache-busting and integrity checks.

New HTTP image endpoint (sketch)

endpointRouteBuilder.MapGet(
    "/agent/persistence/{sessionId}/message/{sequence:long}/content/{contentIndex:int}/image",
    async (HttpContext http, string sessionId, long sequence, int contentIndex) =>
{
    if (string.IsNullOrWhiteSpace(sessionId)) return Results.BadRequest();
    var store = http.RequestServices.GetService<IAgentPersistenceStore>();
    if (store is null) return Results.StatusCode(StatusCodes.Status503ServiceUnavailable);

    // MVP: single-message read via existing ReadMessagesAsync; follow-up: ReadMessageBySequenceAsync.
    var messages = await store.ReadMessagesAsync(
        new ReadMessagesRequest { AgentSessionId = sessionId },
        http.RequestAborted).ConfigureAwait(false);

    if (sequence < 0 || sequence >= messages.Length) return Results.NotFound();
    var msg = messages[sequence];
    if (contentIndex < 0 || contentIndex >= msg.Contents.Count) return Results.NotFound();
    if (msg.Contents[contentIndex] is not DataContent data
        || string.IsNullOrEmpty(data.MediaType)
        || !data.MediaType.StartsWith("image/", StringComparison.OrdinalIgnoreCase))
    {
        return Results.NotFound();
    }

    http.Response.Headers.CacheControl = "private, max-age=3600";
    return Results.File(
        fileContents: data.Data.ToArray(),
        contentType: data.MediaType,
        fileDownloadName: null,
        enableRangeProcessing: false);
});

Notes:

• Content-Type is taken directly from DataContent.MediaType (already validated to start with image/).
• Registration inside AgentPersistenceEndpointRouteBuilderExtensions.MapAgentPersistenceEndpoints means both hosts (WorkspacesWebHost and Phantom.Workspaces.Web.Server) expose the route automatically — see WorkspacesWebHost missing /transport/connect mapping breaks reverse-HTTP hub registration (404 instead of 101) #1209.
• Read path uses IAgentPersistenceStore (not AgentChatSessionCache) so it survives reload and works for closed sessions.

HTML renderer change (sketch)

ChatOutputHtmlRenderer.RenderContent gains string agentSessionId, long messageSequence, int contentIndex (already implicit from the caller loop):

case DataContent data when IsImageMediaType(data.MediaType):
{
    var mediaType = string.IsNullOrWhiteSpace(data.MediaType) ? "image" : data.MediaType!;
    var url = $"/agent/persistence/{Uri.EscapeDataString(agentSessionId)}"
            + $"/message/{messageSequence}/content/{contentIndex}/image";
    var payload = SerializeContentJson(data);
    return $$"""
        <a class="chat-image-link" href="{{url}}" target="_blank" rel="noopener"
           data-details-target="{{contentId}}">
          <img class="chat-image-preview" src="{{url}}"
               alt="{{HtmlEncode(mediaType)}}" loading="lazy" />
        </a>
        """;
}
case DataContent data:
    return TextBlock(contentId, "chat-monospace",
        string.IsNullOrWhiteSpace(data.MediaType) ? "[data]" : $"[{data.MediaType}]",
        SerializeContentJson(data));

CSS added to chat-output-shell.html:

.chat-image-preview {
    max-width: 240px;
    max-height: 240px;
    object-fit: contain;
    border-radius: 4px;
    cursor: zoom-in;
}
.chat-image-link { display: inline-block; }

Base64-in-HTML avoidance. Do NOT emit src="data:image/png;base64,…" — pasted screenshots are routinely multi-megabyte, and IChatOutputHtmlSink.UpdateContent re-marshals the entire fragment string across the WebView boundary on every change. Using an HTTP URL keeps the HTML fragment small, lets the WebView cache, and defers the payload to a background GET.

Coverage for generated images. Because ChatMessageHtmlModel.Render (ChatOutputHtmlModels.cs:201-210) invokes RenderContent for every AIContent regardless of Role, and MongoDbAgentPersistenceStore.StoreAsync persists assistant messages the same way, provider-generated DataContent items land on this same branch with no additional code.

Expected Tests

Class names and naming style mirror existing suites (Subject_Scenario_ExpectedOutcome, per AgentChatOutputControlTests, RunningSubAgentsHtmlTransformerTests, WebAgentPersistenceStoreTests, AgentPersistenceEndpointTests).

Test Name	Class	What It Verifies
AgentImageEndpoint_WhenRequestedByMessageAndIndex_ReturnsImageBytesWithMediaType	AgentPersistenceEndpointTests	GET returns 200, body equals DataContent.Data, Content-Type equals DataContent.MediaType.
AgentImageEndpoint_WhenSequenceOutOfRange_Returns404	AgentPersistenceEndpointTests	Nonexistent sequence → 404.
AgentImageEndpoint_WhenContentIndexOutOfRange_Returns404	AgentPersistenceEndpointTests	Valid message but no content at index → 404.
AgentImageEndpoint_WhenContentIsNotImage_Returns404	AgentPersistenceEndpointTests	DataContent with non-image media type or TextContent at index → 404.
AgentImageEndpoint_WhenPersistenceStoreUnavailable_Returns503	AgentPersistenceEndpointTests	Matches existing StoreAsync_WhenServerReturns503_ThrowsHttpRequestException shape.
AgentImageEndpoint_WhenSessionIdMissing_Returns400	AgentPersistenceEndpointTests	Empty/whitespace sessionId route param → 400.
ReadMessagesAsync_WithImageDataContent_RoundTripsBytesAndMediaType	WebAgentPersistenceStoreTests	Extends existing ReadMessagesAsync_WithComplexMessageContents_RoundTrip with DataContent(image/png) payload.
MongoDbAgentPersistenceStore_ReadMessageBySequence_ReturnsSingleMessage	MongoDbAgentPersistenceStoreSlowTests	If the optional single-message read is added, verify it filters on AgentSessionId + Sequence.
ChatHtml_WhenMessageHasImageContent_EmitsImgTagPointingAtHostEndpoint	ChatOutputHtmlRendererTests (new) or AgentChatOutputControlTests	Rendered fragment contains <img class="chat-image-preview" src="/agent/persistence/{sessionId}/message/{seq}/content/{idx}/image".
ChatHtml_WhenMessageHasImageContent_WrapsImgInAnchorForFullSize	same	Fragment includes surrounding <a class="chat-image-link" href="{url}">.
ChatHtml_WhenMessageHasImageContent_DoesNotInlineBase64	same	Emitted fragment MUST NOT contain data:image/ or the base64 payload.
ChatHtml_WhenAssistantGeneratesImage_RendersPreview	same	With ChatRole.Assistant + DataContent(image/png), same <img> output is produced (role-agnostic).
ChatHtml_WhenDataContentIsNonImage_RendersMonospaceLabel	same	Non-image DataContent still uses the existing [mediaType] monospace branch.
ChatOutputShellHtml_HasImagePreviewMaxSizeCss	AgentChatOutputControlTests	Shell HTML resource contains .chat-image-preview { max-width: 240px; max-height: 240px; ... }.
AgentChatOutputControl_ImagePreviewClick_RaisesUrlNavigationRequested	AgentChatOutputControlTests	Click on the <a class="chat-image-link"> triggers the existing anchor interceptor, delivering the endpoint URL through UrlNavigationRequested (companion to existing AgentChatOutputControl_OpenUrlMessage_RaisesUrlNavigationRequested).