docs: Ask AI chat grounded in the docs vector store by ouiliame · Pull Request #5172 · simstudioai/sim

ouiliame · 2026-06-22T20:46:02Z

Summary

Adds an Ask AI chat to the docs site so readers can ask questions about Sim in natural language and get answers grounded in the documentation.

Floating launcher → chat panel (components/ai/ask-ai.tsx) built on the Vercel AI SDK's useChat. Assistant replies render as markdown via streamdown (the same AI-streaming markdown renderer the main app's chat uses), with source chips linking back to cited pages.
Streaming API route (app/api/chat/route.ts) using streamText with the OpenAI provider. A searchDocs tool runs a vector search over the existing docs_embeddings store and returns source links, so the model answers from real docs rather than memory.
Reuses the OPENAI_API_KEY already in the environment (same key the docs search uses for embeddings). Model defaults to gpt-5.4-mini, overridable via OPENAI_CHAT_MODEL.

Abuse hardening

This endpoint proxies a paid LLM, so an unauthenticated public route is a target for scripted "free inference". Shipped in this PR (cost caps per request):

Max messages, max input size (413), max output tokens, reduced tool-step limit
Lenient same-origin check (rejects obvious cross-origin; DOCS_ALLOWED_ORIGINS to extend)

Infra-side follow-ups (dashboard/provisioning, not code) — do before public launch:

OpenAI hard spend cap + alert (backstops worst case regardless of code)
Durable per-IP rate limit (Upstash/Vercel KV — the per-request caps bound cost but not volume)
Vercel Firewall / BotID (or Turnstile) to block headless traffic at the edge

Notes

The LLM-text plumbing (llms.txt, llms-full.txt, .md/.mdx routes, Accept negotiation) already existed — this PR adds only the Ask AI chat.
Embeddings stay on OpenAI (text-embedding-3-small); only the chat completion uses the chat model. Branch is off staging; bun run build and type-check pass for the docs app.

🤖 Generated with Claude Code

vercel · 2026-06-22T20:46:08Z

@ouiliame is attempting to deploy a commit to the Sim Team on Vercel.

A member of the Team first needs to authorize it.

vercel · 2026-06-22T20:46:51Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	Jun 22, 2026 9:14pm

cursor · 2026-06-22T21:10:13Z

PR Summary

Medium Risk
Introduces an unauthenticated public endpoint that calls a paid LLM and the embeddings DB; mitigations exist but in-memory rate limiting is weak on serverless and Origin checks are not a strong security boundary.

Overview
Adds an Ask AI experience to the docs site: a floating launcher opens a chat panel on every localized docs page, wired through the Vercel AI SDK (useChat + streaming markdown via streamdown) and source chips from searchDocs tool results.

A new POST /api/chat route streams OpenAI completions (default gpt-5.4-mini) with a server-only searchDocs tool that retrieves chunks from docs_embeddings using the same hybrid keyword + English vector + RRF approach as site search, scoped by the reader’s locale. Client history is sanitized (text-only user/assistant) so grounding cannot be faked from the browser.

The route includes abuse controls for this public, paid-LLM proxy: per-request size/token/step limits, lenient Origin allowlisting (DOCS_ALLOWED_ORIGINS), and a best-effort in-memory per-IP rate limit (20/min). Docs app dependencies add ai, @ai-sdk/*, streamdown, and zod.

^{Reviewed by Cursor Bugbot for commit d5610a4. Bugbot is set up for automated code reviews on this repo. Configure here.}

greptile-apps · 2026-06-22T21:15:30Z

Greptile Summary

Adds a floating "Ask AI" chat panel to the docs site, grounded in the existing docs_embeddings vector store via a new /api/chat streaming route built on AI SDK v5 streamText and useChat. The implementation includes thoughtful per-request abuse hardening (message count cap, user-input char limit, total payload backstop, origin filter) with acknowledged infra-level follow-ups before public launch.

app/api/chat/route.ts: New streaming POST route — validates and size-caps incoming messages, runs a pgvector similarity search via the searchDocs tool, and streams the answer back with toUIMessageStreamResponse.
components/ai/ask-ai.tsx: Client-side floating launcher + chat panel using DefaultChatTransport / useChat; correctly splits instant-scroll-on-open from smooth-scroll-on-new-messages; source chips render with rel="noopener noreferrer".
package.json: Adds ai v5, @ai-sdk/openai, @ai-sdk/react, streamdown, and zod to the docs app.

Confidence Score: 5/5

Safe to merge; the two new files are well-scoped to the docs app with no impact on the main Sim application.

The route and component are carefully written with structural validation, size caps, and correct AI SDK v5 patterns. Both findings are non-blocking style/configuration nits with no effect on the happy path.

No files require special attention, though operators configuring DOCS_ALLOWED_ORIGINS should be aware the allowlist expects hostnames rather than full origin strings.

Important Files Changed

Filename	Overview
apps/docs/app/api/chat/route.ts	New streaming chat API route. Solid abuse-hardening (message caps, size limits, origin check). Minor: DOCS_ALLOWED_ORIGINS allowlist compares hostnames but env var name implies full origins — silent misconfiguration footgun.
apps/docs/components/ai/ask-ai.tsx	New floating chat component using AI SDK v5 useChat with DefaultChatTransport. Correctly splits instant-scroll (open) from smooth-scroll (messages) effects. Source chips include rel="noopener noreferrer". Minor: smooth-scroll effect fires even when panel is closed.
apps/docs/app/[lang]/layout.tsx	Minimal addition: mounts the AskAI component inside the existing RootProvider, correctly passing the active locale.
apps/docs/package.json	Adds @ai-sdk/openai, @ai-sdk/react, ai v5, streamdown, and zod dependencies to the docs app.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant User
    participant AskAI as AskAI Component
    participant Route as /api/chat
    participant OpenAI as OpenAI
    participant DB as docs_embeddings

    User->>AskAI: Types question, presses Enter
    AskAI->>Route: "POST /api/chat {messages, locale}"
    Route->>Route: isAllowedOrigin + validate + size caps
    Route->>OpenAI: streamText with searchDocs tool
    OpenAI-->>Route: Tool call: searchDocs(query)
    Route->>OpenAI: generateSearchEmbedding(query)
    OpenAI-->>Route: embedding vector
    Route->>DB: "SELECT ORDER BY embedding <=> vector LIMIT 24"
    DB-->>Route: top 24 chunks
    Route->>Route: Filter by locale → top 6 chunks
    Route-->>OpenAI: tool result (title, url, content)
    OpenAI-->>Route: streaming answer text
    Route-->>AskAI: UIMessageStream
    AskAI-->>User: Streamed answer with cited doc links

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant User
    participant AskAI as AskAI Component
    participant Route as /api/chat
    participant OpenAI as OpenAI
    participant DB as docs_embeddings

    User->>AskAI: Types question, presses Enter
    AskAI->>Route: "POST /api/chat {messages, locale}"
    Route->>Route: isAllowedOrigin + validate + size caps
    Route->>OpenAI: streamText with searchDocs tool
    OpenAI-->>Route: Tool call: searchDocs(query)
    Route->>OpenAI: generateSearchEmbedding(query)
    OpenAI-->>Route: embedding vector
    Route->>DB: "SELECT ORDER BY embedding <=> vector LIMIT 24"
    DB-->>Route: top 24 chunks
    Route->>Route: Filter by locale → top 6 chunks
    Route-->>OpenAI: tool result (title, url, content)
    OpenAI-->>Route: streaming answer text
    Route-->>AskAI: UIMessageStream
    AskAI-->>User: Streamed answer with cited doc links

_{Reviews (3): Last reviewed commit: "docs: validate message shape on Ask AI r..." | Re-trigger Greptile}

waleedlatif1 · 2026-06-22T21:21:49Z

@ouiliame run the review loop and fix the comments and re-run @greptile and @cursor review until 5/5

ouiliame · 2026-06-22T22:32:21Z

@greptile review
@cursor review

ouiliame · 2026-06-22T23:33:32Z

@greptile review
@cursor review

gitguardian · 2026-06-24T20:45:53Z

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
While these secrets were previously flagged, we no longer have a reference to the
specific commits where they were detected. Once a secret has been leaked into a git
repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.

^{_{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}}

ouiliame · 2026-06-25T18:41:01Z

@cursor review

ouiliame · 2026-06-25T18:43:29Z

@cursor review

ouiliame · 2026-06-25T19:07:18Z

@cursor review

ouiliame · 2026-06-25T19:12:35Z

@cursor review

ouiliame · 2026-06-25T19:15:52Z

@cursor review

ouiliame · 2026-06-25T19:20:05Z

@cursor review

ouiliame · 2026-06-25T19:41:40Z

@cursor review

ouiliame · 2026-06-25T19:45:03Z

@cursor review

ouiliame · 2026-06-25T22:48:41Z

@cursor review

Adds an Ask AI chat to the docs site. A floating launcher opens a chat panel backed by the Vercel AI SDK (OpenAI provider, OPENAI_API_KEY from the environment). A searchDocs tool runs locale-scoped vector/keyword search over the existing docs embeddings so answers cite real pages. The public endpoint is hardened: per-request size/token/step caps, message sanitization (no client-injected tool results or system prompts), origin checks, and a per-IP rate limit. Non-English retrieval uses keyword search; English vector search applies a similarity threshold. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 3066d06. Configure here.}

- searchDocs: wrap the keyword query in try/catch too, so each retrieval path (keyword, vector) is independent best-effort - ask-ai: gate the loading ellipsis to the in-progress (last) message so older empty bubbles don't re-show it while a later request streams Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

vercel Bot deployed to Preview June 22, 2026 20:54 View deployment

vercel Bot deployed to Preview June 22, 2026 21:04 View deployment

ouiliame marked this pull request as ready for review June 22, 2026 21:10

ouiliame requested a review from a team as a code owner June 22, 2026 21:10

cursor Bot reviewed Jun 22, 2026

View reviewed changes

Comment thread apps/docs/app/api/chat/route.ts Outdated

Comment thread apps/docs/app/api/chat/route.ts

vercel Bot deployed to Preview June 22, 2026 21:14 View deployment

greptile-apps Bot reviewed Jun 22, 2026

View reviewed changes

Comment thread apps/docs/app/api/chat/route.ts Outdated

Comment thread apps/docs/components/ai/ask-ai.tsx Outdated

Comment thread apps/docs/components/ai/ask-ai.tsx

greptile-apps Bot reviewed Jun 22, 2026

View reviewed changes

Comment thread apps/docs/app/api/chat/route.ts

cursor Bot reviewed Jun 24, 2026

View reviewed changes

Comment thread apps/docs/app/api/chat/route.ts Outdated

Comment thread apps/docs/app/api/chat/route.ts