Skip to content

feat(tui): add an External llama.cpp pane to the LLM tab - #50

Open
sachin-detrax wants to merge 2 commits into
AtomicBot-ai:mainfrom
sachin-detrax:feat/llm-panel-external-mode
Open

feat(tui): add an External llama.cpp pane to the LLM tab#50
sachin-detrax wants to merge 2 commits into
AtomicBot-ai:mainfrom
sachin-detrax:feat/llm-panel-external-mode

Conversation

@sachin-detrax

Copy link
Copy Markdown
Contributor

Closes #49.

What

Adds external as a third pane of the LLM tab, cycled with ←/→:

Mode: Local | Cloud | External llama.cpp
Press ←/→ to switch mode

External llama.cpp
> base URL http://127.0.0.1:8080 [not active] · Enter: point the chat route at an external llama.cpp

  managed daemon: running pid 42 on 127.0.0.1:19091 · s start/stop
  ← Local pane: pick a managed model to switch back

The pane is a single row because an external llama-server is one base URL. Enter opens an inline editor seeded with the current URL; saving it runs the existing /health probe and only then persists localModels.url + mode: "external", so an unreachable address never steals a working route. A scheme-less host gets http:// prepended; unparseable input keeps the editor open behind an invalid URL hint rather than closing and silently discarding the edit.

Once the route is live the row reads * base URL http://192.168.1.50:8080 [ok · 12ms] and the status line switches to mode external · <url>.

Going back to managed is unchanged: pick a model in the Local pane, which already persists mode: "managed".

Fixes in the persist path

Both are shared with /models <base-url>, which had the same defects:

  • Point the chat route at local-llama when a cloud provider is active, so the saved URL is not inert. Skipped when the route is already local, so editing a URL does not emit a spurious "Switched active text provider" line.
  • Call localModels.refresh() so the panel stops reporting managed mode.

Both run after the probe succeeds, so a failed switch changes nothing.

Managed daemon teardown

Switching to external now stops the managed chat daemon, which would otherwise hold its VRAM for a route nothing uses. stopChatDaemonOnly() deliberately leaves the embedding daemon and memory.embeddings.enabled alone — unlike the cloud path's stopDaemon(), which stops both and turns hybrid recall off. It no-ops when nothing is running.

Skipped when the new URL is the managed daemon's own loopback address: driving a daemon started with atomic-agent models start is a supported setup, and stopping it would kill the server just pointed at. pointsAtManagedDaemon(url, port) takes the port as an argument so it stays pure and testable.

Implementation notes

  • LlmPanelMode gains external; the two cursor fields become three behind cursorFieldFor(mode), replacing the mode === "cloud" ? … : … branches in the reducer, selectors, and key bindings.
  • / now carry a direction (switchLlmMode(state, dispatch, delta)); with three panes a plain toggle no longer works.
  • The URL editor is a modal in handleLlmModalKey, so panel hotkeys (s, n, c, /) go into the buffer while it is open.
  • parseExternalUrl is shared between the Enter guard and the modal renderer so the hint and the guard cannot disagree.

Testing

  • src/tui/llm-panel/llm-panel-external.test.ts — 10 tests: pane cycling in both directions, row active/inactive state, editor open/type/backspace/save/cancel, scheme-less normalization, the unparseable-input guard, per-pane cursors.
  • pointsAtManagedDaemon covered in src/tui/persist-user-local-models-config.test.ts (loopback spellings, other ports, other hosts, garbage input).
  • Verified by rendering the real panel through the real key handler end to end (throwaway harness): the frames above are actual output, and onPersistLlamaUrl(http://192.168.1.50:8080) fired from typing 192.168.1.50:8080.
  • tsc --noEmit clean, npm run build succeeds.
  • src/tui: 656 passed / 6 failed, the same 6 that fail on main (splash-banner copy, chat-log, persist-embedding, two TuiApp smoke tests, one llm-panel selectors test). One of them, TuiApp (smoke) > shows the two-mode LLM panel, asserts header copy this PR changes — the assertion is updated, but the test still fails for its pre-existing reason (the test terminal is too short to render the full header, so it takes the compact path). Not fixed here.

Not included

  • A /models external slash subcommand.
  • Making bare /models land on the External pane when it is the active route. resolveModeFromActiveRoute would need configMode, which is still at its default until the first refresh — managed users would land on the wrong pane at startup.

Switching a managed install back to an external llama-server was only
reachable through `/models <base-url>` — the LLM tab showed the current
mode but had no way to change it. Add `external` as a third pane
alongside Local and Cloud, cycled with ←/→.

The pane is a single row because an external llama-server is one base
URL. Enter opens an inline editor seeded with the current URL; saving it
runs the existing `/health` probe and only then persists
`localModels.url` + `mode: "external"`, so an unreachable address never
steals a working route. A scheme-less host gets `http://`; unparseable
input keeps the editor open behind an "invalid URL" hint.

Two fixes to the persist path itself, shared with `/models <base-url>`:
point the chat route at `local-llama` when a cloud provider is active
(the saved URL was otherwise inert), and refresh the local-models panel
so the tab stops reporting managed mode. Both run after the probe, and
the route switch is skipped when it is already local so editing a URL
stays quiet.

Going back to managed is unchanged: pick a model in the Local pane. The
managed daemon keeps running after a switch — the pane says so, and `s`
stops it.
Pointing the chat route at an external llama-server left the managed
daemon running and holding its VRAM for a route nothing used. Tear it
down once the switch actually lands — after the /health probe and the
config write, so a failed switch changes nothing.

Chat only: `stopChatDaemonOnly()` leaves the embedding daemon and hybrid
recall alone, unlike the full `stopDaemon()` the cloud path uses, which
also flips `memory.embeddings.enabled` off. It no-ops when nothing is
running so callers can fire it unconditionally.

Skipped when the new URL is the managed daemon's own loopback address —
driving a daemon you started yourself with `atomic-agent models start`
is a supported setup, and stopping it would kill the server just pointed
at. `pointsAtManagedDaemon` takes the port as an argument so it stays
pure and testable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No way to switch back to an external llama.cpp from the LLM tab

2 participants