Skip to content

andreaborio/dsbox

Repository files navigation

DSBox logo

DSBox

Your Mac. Your model. One switch.

A polished Apple Silicon desktop app for running andreaborio/ds4 with Metal AUTO residency, SSD streaming, local chat, coding-agent endpoints, and honest macOS telemetry.

Latest release CI macOS 13 or later Apple Silicon MIT license

Download for Apple Silicon · Installation guide · Run from source

DSBox local chat

A local-first workspace with persistent threads, a compact model switcher, optional reasoning, and a clean Codex-style interface.

Why DSBox

  • One power control. Once a model is selected, DSBox can prepare the checkout, build ds4-server, validate flags, start it, and wait for real readiness.
  • Models without path hunting. Scan the Mac, choose a GGUF with Finder, or review and download a Hugging Face variant inside DSBox.
  • SSD streaming without artificial lockouts. DSBox warns when a model may be very slow, but does not block an experiment merely because the GGUF is larger than unified memory.
  • Resident when it safely fits. Qwen3.6 uses DS4's guarded Metal AUTO planner, keeping the complete model resident when the working set and current pressure allow it and falling back to SSD otherwise.
  • A complete local chat. Threads, reasoning, stop control, automatic scrolling, syntax-highlighted code, one-click copy, and response-level prefill/generation timings.
  • Bring your coding agent. Stable OpenAI, Responses, and Anthropic-style loopback endpoints with ready-to-copy configurations.
  • Telemetry that says what it knows. Memory pressure, committed memory, swap, CPU, process RSS, disk, and generation speed are reported; unsupported GPU metrics remain N/A.

Three steps

  1. Choose a model. Use a validated GGUF already on the Mac or explicitly confirm an in-app catalog download.
  2. Turn on DSBox. The Server screen prepares and launches DS4 with Metal and guarded automatic memory planning.
  3. Chat or connect an agent. Use the built-in interface, or copy the endpoint for Codex CLI, Claude Code, OpenCode, Pi, or another compatible client.
Models Server
DSBox model catalog DSBox server power screen
Browse revision-pinned sources, compare variants, and see hardware guidance before downloading. A single power surface keeps startup, adaptive defaults, runtime state, and safe shutdown understandable.

Models, made transparent

Use a GGUF already on the Mac

Scan this Mac checks Spotlight first and falls back to a bounded filesystem scan. Before adding a result, DSBox reads the GGUF v3 header, architecture metadata, and tensor index to verify that the file uses a layout supported by DS4. It never reads the model-weight payload during this check, so validation stays lightweight even for very large files. Verified findings are stored in ~/.dsbox/local-models.json with user-only permissions, so the chat model switcher opens immediately instead of scanning the disk again. Deleted, unreadable, corrupt, multipart, or incompatible entries are pruned automatically.

Choose GGUF file… opens the native Finder picker. DSBox uses the selected model in place: it does not copy or upload it, and it never asks a non-technical user to type a path.

A generic GGUF container is not enough: DS4 requires its own architecture metadata and tensor layout. Finder selection, disk scan, model switching, and server startup all use the same compatibility gate, with a specific explanation when a file cannot run.

Download inside DSBox

The catalog reads DS4-oriented sources from the Hugging Face andreaborio profile and includes a checksum-pinned, DS4-native DwarfStar model from antirez/deepseek-v4-gguf. Unsloth repositories remain visible for provenance, but their current standard multipart GGUF builds cannot be selected or downloaded because DS4 does not support that layout. The exact revision, total size, file count, free-space check, and hardware advisory are shown before any compatible download begins.

Downloads are:

  • explicitly confirmed—turning on the server never silently starts one;
  • pinned to a Hugging Face revision;
  • resumable after interruption;
  • staged and committed atomically;
  • size-verified, with SHA-256 verification when the source publishes it;
  • enabled only when the model layout and runtime contract are explicitly verified for DS4.

Recommendations are made by DSBox, never presented as an endorsement from Andrea Borio, Antirez, or Unsloth.

Switch models from chat

Installed inventory models appear beside the Thinking control. If DS4 is off, a selection applies to the next start. If it is running, DSBox validates the replacement first, restarts the server, and attempts to restore the previous model and runtime if the new launch fails. Switching is disabled while a generation, download, or conflicting runtime operation is active.

Install the macOS app

DSBox ships as an arm64 Electron app for macOS 13 or later.

  1. Download DSBox-<version>-macOS-arm64.dmg and SHA256SUMS.txt from the latest release.

  2. Verify the download from the same folder:

    shasum -a 256 -c SHA256SUMS.txt
  3. Open the DMG and drag DSBox to Applications.

  4. The community build is ad-hoc signed but not notarized. On first launch, Control-click DSBox, choose Open, then confirm Open.

The checksum-first Gatekeeper and “app is damaged” recovery flow is documented in docs/INSTALL-macOS.md. Only bypass quarantine for an artifact whose checksum you have verified.

There is no automatic app updater yet. Upgrade by quitting DSBox and replacing the application with the newer DMG; models, configuration, downloads, and local threads live outside the app bundle.

Run from source

Requirements: Apple Silicon, macOS 13+, Node.js 22+, Xcode Command Line Tools, and enough free SSD space for the selected model.

xcode-select --install
git clone https://github.com/andreaborio/dsbox.git
cd dsbox
./start.command

start.command installs JavaScript dependencies, builds the UI, starts the loopback control plane, and opens DSBox. For development:

npm ci
npm run dev
  • UI: http://127.0.0.1:5173
  • Gateway and control plane: http://127.0.0.1:4242

Coding-agent endpoints

The public loopback gateway stays stable even if the internal DS4 port changes.

Protocol Base URL Endpoint
OpenAI Chat http://127.0.0.1:4242/v1 /chat/completions
OpenAI Responses http://127.0.0.1:4242/v1 /responses
Anthropic Messages http://127.0.0.1:4242 /v1/messages
Model discovery http://127.0.0.1:4242/v1 /models

The Agents screen generates configurations using the currently selected model and gateway settings. If gateway authentication is enabled, copy the real key from Settings instead of using a placeholder.

Example Codex provider:

[model_providers.ds4]
name = "DS4 local"
base_url = "http://127.0.0.1:4242/v1"
wire_api = "responses"
stream_idle_timeout_ms = 1000000

Then use the model ID shown by DSBox:

codex --model <selected-model-id> -c model_provider=ds4

SSD streaming and safety

DSBox does not pretend that “fits on SSD” means “will be fast.” A model larger than unified memory may run through DS4 SSD streaming, but speed depends on model structure, quantization, storage, thermal state, and cache warmth. Hardware guidance is advisory; insufficient disk space is the hard download blocker.

Adaptive mode lets DS4 calculate its expert-cache budget from live model geometry and available memory. DSBox independently watches macOS pressure and swap activity while the runtime is active, and performs a safety stop if pressure becomes unsafe or required signals repeatedly disappear. Manual context, cache, KV-disk, trace, imatrix, flags, and environment controls remain available in Settings.

For the validated Qwen3.6-35B-A3B path, DSBox requires DS4 1fdfe08 or newer and deliberately passes neither --ssd-streaming nor --resident. DS4 therefore admits full residency only after its own working-set and live memory-pressure checks, with SSD streaming as the safe fallback. DSBox keeps its independent one-second pressure and swapout watchdog armed in either outcome.

Recorded Qwen3.6 reference results

On a MacBook Pro M5 Pro with 64 GB unified memory, the normalized 19.37 GiB Qwen3.6-35B-A3B-ds4-Q4_K_S.gguf artifact ran fully resident on Metal. A balanced six-run serial/parallel router A/B improved median generation from 37.00 to 62.96 t/s (+70.2%) with identical output hashes. Five quiet-desktop confirmations measured median prefill at 223.73 t/s and generation at 65.63 t/s; the same binary measured 50.20 t/s generation under active compositor and Codex GPU contention. The optimized Thinking sampler improved a short-context A/B from 39.71 to 58.59 t/s (+47.6%). At positions 1,428-1,628, parallel full- attention GQA improved an adjacent A/B from 36.18 to 44.06 t/s (+21.8%) with an identical response hash. A 504-token DSBox-like Thinking turn measured 320.30 t/s prefill and 41.04 t/s generation while its attention prefix grew from 1,426 to 1,930 tokens. These changes schedule sampling and attention more efficiently; they do not change or further quantize model weights. Every recorded process reported zero swaps. A final active-desktop parity check measured 46.61 versus 46.54 t/s at short context and 33.03 versus 33.19 t/s at a 1,403-token prefix without/with Thinking, confirming that full-vocabulary sampling no longer adds a material decode penalty. These are bounded machine-local measurements, not guarantees for arbitrary prompts or foreground GPU load; the full methodology is in the DS4 tests/qwen/README.md.

Recorded DeepSeek V4 Flash reference results

The table below uses the 86.72 GB DeepSeek V4 Flash IQ2XXS/SExpQ8 GGUF and the immutable reference andreaborio/ds4@91e0f5d. Short runs are highly sensitive to prompt length and macOS page-cache state; these are measurements, not hardware guarantees.

Mac Bounded workload Generation throughput
M1 Pro, 16 GB DSBox API, 9 prompt + 2 output tokens 0.30 t/s cold; ~0.52 t/s warm
M5 Pro, 64 GB two DSBox API requests, 22–23 prompt + 64 output tokens 9.88 / 12.86 t/s
M5 Pro, 64 GB ds4-bench, 128 prompt + 64 decode tokens 13.05 / 13.59 t/s

Privacy and current limits

  • Inference, models, configuration, and thread history stay on the Mac. Threads use browser/Electron local storage; they are not encrypted or synchronized.
  • The control plane and internal DS4 server bind to loopback only. Mutating control actions require a custom header, and optional Bearer/x-api-key gateway authentication is available.
  • DSBox is currently text-only. Image, audio, video, and file parts are rejected before they can reach DS4.
  • Automatic web search is intentionally transparent in the composer, but it is not fully offline: when the local router detects an explicit or time-sensitive search request, a normalized query of at most 400 characters is sent to DuckDuckGo Lite. Search failure falls back to local inference.
  • Metal utilization remains N/A because macOS does not expose a reliable per-process value without elevated tooling. DSBox does not use sudo or powermetrics.
  • The community DMG is ad-hoc signed and not notarized. A fully trusted first launch requires Apple Developer ID signing and notarization.

Architecture

flowchart LR
    UI["React + Electron UI"] -->|"control API + SSE"| CP["DSBox control plane"]
    HF["Hugging Face sources"] -->|"revision-pinned downloads"| CP
    CP -->|"argv spawn + lifecycle"| DS4["ds4-server · Metal"]
    UI -->|"local chat"| GW["Gateway · 127.0.0.1:4242"]
    AG["Coding agent"] -->|"OpenAI / Responses / Anthropic"| GW
    GW -->|"text requests + streamed responses"| DS4
    DS4 --> SSD["GGUF · expert cache · KV disk"]
    CP -->|"macOS metrics + logs"| UI
Loading

The gateway parses and validates incoming requests, rejects unsupported media, removes hop-by-hop headers, and relays response status, safe headers, and streaming bytes. It does not translate semantic text, tool, or reasoning fields.

Development and releases

npm run typecheck
npm test
npm run build

Build and verify the macOS artifacts:

npm run pack:mac
npm run dist:mac
npm run verify:mac

A version tag matching package.json runs the same checks on GitHub Actions and publishes the DMG, SHA-256 file, and installation guide.

Security

Please report security issues privately to the repository owner. Do not expose DSBox or ds4-server directly on 0.0.0.0; use an SSH tunnel for access from another machine.

License

MIT

About

fully managed local inference dashboard based on my fork of ds4, optimized for metal

Resources

License

Contributing

Security policy

Stars

2 stars

Watchers

0 watching

Forks

Packages

 
 
 

Languages