feat: Ship the CLI as a Bun standalone binary#195
Open
gjtorikian wants to merge 18 commits into
Open
Conversation
Users previously needed a Node.js runtime to run the CLI, and every release shipped transpiled JS through a single npm package. Compiling with Bun produces one self-contained binary per platform, so the CLI runs with no runtime prerequisite and distributes directly through GitHub Releases. A compiled binary cannot discover package assets on disk at runtime, so integrations, bundled skills, and the Agent SDK executable move to generated manifests: the first two are embedded statically, while the Agent SDK is downloaded on first agent use and verified against a sha256 pinned at build time. Every release binary is smoke tested on native hardware for all five targets (including the new `workos internal verify-assets` command) before the draft release publishes, so a broken binary can never become `latest`. npm remains a secondary channel: a thin launcher package plus one platform package per binary (the esbuild pattern) preserves `npm install -g workos` and `npx workos`. BREAKING CHANGE: The npm package no longer exports a library API — `main`/`exports` are gone and it only provides the `workos` binary. Development now requires Bun >= 1.3.0 instead of Node >= 22.11.
The Bun standalone binaries only covered glibc Linux, so the CLI could not run on Alpine and other musl systems, and Windows ARM users were left running the x64 build under emulation. Runtime musl detection mirrors the napi-rs loaders so the npm launcher and the Agent SDK download both resolve the same target the binary was compiled for, and musl artifacts are smoke tested in real Alpine containers because no glibc host can prove they run.
Greptile SummaryThis PR ships the WorkOS CLI as a self-contained Bun standalone binary for 8 platforms, moving GitHub Releases to the primary distribution channel while keeping npm as a thin launcher (esbuild pattern). Skills and integration metadata that previously relied on
Confidence Score: 5/5Safe to merge; all intentional design trade-offs are documented and mitigated by the new verify-assets command and the release pipeline's per-platform smoke gates. The core security-sensitive paths (SHA256 verification on first download, atomic install, uid ownership checks on the extraction root) are correctly implemented and well-tested. The non-blocking findings are: a missing compressed-download size cap before decompression, a TOCTOU on the cache-hit stat path, a fragile redirect-regex in the update check, and an analytics field rename. None affect correctness of the binary distribution or the agent SDK installation flow. src/lib/agent-sdk-assets.ts deserves a second look on the download accumulation loop (no compressed-size cap) and the existsSync/statSync TOCTOU on the cache-hit path. Important Files Changed
|
A partial npm publish failure left the release permanently half-published: re-running the job hit npm's cannot-publish-over- existing-version error on the first already-live package and aborted before reaching the unpublished ones, including the launcher. Guarding each publish with `npm view` makes re-runs converge, so the job comment's "re-run just this job" recovery actually works.
On Alpine, `bun install` (postinstall runs generate) pinned the glibc Agent SDK package while the runtime's musl detection demanded the musl one, so every dev agent use threw a target mismatch. The keyring-binding check in build.ts assumed glibc the same way. Both scripts now mirror the runtime's isMuslRuntime() when WORKOS_BUILD_TARGET is unset; explicit targets are unchanged.
The first-run download (~100MB from the npm registry) had no abort signal, so a stalled connection hung `workos install` forever mid-progress, and any transient network error failed the run outright. A stall timer that resets on each received chunk catches both connect hangs and mid-stream stalls without penalizing slow links, and a single retry absorbs transient failures. Checksum mismatches stay hard failures and are never retried.
After the first install the cached executable is only revalidated by file size, so silent size-preserving corruption (disk fault, antivirus quarantine/restore) passed unnoticed until runtime. `internal verify-assets` is the diagnostic a user with a corrupted cache gets pointed at, so it now re-hashes the executable against the pinned manifest digest and fails with a distinct error code and a delete-the-cache remedy.
The concurrent-extraction recovery in materializeFile (accept a winner's byte-identical copy, re-throw on divergence) had no coverage; only the happy path was exercised. A mocked renameSync that plants the winner's file before failing forces both branches and proves no temp files are orphaned either way.
Nothing in src/ imports it, so it reads as removable — but ink's devtools.js statically imports it and `bun build --compile` cannot prove the DEV-gated branch dead, so removing the dep fails the compile. `--external` compiles but crashes the binary on every command. Documenting the measured ~742 KiB cost and the failed alternatives keeps a well-meaning cleanup from breaking the build.
Conflicts and semantic resolutions against #192: - src/lib/validation/validator.ts: kept this branch's static JSON rule imports (required for the compiled binary) alongside main's new detectPort import; main's port-detection call sites auto-merged. - src/bin.ts: took main's $0 default handler (JSON command tree via buildCommandTree, else parser.showHelp()) — it supersedes this branch's one-line scriptName fix because the parser already carries .scriptName('workos'). - src/bin-default-command.integration.spec.ts (new on main): converted the subprocess spawn from `node --import tsx` to `bun --preload`, matching bin-command-telemetry.integration.spec.ts — tsx is no longer a dependency on this branch. - 12 new tests from #192 asserted the `npx workos@latest` hint form when npm-exec variables are present; this branch always emits the standalone `workos` form, so those tests now assert the hints are invariant to npm env, matching recovery-hints.spec.ts.
The only npm-channel check was running the launcher script directly with NODE_PATH, which bypasses everything that can actually break for users: registry fetch, optionalDependencies platform selection, npx cache and bin linking, and the launcher's no-binary error path. A manual dress rehearsal against a local registry surfaced real gaps the shortcut can't see (npm nests global deps inside the package; a brew-installed `workos` shadows bare `npx workos` unless the prefix and PATH are isolated). Codifying it makes `npx workos` a gated guarantee: every PR runs it, and the release pipeline runs it after generating the real packages — so a packaging regression fails before anything touches npmjs.org. The registry has no uplinks, proving the install is self-contained.
The smoke gates proved the binary starts (--version, --help, verify-assets) but nothing executed real subcommands and asserted the non-TTY contract that agents and CI pipelines script against: exit codes (0 success, 1 error, 4 auth required), structured JSON errors on stderr, and JSON output. Seven of the eight platform binaries never ran a user-facing command before shipping. command-smoke.sh is POSIX sh so the same checks run inside the --network none debian container on PRs, inside the Alpine containers for musl, under Git Bash on the Windows runners, and directly on the mac/linux legs — every release binary now executes the contract on native hardware before the draft release publishes. It sandboxes HOME/USERPROFILE and uses --insecure-storage so host auth state can never leak in, keeping the exit-4 assertion deterministic.
gjtorikian
changed the title
The smoke gates covered offline behavior only — no shipped binary ever executed a command that talks to the WorkOS API before release. With a dedicated staging-environment key the contract smoke now also runs an authenticated section: organization list plus a create → get → delete round-trip, exercising key resolution, real HTTP, and JSON output on the write path. The key is withheld from the offline checks so the exit-4 assertion stays deterministic, a cleanup trap deletes the round-trip org even when a mid-flight check fails, and the CI step skips itself when the WORKOS_SMOKE_API_KEY secret is absent — fork PRs receive no secrets, so this cannot fail there.
…ng API URL The authenticated section discarded stderr, so a CI failure showed exit codes with no cause. The CLI's structured errors are key-free by design (keys are masked in all output), so printing them is safe and turns a blind failure into a diagnosis. WORKOS_SMOKE_API_URL lets the smoke key target a non-default API host, guarded so an unset secret cannot inject an empty WORKOS_API_URL.
A keyring or file blob missing the required token fields — left by a partial write or an older schema — was returned as-is by getCredentials(). Consumers assume accessToken/expiresAt/userId exist, so `new Date(undefined).toISOString()` threw on every authenticated command AND on auth status, bricking the CLI until the entry was deleted by hand. Found on a real machine while smoke testing this branch: the entry held only the staging sub-object. The bug predates the Bun migration (main has the same code), but the binary upgrade makes stale keyring entries a mainstream path, so validate required fields at both read sites and degrade to "not logged in — run workos auth login", which also overwrites the bad entry on the next login. Malformed file blobs are no longer migrated into the keyring either.
When the version gate refused an install (Next.js < 15.3, React Router < 6), the integration returned an empty summary, which the runAgent wrapper mapped to success:true — so agents and CI scripting `workos install --json` saw "Successfully installed WorkOS AuthKit!" with exit 0 while nothing was installed. Found by running the compiled binary against the bundled Next.js 14 fixture. Gates now throw InstallDeclinedError, which rides the machine's existing error path: exit 1, a structured stderr error and NDJSON error/complete events carrying unsupported_framework_version, while the human flow keeps its friendly guidance (adapters recognize the decline code and skip the generic failure styling and AI-service message rewrites).
The typescript-strict fixture pinned Next.js ^14.2.0 while the installer's own version gate requires >= 15.3.0, so the bundled fixture could never exercise the agent path — every eval or manual run against it hit the gate instead. Next 15 pairs with React 19, whose types drop the global JSX namespace, so the annotations move to ReactElement. Verified with tsc --noEmit and next build.
gjtorikian
changed the title
Contributor
Author
|
@greptile-apps bypass the limit |
The recent malformed-credentials fix made getCredentials() return null for a blob missing required token fields, but hasCredentials() still reported true from a bare file/entry probe. A caller that gated on hasCredentials() alone would treat a malformed blob as logged-in and then hit the very null the fix was meant to prevent. Validating at both read sites keeps the two functions from ever disagreeing, without triggering getCredentials()'s keyring migration as a side effect of a boolean check.
Three robustness gaps surfaced while reviewing the first-run download path: - The stale-cache reap deleted every sibling version dir unconditionally, so an upgrade run could remove the executable out from under a concurrently running other-version CLI. A 24h staleness guard (matching the skills reaper) leaves fresh dirs alone. - gunzip ran unbounded before the sha256 gate, so a compromised or corrupt registry response could expand a gzip bomb in memory before verification had a chance to reject it. Capping output at the pinned executable size plus headroom bounds it. - A retry restarted the byte count at zero, which froze the progress readout until it re-passed the previous peak. An onRetry hook resets the throttle and tells the user it is retrying.
Contributor
Author
|
@greptile-apps bypass the limit |
|
Could you clarify what you mean by "bypass the limit"? I don't see an active review thread with a specific limit being discussed. Are you referring to:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
workos internal verify-assetscommand), so a broken binary can never becomelatest.npm install -g workosandnpx workos.@workos/mcp-docs-serverMCP server from install agent runs — spawning it vianpx -yassumed Node.js on the user's machine, which the standalone binary can no longer do. Inlined skills reference content (already the primary docs path) replaces it, and a regression test (no-skill-tool.spec.ts) guards against reintroduction.Issues encountered
The two hardest problems shared a root cause: in a compiled binary the module graph lives in Bun's virtual filesystem (
$bunfs), so nothing that used to be "a file next to the package" exists on disk anymore.Skills had to become embed-and-extract. The install agent can only load skills from a real directory, but the compiled binary has no package directory to point it at — and dev-mode tests resolve assets from
node_modules, so they pass even when the in-binary path is completely broken. Skills are now embedded via a generated manifest and materialized on demand into a version-keyed temp dir: atomic write-to-temp + rename, concurrent extraction races resolved by byte-comparing the winner, uid-ownership/0700checks on the shared tmp root, and reaping of older versions' trees so tmp doesn't grow forever. Because no ordinary test can cover the in-binary extraction path, the newworkos internal verify-assetscommand exercises it directly, and CI runs it against every release binary.workos installdownloads the Anthropic Agent SDK on first use. The agent spawns the Claude Agent SDK's nativeclaudeexecutable (~230 MB, one per platform). Embedding it would balloon every release binary, so the compiled CLI fetches the@anthropic-ai/claude-agent-sdk-<target>tarball from the npm registry on first agent use instead — pinned at build time to an exact version and sha256 of the executable (any mismatch refuses to install), extracted with a minimal in-house ustar reader so there's no npm/tar dependency at runtime, and installed atomically under~/.workos/cache/agent-sdk/<version>-<target>with cleanup of prior versions. A glibc/musl target guard fails fast with a clear error rather than crashing inside a mismatched SDK binary.Breaking changes
main/exportsare gone and it only provides theworkosbinary.🤖 Generated with Claude Code