experiment: compile hyperion for bare metal (Hermit unikernel, virtio-net) — DRAFT, do not merge - #959
Draft
andrewgazelka wants to merge 4 commits into
Draft
experiment: compile hyperion for bare metal (Hermit unikernel, virtio-net) — DRAFT, do not merge#959andrewgazelka wants to merge 4 commits into
andrewgazelka wants to merge 4 commits into
Conversation
Names the five things a hosted OS and a unikernel disagree about -- the open-file limit, the wall clock, durable storage, AF_UNIX, and vCPU count -- and puts one cfg arm behind them. Hosted is the default and behaves exactly as hyperion did before, so a normal build sees no change.
A separate cargo workspace, so the Hermit git dependency and its nightly never enter the main lockfile. The same source builds as an ordinary Linux binary and as an x86_64-unknown-hermit machine image; on the latter it boots under QEMU, brings up virtio-net, takes a DHCP lease and serves the handshake/status/ping exchange with valence_protocol.
One vendor FOD covers all three cargo runs -- application, -Z build-std of std, and the nested kernel build -- because the kernel's build script strips every CARGO_* and RUST_* variable and can only be reached through $HOME/.cargo/config.toml.
Leads with what runs and what does not, and ends with an explicit list of what was never verified, including that nix build .#default was already broken on main before this branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Experimental. Do not merge. Opened as a draft so the findings are reviewable; the code is a probe, not a feature.
What works
nix build .#bare-metalproduces anx86_64-unknown-hermitunikernel image.nix run .#bare-metal-vmboots it under QEMU, where it brings up virtio-net, takes a DHCP lease, and 13 ms after boot answers a Minecraft server-list ping decoded with the samevalence_protocolthe real server uses. The identical source also builds and runs as an ordinary Linux/macOS binary.Two survey results made that possible and are the most transferable findings here:
valence_protocolcross-compiles to Hermit unmodified, and so doesbevy_ecs.What does not work
The game server does not build. 11 of 24 workspace crates compile for the target; 13 do not,
hyperionandbedwarsamong them. Four dependencies account for nearly all of it —socket2,openssl-sys(pulled in byreqwest's default features despiterustls-tlsbeing requested),libz-ng-sys, andring— and behind those sitheed/LMDB,memmap2,libdeflaterandndarray+blas, which a machine with no filesystem cannot host at all.There is no virtio-mem anywhere in Rust guest land. Hermit has no such driver (
src/drivers/is console, fs, net, vsock);virtio-drivershas no driver, no issue, and no PR. Ballooning is an unmerged PR from June. If elastic memory is a requirement, that is Firecracker with a Linux guest, not a Rust unikernel.Tokio on Hermit means two unmaintained forks (
hermit-os/socket2,hermit-os/tokiopinned to 1.45.0, last commit 2025-05-08).mioandpollinghave genuine upstream support; tokio does not.The trap worth knowing before anyone else tries this
Hyperion's pinned
nightly-2025-02-22builds a Hermit image that boots, prints, and then fails every socket call withbind: Kind(Uncategorized). That nightly'sstdis built againsthermit-abi 0.4; kernel 0.13.2 answers0.5. Boot andprintln!happen to line up across the gap; sockets do not. The identical source onnightly-2026-07-01binds first try.bare-metal/therefore carries its own toolchain pin.Separately: the repo's
.cargo/config.tomlputs-Ctarget-cpu=nativein[build] rustflags, which applies to every target, so any cross-compile dies insidecorewithrustc-LLVM ERROR: 64-bit code requested on a subtarget that doesn't support it!. SettingRUSTFLAGSoverrides it, which is why that file is untouched here.The durable part: a seam, not a fork
crates/hyperion-platformnames the five things a hosted OS and a unikernel actually disagree about — the open-file limit, the wall clock, durable storage,AF_UNIX, and vCPU count — plus aCAPABILITIESconstant so a call site asks "is there a filesystem?" rather than "am I on Unix?". The backend is onecfgarm; a third platform is a newbackend/*.rsand a row in a table.Hermit ships a real
std, so this is deliberately not a portability layer. Almost nothing needs wrapping, and a wider seam would be a second standard library to maintain.Blast radius
No existing crate was modified. Everything is new files plus two additive edits:
Cargo.tomlcrates/hyperion-platformas a memberflake.nixcallPackage, twopackagesentries, oneappsentryflake.lockCargo.lockWiring the seam into
hyperionwas deliberately skipped: it would buy nothing today (hyperionis blocked on four C libraries, not oncfg(unix)) and would put a diff into files other streams are editing. The seven call sites it is for are enumerated in the doc.What it would take to close the gap
The realistic next milestone is not "the server boots". It is the proxy: it needs only the
socket2andtokioforks and has no filesystem dependency worth the name. Gettinghyperionitself to boot means replacing persistence (LMDB), chunk storage (mmap'd Anvil regions), and the compression backend — that is a redesign, not a port.Cheap wins that are worth doing regardless of this branch:
bvh-regionhasproptestin[dependencies], not[dev-dependencies]. It ships a test framework into release builds on Linux too.reqwestis requested withrustls-tlsbut keeps default features, soopenssl-syscomes along as well.Not verified
nix build .#defaultfails on this branch — and already failed on unmodified313503c, reproduced in a detached worktree (A hash was specified for divan-0.1.17, but there is no corresponding git dependency). Pre-existing, but it means the existing nix path could not serve as a regression check.cargo build --workspacewas run. "The normal build is unchanged" rests on the diff, not on a measurement.aarch64-darwincross tox86_64-unknown-hermit, QEMU without KVM. Linux untested; aarch64/riscv64 Hermit untested.heed,memmap2,libdeflater,ndarray+blas,tracing-tracy,valence_anvil) is reasoned from manifests. No build reached them.socket2/tokioforks are named as the route to the proxy but were not built here.Full write-up, including the prior-art comparison against Motor OS, Unikraft,
virtio-drivers+smoltcp, Nanos/OSv and Firecracker with versions and dates:docs/bare-metal.md.