runtime: memoize the default VM template; build stdlib tables one-shot - #398
Conversation
`Lua.new()` cost ~40us, and ~80% of that was `Stdlib.install/1`: every
standard-library symbol written into `_G` one key at a time, and every
library table folded key-by-key through the generic mutation path. Callers
that build a VM per request paid the whole install every time, for a value
that never differs.
The built VM is pure and deterministic — no refs, no pids, byte-identical
across calls — and every later mutation is copy-on-write, so a single
template can seed unlimited independent VMs. `Lua.VM.Bootstrap.fetch/2`
memoizes one in `:persistent_term`, written exactly once per node (a `put`
over a live key forces a global process scan). Two keys: the fully
default-sandboxed `%Lua{}` for the default arguments, and the pre-sandbox
install behind custom `:sandboxed` / `:exclude`. Limits and `:debug` are
patched onto the finished struct, so they stay per-instance.
A template holds closures into the stdlib modules; reload those twice in a
shell and the captured funs become badfuns. In `:interactive` mode the
template is fingerprinted by the `module_info(:md5)` of every module
reachable through a captured fun, and a cache hit re-checks it (~0.2us).
`:embedded` mode — releases — never reloads code, so no fingerprint is
taken and a hit is the bare `get`.
Also cuts the install itself, which still runs once per node and on every
`Stdlib.install/1` caller:
* `Table.from_data/1` builds an all-string-keyed map in one shot instead
of sorting and folding `put/3`. The resulting struct is field-for-field
what the fold produced, so iteration order is untouched — pinned by a
new property test.
* `State.set_globals/2` seeds the base globals in one `_G` update.
* `preload_stdlib_modules/1` re-cached five modules `install_library/2`
had already cached; deleted.
* `package.loaded` is resolved once and threaded, not re-walked from the
`package` global per library.
Measured on an M-series laptop, MIX_ENV=prod, median of 5000:
Lua.new() 40.0us -> 0.42us (95x)
Lua.new(sandboxed: []) 30.8us -> 0.29us
Lua.new(sandboxed: [[..]]) 31.8us -> 0.50us
Lua.new(exclude: [[..]]) 39.3us -> 6.5us (own sandbox pass)
Stdlib.install(State.new()) 33.2us -> 13.1us (2.5x, once per node)
Lua.eval!(Lua.new(), "1+1") 44.6us -> 3.1us
The boot VM is unchanged: 3106 words, 12 tables, 37 globals, `g_ref`
`{:tref, 0}`, `table_next_id` 12, and identical `pairs` order over `_G`,
`package`, `package.loaded` and all six library tables.
🤖 Automated review — Opus + Sonnet consolidated findingsTwo independent AI reviewers (Opus, Sonnet) reviewed this PR. The core is solid: boot state is bit-identical to 🟠 MEDIUM — fingerprint misses modules that shape the template without leaving a closure in it
The staleness guard fingerprints only modules reachable via captured funs — 🟠 MEDIUM —
|
Hardens the memoized-template staleness story: - The interactive-mode fingerprint now covers an explicit list of template-shaping modules (Lua, Bootstrap, Limits, State, Stdlib, Table) in addition to modules reachable through captured funs, so a struct-layout or default change invalidates the cached template even when the changed module leaves no closure in it. - Bootstrap.reset/0 erases every memoized template, giving embedded-mode hosts that hot-load new code an invalidation path; stored keys are tracked in a small persistent_term registry. - A fingerprinted module that can no longer be loaded now reads as stale and triggers a rebuild instead of raising UndefinedFunctionError. - The moduledoc no longer claims the key is written exactly once per node; it documents the actual best-effort cold-start semantics (concurrent first callers may each build and put equal terms). - Table.from_data/replace_data property coverage: replace_data/2 is now exercised over lived-in tables (array/hash split, dead keys, order memo, metatable), and generated map values include nil so the all-string one-shot build's nil bail-out is hit. - CHANGELOG entry for the Lua.new/1 template memoization speedup. Claude-Session: https://claude.ai/code/session_01BYHGFLoHBJAsUrTzjVp5nC
|
Pushed cd6f129 addressing the confirmed review findings. The interactive-mode fingerprint now also covers an explicit list of template-shaping modules ( |
🤖 Automated risk assessment —
|
main |
this PR | |
|---|---|---|
Lua.new() warm |
65.3 µs | 1.14 µs |
two new() pointer-identical (:erts_debug.same/2) |
false | true |
| heap cost of 1,000 live VMs | 13,817 B/VM | 31.6 B/VM |
100 VMs after Lua.eval! |
15,713 B/VM | 858 B/VM |
| 32 procs × 500 (new+eval) | 8.44 µs/op | 0.46 µs/op |
:erlang.memory(:total) end of run |
97.3 MB | 71.3 MB |
With default options the struct update in new/1 writes values identical to what's already there, so BEAM returns the map unchanged and callers share the persistent term itself. A process that mutates its VM copies only the path it touches (858 B for a trivial script); its worst case is copying the full 25 KB — which is what main paid unconditionally on every new/1.
Retention check (PR #399's sub-binary hazard)
Structural walk of both stored terms: 0 refs, 0 pids, 0 ports; 335 binaries totalling 1,849 B, every one under 64 bytes (heap binaries, which cannot reference an off-heap refc binary). byte_size vs :binary.referenced_byte_size on every binary: no sub-binary retains more than it shows. Structurally impossible here anyway — the builders take no input, so no lexer/parser output is in scope.
Write cost and staleness
Writes, not storage, are what make :persistent_term expensive (a put over a live key eventually forces referencing processes to major-GC). Traced: cold Lua.new() → exactly 2 builds; next 5,000 warm calls → 0; 64 concurrent tasks × 200 new+eval → 0. So the cost is paid twice at boot, before anything holds a reference.
The highest-value check was whether a fingerprint mismatch could rebuild on every call, turning a one-time cost into a per-call global scan. It can't: module_info(:md5) only changes when code is actually replaced (recompiling unchanged source yields the same md5, so an IEx recompile of untouched files invalidates nothing). Forcing a genuine mismatch by recompiling Lua.VM.Table with an extra function produced one rebuild, then 0 builds across 2,000 subsequent Lua.new() calls.
Findings
- 🟡 LOW/MED (fixed in follow-up):
reset/0depends on a racy{Lua.VM.Bootstrap, :keys}registry —register/1is an unsynchronized read-modify-write. 64 concurrent first-fetches of distinct keys stored all 64 but registered only 10;reset/0would have missed 54. Not a leak or corruption, butreset/0is the escape hatch for embedded releases after:code.load_file/1, and a missed key leaves a stale template serving closures into replaced code. Since the key set is fixed by construction, a module attribute removes the race and oneputper build. - 🟡 LOW (docs):
reset/0de-shares the template for anything currently holding a VM. Quantified: 10,000 holders →reset/0returned in 80 µs but process memory went 43 MB → 364 MB (~33 KB/holder) as each copied the old literal to its own heap. Bounded, one-time, lands nearmain's baseline; versions don't accumulate across repeated resets. Worth a doc line so nobody calls it per-request. - 🟡 LOW (theoretical): if a fingerprinted module were made unloadable,
:code.ensure_loaded/1pays a code-path scan everyfetch(133 µs measured for a nonexistent module) andnil === nilcompares equal so it wouldn't even rebuild. Requires someone to:code.delete/1one of Lua's own modules. - 🟡 Drift guardrail:
fetch/2takes a caller-supplied key. Nothing today derives one from input, but if a future call site did, both store and registry grow unbounded — 2,000 dynamic keys reached 1.36 MB and 312 µs/key. The follow-up commit makes the compile-time-literal invariant explicit in the docs.
Not re-listed: #415 (interactive recheck cost — measured 1.41 µs for the 12-module recheck against a 1.14 µs warm new(), consistent with the ~52% figure), and the already-fixed fingerprint coverage, reset/0 addition, cold-start-race documentation, and deleted-module tolerance.
`reset/0` used to erase whatever keys a `{Lua.VM.Bootstrap, :keys}`
persistent_term entry listed, and `register/1` maintained that entry with
an unsynchronized read-modify-write. Concurrent first fetches therefore
lost updates: each racing builder read the list, added its key, and put
the result back, so a put built on a stale read dropped keys another
process had already recorded. A truncated registry silently narrows what
`reset/0` erases, and a template that survives the reset keeps handing
out closures into replaced code — the exact failure the `:embedded`-mode
escape hatch exists to prevent.
No bookkeeping is needed: the key set is fixed by construction. Both
`fetch/2` call sites pass compile-time literal tuples with argument-less
builders, so the keys now live in a module attribute and `reset/0` erases
exactly those (plus the obsolete registry entry, for releases upgraded
from a build that wrote one). That invariant is what keeps the store
bounded — `:persistent_term` has no eviction — so it is now stated on
`fetch/2` rather than left implicit, and it drops one `put` per build.
Claude-Session: https://claude.ai/code/session_01BYHGFLoHBJAsUrTzjVp5nC
|
Fixed a lost-update race in the reset path (3bee144). |
1 participant
What
Lua.new()goes from ~40 µs to ~0.4 µs by memoizing the built VM, andLua.VM.Stdlib.install/1— which still runs once per node, and on everydirect caller — goes from ~33 µs to ~13 µs by building the tables it
installs in one shot instead of key by key.
Why
Profiling
Lua.new()put ~80% of its cost inStdlib.install(State.new()):2,581 function calls, 154
Table.put/3calls, every standard-library symbolwritten into
_Gone key at a time. Embedders that build a VM per requestpaid that on every single call, for a value that never differs.
It never differs because it can't: the boot VM is a pure function of the
sandbox options. No refs, no pids, no timestamps —
Lua.new()andLua.new()serialize to identical binaries. And every downstream mutation iscopy-on-write, so one template can seed unlimited independent VMs.
How
Lua.VM.Bootstrap.fetch/2memoizes a template in:persistent_term,written exactly once per node (a
putover a live key forces a global scan ofevery process, so the key is never refreshed on the happy path). Two keys:
{Lua, :default_lua}— the fully default-sandboxed%Lua{}. Hit when:sandboxedis the default deny-list and:excludeis empty. Both comefrom the same module literal, so the match is a pointer compare.
{Lua, :base_state}— the pre-sandbox install, shared by every custom:sandboxed/:excludecall, which then pays only for its own sandboxpass.
:max_call_depth,:max_string_bytes,:max_instructionsand:debugarepatched onto the finished struct, exactly as before, so they stay per-instance
and never enter the cache.
Install itself got the follow-on cleanups the profile pointed at:
Table.from_data/1builds an all-string-keyed map (every stdlib librarytable) directly instead of
Enum.sort+ N ×put/3.State.set_globals/2seeds the base globals in a single_Gupdate.preload_stdlib_modules/1re-cached five modules thatinstall_library/2had already cached — pure duplicate work, deleted.
package.loadedis resolved once and threaded through the library installsrather than re-walked from the
packageglobal twelve times.Correctness hazards addressed
Module-reload staleness. A template holds ~116 closures into the
Lua.VM.Stdlib*modules. Reload those twice in a shell and the first versionis purged, turning every captured fun into a
badfun. At build time, in:interactivemode only, the term is walked for closures and fingerprinted byeach reachable module's
module_info(:md5); a cache hit re-checks thefingerprint (~0.2 µs of the 0.42 µs) and rebuilds on a mismatch. Under
:embeddedmode — releases — code never reloads, no fingerprint is taken, anda hit is the bare
get.Mutation isolation. Terms are immutable and
%Lua{}mutation iscopy-on-write, so VMs derived from one template cannot see each other. Pinned
by tests:
Lua.set!(flat and nested), a global assigned from Lua, and astdlib table extended from Lua all fail to appear in the next
Lua.new(), andtable_next_idis still 12.pairsorder. The one-shot table build produces the struct the foldproduced, field for field —
order_tailin descending key order,borderuntouched — so nothing observable moves. A new property test asserts
from_data/1 ==the sortedput/3fold over generated maps mixing dense,sparse and string keys. Verified directly, too:
pairsorder over_G,package,package.loaded,string,math,table,os,utf8anddebugis unchanged.Boot invariants.
g_refis{:tref, 0},table_next_idis 12, 12tables, 37 globals, and the term is still exactly 3106 words — the same size
as before this PR.
Measured
M-series laptop,
MIX_ENV=prod, median of 5000 iterations (machine wasotherwise idle; ±10% run to run):
Lua.new()Lua.new(sandboxed: [])Lua.new(sandboxed: [[:os, :exit]])Lua.new(exclude: [[:require], [:package]])Stdlib.install(State.new())Lua.eval!(Lua.new(), "return 1+1")The
exclude:row is the level-1 path: it skips the install but still runs itsown 25-path sandbox pass, which is now what it costs.
Validation
mix test --include lua53— 2653 passed, 16 skipped (was 2638 before thenew tests).
mix format --check-formatted,mix compile --warnings-as-errors,mix dialyzer(0 errors),mix docs --warnings-as-errors.baseline in
ROADMAP.md.