gc: run the allocation-point GC trigger outside the &mut Arena borrow (#7022) - #7050
Conversation
…orrow (#7022) `Arena::alloc(&mut self, ..)` called `gc_check_trigger()` from inside its own borrow. A collection allocates into the arenas — promotion and C4b evacuation call `arena_alloc_gc_old`, an evacuating minor (#7019) fills a survivor semispace — and either can reach `Arena::install_fresh_block` → `self.blocks.push(..)` on the SAME arena the allocating frame is holding. The Vec growth frees the buffer the outer frame goes on to index, and `&mut` carries `noalias`, so the outer frame may also have cached `blocks.ptr`/`len` across the call. Measured on the #7022 reproducer: `install_fresh_block gen=Old space=Old` fires while `Arena::alloc` on the old arena holds the borrow, 204 times in one run, and `self.blocks`'s length changes underneath it. The crashing stack is exactly that shape — `js_array_grow` → `arena_alloc_gc_old` → `Arena::alloc` → `gc_check_trigger` → full mark-sweep → `ValidPointerSetBuilder::step` → `RawVec::grow_one` → `_mi_theap_realloc_zero`, faulting on a corrupt mimalloc heap. `arena_cell_alloc(*mut Arena, ..)` is the new collecting entry point: current block under a borrow that ends with the statement, `gc_check_trigger()` with no arena borrow live, then a fresh borrow for the slow path. `Arena::alloc` is now collection-free, and the five thread-local entry points route through the new one. The two that also touch `INLINE_STATE` keep those borrows short for the same reason: `resync_inline_to_current` mutates `INLINE_STATE` from inside the collection. Same commit, same compiler, only the four arena files differing; `test_gap_repsel_gc_stress` under `PERRY_GC_HEAP_LIMIT=8 PERRY_GC_INCREMENTAL=0 PERRY_CONSERVATIVE_STACK_SCAN=off`, auto-optimized release binaries: 139 x 20 before / 0 x 20 after at the default 16 MB nursery cap, clean across a 13-cap sweep, and 5/5 -> 0/5 on the force-evacuate, force+verify and from-space-scan arms. Claude-Session: https://claude.ai/code/session_01G4k1vE6PVb53m2dRZ2aDtv
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughChangesThe arena allocation flow now separates collection-free bump allocation from GC-triggering allocation, invokes the trigger outside active Arena GC borrow safety
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Allocator
participant arena_cell_alloc
participant Arena
participant gc_check_trigger
Allocator->>arena_cell_alloc: request allocation
arena_cell_alloc->>Arena: try_alloc_current(size, align)
arena_cell_alloc->>gc_check_trigger: run allocation-point trigger
gc_check_trigger-->>arena_cell_alloc: GC completes
arena_cell_alloc->>Arena: alloc_after_gc(size, align)
arena_cell_alloc-->>Allocator: return allocated pointer
Possibly related issues
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/perry-runtime/src/arena/block.rs`:
- Around line 458-481: Restructure the fresh-block path around
Arena::alloc_after_gc and alloc_block so gc_try_emergency_reclaim runs without
an active ArenaBorrowGuard or &mut Arena borrow. Perform GC/emergency reclaim
and block allocation in a borrow-less stage, then reacquire the borrow only for
install_fresh_block and the final allocation; preserve the existing allocation
behavior and failure handling.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 281a69cb-a485-4958-b150-37e7303dcab2
📒 Files selected for processing (6)
changelog.d/7050-gc-allocation-point-trigger-outside-arena-borrow.mdcrates/perry-runtime/src/arena/allocators.rscrates/perry-runtime/src/arena/block.rscrates/perry-runtime/src/arena/inline.rscrates/perry-runtime/src/arena/mod.rscrates/perry-runtime/src/arena/tests.rs
| pub(crate) unsafe fn arena_cell_alloc(arena: *mut Arena, size: usize, align: usize) -> *mut u8 { | ||
| // Try current block first, under a borrow that ends with this statement. | ||
| { | ||
| let _borrow = ArenaBorrowGuard::new(); | ||
| if let Some(ptr) = (*arena).try_alloc_current(size, align) { | ||
| return ptr; | ||
| } | ||
| } | ||
|
|
||
| // Current block is full. Check the GC trigger first — if it fires and | ||
| // reclaims at least one fully-empty block (via `arena_reset_empty_blocks`), | ||
| // we may be able to reuse that block instead of pushing a new one. | ||
| // | ||
| // Threshold pressure is paid through bounded mutator-assist work. A | ||
| // completed assist cycle may reset blocks before we retry; an incomplete | ||
| // cycle leaves the debt active for later host or allocator steps. | ||
| // | ||
| // NO ARENA BORROW IS LIVE HERE. See the function docs. | ||
| note_gc_trigger_arena_borrow_depth(); | ||
| crate::gc::gc_check_trigger(); | ||
|
|
||
| let _borrow = ArenaBorrowGuard::new(); | ||
| (*arena).alloc_after_gc(size, align) | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Inspect gc_try_emergency_reclaim to check whether it can allocate into
# the arena currently being extended by install_fresh_block/alloc_block.
ast-grep outline crates/perry-runtime/src/gc.rs --items all --match 'gc_try_emergency_reclaim' 2>/dev/null \
|| rg -n -A 40 'fn gc_try_emergency_reclaim' crates/perry-runtime/src/**/*.rsRepository: PerryTS/perry
Length of output: 165
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== files =="
git ls-files 'crates/perry-runtime/src/*.rs' | sed -n '1,120p'
echo
echo "== locate allocator/gc block identifiers =="
rg -n 'fn (gc_try_emergency_reclaim|gc_check_trigger|install_fresh_block|alloc_fresh_block|alloc_block|alloc_after_gc|try_alloc_current)|gc_try_emergency_reclaim|alloc_after_gc|arena_cell_alloc' crates/perry-runtime/src || true
echo
echo "== block outline =="
ast-grep outline crates/perry-runtime/src/arena/block.rs --items all 2>/dev/null | sed -n '1,220p' || trueRepository: PerryTS/perry
Length of output: 10356
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== block.rs alloc_block and allocation path =="
sed -n '1,80p;280,380p;440,560p' crates/perry-runtime/src/arena/block.rs | cat -n
echo
echo "== gc/mod.rs emergency reclaim =="
sed -n '300,365p' crates/perry-runtime/src/gc/mod.rs | cat -n
echo
echo "== arena_alloc_gc_old / arena_alloc_gc_used arena allocation =="
rg -n -A 80 -B 5 'arena_alloc_gc_(old|used)|arena_alloc_gc|alloc_gc|gc_(root|mark|sweep|compact|try_emergency|check_trigger)|spawn_for_promise_deferred|arena_cell_gc_alloc' crates/perry-runtime/src/gc crates/perry-runtime/src/arena | sed -n '1,260p'Repository: PerryTS/perry
Length of output: 41383
Move the emergency-reclaim borrow guard outside alloc_block.
alloc_after_gc runs under ArenaBorrowGuard::new(), then calls alloc_fresh_block → alloc_block. When alloc(layout) fails, alloc_block calls gc_try_emergency_reclaim(), which can run a full collection and allocate into arenas. If that collection grows the same arena currently extended by install_fresh_block, self.blocks can change underneath the live &mut Arena borrow. Split the fresh-block allocation into the “borrow-less GC/alloc” + “borrow-only install/allocate” stages as with normal GC triggers.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/perry-runtime/src/arena/block.rs` around lines 458 - 481, Restructure
the fresh-block path around Arena::alloc_after_gc and alloc_block so
gc_try_emergency_reclaim runs without an active ArenaBorrowGuard or &mut Arena
borrow. Perform GC/emergency reclaim and block allocation in a borrow-less
stage, then reacquire the borrow only for install_fresh_block and the final
allocation; preserve the existing allocation behavior and failure handling.
* gc: reserve arena blocks with no arena borrow live (#7051) #7050 moved the allocation-point GC trigger out of Arena::alloc's &mut self borrow. The emergency-reclaim path kept the same shape: alloc_after_gc -> alloc_fresh_block -> install_fresh_block -> alloc_block, all under &mut self, and alloc_block calls gc_try_emergency_reclaim() when the OS refuses memory. That collection allocates into the arenas, so self.blocks.push(..) could grow the Vec underneath the live borrow -- the same aliasing violation, on the path that runs under heap exhaustion. Splits block acquisition in two: reserve_arena_block(min_size) -- may collect; NO arena borrow may be live across it. Used by arena_cell_alloc, which already runs borrow-free at that point. alloc_block_no_gc(min_size) -- never collects; for the callers that hold a live borrow, two of which are already executing inside a collection. Arena::install_reserved_block installs a block reserved by the former. The cfg(test) borrow-depth probe from #7050 now covers this path too, and try_alloc_block gains an injectable failure hook so a test can force the null-alloc branch that would otherwise need real heap exhaustion. Refs #7051, #7022. * changelog: fragment for #7052 --------- Co-authored-by: Ralph Küpper <ralph@skelpo.com>
Closes #7022. Closes #7023.
The defect
test_gap_repsel_gc_stressSIGSEGVs deterministically in the 12gc_repsel_matrix.sh --arms allarms carryingPERRY_CONSERVATIVE_STACK_SCAN=off— i.e. exactly the arms in which #7019's default-on evacuating young-gen scavenge is eligible at all (CopiedMinorEligibilityfalls back withConservativeStackotherwise). The fault lands inside mimalloc'srealloc, called from the collector's ownVecgrowth, so the malloc heap is already corrupt on arrival.Crashing stack (release +
PERRY_DEBUG_SYMBOLS=1, under lldb):Root cause
Arena::alloc(&mut self, ..)callscrate::gc::gc_check_trigger()from inside the&mut selfborrow.A collection allocates into the arenas: promotion and C4b evacuation call
arena_alloc_gc_old, an evacuating minor fills a survivor semispace, and either can reachArena::install_fresh_block→self.blocks.push(..)— on the sameArenathe allocating frame is holding. AVecgrowth there frees the buffer the outer frame goes on to index (self.blocks[idx],for i in 0..self.blocks.len()), and&mutcarriesnoalias, so the outer frame is equally entitled to have cachedblocks.ptr/lenacross the call.This is not hypothetical. Instrumenting
install_fresh_blockwith a "GC triggered fromArena::alloc" depth counter on the reproducer:204 re-entries in one run, on Old, Survivor0 and Survivor1, including pushes at
len == cap(the reallocating case).The hazard predates #7019 but was latent: before the moving minor, a collection triggered from an allocation point did not itself install arena blocks anywhere near this often. That is why this is a PASS → FAIL correctly attributed to #7019 without #7019 containing the defect. It also explains the discriminator: the copying minor is only eligible when the conservative stack scan is off, so only those 12 arms reach the re-entrant shape at volume.
The fix
There are two places a collection can be entered from an arena allocation, and both had the borrow live across it.
1. The allocation-point trigger.
arena_cell_alloc(*mut Arena, size, align)is the new collecting entry point:gc_check_trigger()with no arena borrow live;2. The out-of-memory path (CodeRabbit 🟠 Major on the first cut — correct, and it is the same violation).
alloc_blockcalledgc_try_emergency_reclaim()on a nullalloc(layout), reached viaalloc_after_gc→alloc_fresh_block→install_fresh_block→alloc_block, all under&mut self. That emergency collection allocates into the arenas exactly like the trigger does. Block reservation is now split out:try_alloc_block— one raw malloc, never collects;reserve_arena_block— try, then one emergency reclaim, then retry. Called only fromarena_cell_alloc, with no arena borrow live, which then re-acquires a borrow purely toinstall_reserved_blockand bump;alloc_block_no_gc— used by the remaining&mut selfpaths (Arena::alloc, the C4b evacuation path'salloc_excluding_pages,arena_start_fresh_general_block,ArenaBlock::new). These deliberately do not reclaim: two of them already run inside a collection, where starting another is precisely what must not happen, and the mutator path — where heap exhaustion actually surfaces — keeps it.Arena::alloc(&mut self, ..)is now collection-free on every path, not just the common one. The five thread-local entry points —arena_alloc,arena_alloc_longlived,arena_alloc_old,arena_alloc_gc_survivor,js_inline_arena_slow_alloc— route througharena_cell_alloc. The two that also touchINLINE_STATEkeep those borrows short for the same reason:Arena::resync_inline_to_currentmutatesINLINE_STATEfrom inside the collection.No behavioural change to when the GC runs: the trigger is still at the same point (current block full), on the same arenas. One deliberate behaviour change on the OOM path:
reserve_arena_blocknow retries the allocation even whengc_try_emergency_reclaim()returnsfalse(it self-suppresses when already collecting), where the old code panicked without retrying. Strictly more forgiving.Verification
Same commit, same compiler; the two arms differ only in the four arena files. Release build, auto-optimized binaries — that turned out to matter: with
PERRY_NO_AUTO_OPTIMIZE=1the crash does not reproduce at all, which is presumably why the harness deliberately does not set it.macOS arm64, node 26.5.0 (
.node-version),PERRY_GC_HEAP_LIMIT=8 PERRY_GC_INCREMENTAL=0 PERRY_CONSERVATIVE_STACK_SCAN=off.PERRY_GC_SCAVENGE_NURSERY_MBOther evacuating arms on top of the same base, 5 runs each (failure = non-zero exit or output mismatch vs the node oracle):
PERRY_GC_FORCE_EVACUATE=1+ PERRY_GC_VERIFY_EVACUATION=1PERRY_GC_FROMSPACE_SCAN=1Matrix
scripts/gc_repsel_matrix.sh --arms all --pressure 8, 440 cells:+22 PASS / -22 FAIL, no cell regressed.
repsel_gc_stressis PASS in all 20 arms (was FAIL in 12). So isrepsel_scalar_replaced_locals— #7023 is the same defect and closes with this (it was FAIL in 11 of 20, intermittently, on the samePERRY_CONSERVATIVE_STACK_SCAN=offdiscriminator). The residual 24 FAILs are the two #6981p4a3numarray rows, arm-for-arm unchanged.Liveness is intact, so the green is not inertness:
evac_minor/force_evac/force_verify/ the sixrep_*_offarms all reportcollected 21/22 moved-objects 21/22 copy-minor 21/22, and byte-exact-vs-node is 415/440 cells.cargo test --release -p perry-runtime --lib -- --test-threads=1: 1522 passed, 0 failed. (Run in parallel, a rotating 3–5 of the known macOS parallel-flake family go red —gc::tests::teardown::map_set_*,object::global_this_webassembly,object::tests::closure_name_and_length_*— a different subset per run, all green single-threaded and unrelated to this change.)cargo fmt --all -- --checkclean.Regression coverage
Three
cargo-test-visible unit tests (per #5960, integration suites don't run per-PR):arena::tests::allocation_point_gc_trigger_runs_with_no_live_arena_borrow— forces the old arena past its current block througharena_cell_allocand asserts the arena-borrow depth observed at the GC trigger is 0 (and that the trigger was actually reached, so the test cannot pass vacuously).arena::tests::raw_arena_alloc_method_never_reaches_the_gc_trigger— pins thatArena::alloc(&mut self, ..)stays collection-free, so a future refactor cannot move the trigger back under the borrow.arena::tests::emergency_block_reclaim_runs_with_no_live_arena_borrow— the probe does reach the out-of-memory path. It needs a nullalloc, which a test cannot get from the OS on demand, so there is acfg(test)one-shot injected block-allocation failure (force_next_block_alloc_failure). It is consumed only byreserve_arena_block's first attempt — the collection that reclaim then runs allocates its own blocks through the non-injectable path, so the injected refusal cannot be eaten by the wrong allocation. The test asserts the reclaim was actually reached (gc_trigger_arena_calls() >= 2, i.e. both triggers) before asserting depth 0, so it cannot pass vacuously.The borrow-depth probe and the failure injection are
cfg(test)-only; production allocation pays nothing. Between the three tests, every site from which a collection can be entered during an arena allocation is now asserted to run with no borrow live — not just the two that were found.Teeth verified by sabotage, each in its own direction. Putting the trigger back inside
Arena::allocand holding the guard across it inarena_cell_alloc:Moving the reservation back under the borrow (
install_fresh_blockcallingreserve_arena_block,arena_cell_allocfalling back toalloc_after_gc):What this says about #7041's classification — please read before reusing it
#7022's dossier classified the defect as "a missing rewrite of an old→young remembered-set edge", from
PERRY_GC_FROMSPACE_SCAN'slost_dirty/dirty_but_missedsplit. That classification was measuring noise. Measured on the reproducer by extending the scan with two extra axes — is the offender's owner itselfGC_FLAG_FORWARDED, and is the offending slot inside the collector's own rewrite enumeration for that owner (visit_gc_rewrite_slots):enumerated=0on every cycle of every run. Every reported offender is a word no collector surface ever treats as a reference:GC_FLAG_FORWARDEDrecord — 2 048 → ~70 000 words per cycle, i.e. 99%+ of the total. These arejs_array_grow's permanent growth stubs (Array.push from inside an async function silently caps at 16 elements when the array is a function parameter #233). A forwarded header's payload is dead storage by construction:visit_gc_rewrite_slot_descriptors,gc_child_slots,scan_dirty_header_onceandtrace_one_worklist_headerall return early on it, and only word 0 (the forwarding address) is live.stub_forwarding_words=0throughout — no forwarding word ever pointed into from-space. Their pages stay dirty forever, which is precisely what produced thedirty_but_missedand then thelost_dirtynumbers.move_youngcopies the whole payload, so e.g. alen=1 cap=16array carries 15 slots of the previous occupant's bytes into to-space, complete with plausibleGcHeaders and forwarding addresses. Dumping an offending owner's payload shows exactly that.decode_root_wordaccepts any 48-bit value; the copying collector's ownCopyingPointerSet::decode_bitsadditionally requires 8-byte alignment, so these can never be relocated and can never be stale.Consequence for the acceptance criterion "
PERRY_GC_FROMSPACE_SCAN→ zero offenders": it is not attainable and never was, and the offender counts are byte-for-byte comparable before and after this fix (the fixed run simply produces 121 scan reports instead of 19 truncated ones). What is zero, on every cycle, before and after, is the population that would be an actual collector defect: live owner, enumerated slot. That measurement is what ruled the missing-rewrite hypothesis out and sent the investigation to the allocator.The scan change that produces the split above is deliberately not in this PR — it would invalidate the matrix run below, and a measurement-tool change does not belong in the same diff as a collector fix. Follow-up filed against #7041/#7035.
Also killed, for the record
Beyond the five hypotheses already in #7022's dossier, this investigation additionally eliminated:
stub_forwarding_words=0across every cycle; the grown array exceedsLARGE_OBJECT_THRESHOLD_BYTESand is born in old-gen, so the stub's target never sits in from-space;enumerated=0, see above;move_young'sMAX_YOUNG_MOVE_BYTESguard silently refusing a >1 MiB young object — unreachable here,arena_alloc_gcroutes anything over 16 KB straight to old-gen.Re-verification after the emergency-path fix
The emergency-path change touches the same allocator, so the sweep was re-run on the final build rather than assumed. The full matrix was not re-run — the change only affects a path that requires
alloc(layout)to return null, which no matrix cell reaches, and the two arms it could plausibly perturb (the fresh-block reservation ordering) are covered by the sweep below.force_evac/force+verify/fromspace_scan× 5repsel_scalar_replaced_locals(#7023) × 10,force_evaccargo test … --test-threads=1cargo fmt --all --checkNotes
lintis red repo-wide for unrelated reasons (stale public benchmark baseline).