Skip to content

fix: bump the nightly past a codegen ICE that broke the Linux release build - #984

Merged
andrewgazelka merged 4 commits into
mainfrom
fix/linux-release-ice
Jul 27, 2026
Merged

fix: bump the nightly past a codegen ICE that broke the Linux release build#984
andrewgazelka merged 4 commits into
mainfrom
fix/linux-release-ice

Conversation

@andrewgazelka

Copy link
Copy Markdown
Member

nix build .#packages.x86_64-linux.bedwars died in the compiler:

error: internal compiler error: compiler/rustc_codegen_ssa/src/mir/operand.rs:291:18:
  not immediate: OperandRef(Uninit @ TyAndLayout {
    ty: unsafe_cell::UnsafeCell<MaybeUninit<runtime::task::Notified<Arc<multi_thread::handle::Handle>>>>,
    ... backend_repr: Scalar(union pointer) ... })

What the differentials said

The upstream issue carries an 8-line reproduction, which turns a 40-minute nix build into a sub-second rustc call, so all of these ran against that rather than the workspace.

  • Optimisation level. ICEs at 2 and 3, clean at 0 and 1. So Tests (ubuntu-latest) being green proves nothing: it is a debug build and cargoUnit compiles at -C opt-level=3.
  • Architecture. Not x86. It ICEs identically on aarch64-apple-darwin, same assertion, same line. The bug is in target-independent codegen_return_terminator.
  • Nightly. Clean on 07-23, ICE on 07-24 through 07-26, clean again on 07-27.
  • A newer tokio. Does not exist. 1.53.1 is the newest published.

The fix

Bump the pin forward one day, to the nightly that carries rust-lang/rust#159825, which fixes #159815. Root cause upstream: #157797 added OperandValue::Uninit and missed the PassMode::Direct | PassMode::Pair branch.

Forward rather than back, deliberately. Rolling back to 07-23 also works, but it is a knob someone has to remember to remove and it strands the workspace three days behind. Bumping forward becomes an ordinary pin the moment the fix reaches beta.

Rejected: opt-level=1 for tokio only, which permanently deoptimises the async runtime of a game server to work around a compiler bug that is already fixed.

Verified

The whole workspace moves with a toolchain pin, so every gate was re-run.

cargo check --workspace --all-targets   rc=0
nix run .#lint                          rc=0
nix run .#test                          rc=0   447 passed, 1 skipped
nix run .#e2e -- --seconds 12           rc=0   3729 LevelChunkWithLight
nix build x86_64-linux bedwars + proxy  rc=0
/nix/store/rabbfr795mbs20n0qwdmjvlfa3ibprv3-bedwars-0.1.0
/nix/store/qp3ykmwsy35df2hrblaaz77frs6xbjyb-hyperion-proxy-0.1.0

Zero occurrences of internal compiler error in the full build log.

(sent by an AI agent via Claude Code, model claude-opus-4-6)

nightly-2026-07-24 through -07-26 carry rust-lang/rust#159815, an ICE in
rustc_codegen_ssa when a function returns an all-uninit MaybeUninit. tokio
1.53.1 hits it in the multi-thread runtime, so `nix build
.#packages.x86_64-linux.bedwars` could not produce a release binary at all,
and nothing could be deployed.

Differentials, on the minimal repro from the upstream issue:

  opt-level 0, 1        OK
  opt-level 2, 3        ICE
  aarch64-apple-darwin  ICE, identically

so it is not x86 codegen and not something a build flag avoids without
deoptimising tokio. Bumping tokio is not available either: 1.53.1 is the
newest published version.

Walking the nightlies gives a three-day window:

  nightly-2026-07-23  rustc 6f72b5dd5  OK
  nightly-2026-07-24  rustc 89c61a754  ICE
  nightly-2026-07-25  rustc da86f4d07  ICE
  nightly-2026-07-26  rustc 008fa22ce  ICE   <- the old pin
  nightly-2026-07-27  rustc dc3f85158  OK    <- the fix landed

rust-lang/rust#159825 merged 2026-07-26T06:29:12Z, so -07-27 is the first
nightly carrying it. That makes this a one-day move forward rather than a
three-day rollback, and it leaves no knob behind: when the fix reaches beta
the pin is just a normal pin again.

Upstream issue: rust-lang/rust#159815
Upstream fix:   rust-lang/rust#159825
@github-actions github-actions Bot added the fix label Jul 27, 2026
@github-actions

Copy link
Copy Markdown

Benchmark Results for general

ray_intersection/aabb_size_0.1                     [  17.3 ns ...  17.2 ns ]      -0.72%*
ray_intersection/aabb_size_1                       [  17.3 ns ...  17.1 ns ]      -1.01%*
ray_intersection/aabb_size_10                      [  17.3 ns ...  17.2 ns ]      -0.57%*
ray_intersection/ray_distance_1                    [   1.5 ns ...   1.5 ns ]      +0.13%
ray_intersection/ray_distance_5                    [   1.5 ns ...   1.5 ns ]      -0.06%
ray_intersection/ray_distance_20                   [   1.5 ns ...   1.5 ns ]      -0.26%
overlap/no_overlap                                 [  15.9 ns ...  15.9 ns ]      +0.36%
overlap/partial_overlap                            [  15.9 ns ...  15.9 ns ]      -0.04%
overlap/full_containment                           [  14.9 ns ...  14.9 ns ]      +0.03%
point_containment/inside                           [   6.0 ns ...   6.0 ns ]      +0.19%
point_containment/outside                          [   6.0 ns ...   6.1 ns ]      +0.62%
point_containment/boundary                         [   6.1 ns ...   6.1 ns ]      -0.19%

Comparing to 8d9680c

The coverage job failed on a test that passes under nextest:

  a_passive_owned_by_the_kit_module_sees_the_damage_pipeline
  assertion left == right failed: the passive should only arm against non-melee

The counter the Porcupine passive incremented was a process-global static.
Nine tests in that file build a Porcupine world, cargo test runs them as
threads in one process, and any two dealing non-melee damage at the same
moment shared one number. nextest gives every test its own process, so
nix run .#test never saw it and never will.

It is a flecs singleton now, so it belongs to the world that observed the
damage. Registered explicitly because this workspace builds flecs with
manual registration, where first use of an unregistered component aborts.

Five consecutive cargo test --all-features --workspace runs pass.
@github-actions

Copy link
Copy Markdown

Benchmark Results for general

ray_intersection/aabb_size_0.1                     [  18.8 ns ...  18.8 ns ]      -0.32%
ray_intersection/aabb_size_1                       [  18.7 ns ...  18.8 ns ]      +0.09%
ray_intersection/aabb_size_10                      [  18.8 ns ...  18.8 ns ]      -0.03%
ray_intersection/ray_distance_1                    [   1.3 ns ...   1.3 ns ]      -0.32%
ray_intersection/ray_distance_5                    [   1.3 ns ...   1.3 ns ]      -0.04%
ray_intersection/ray_distance_20                   [   1.3 ns ...   1.3 ns ]      -0.13%
overlap/no_overlap                                 [  15.9 ns ...  15.8 ns ]      -1.02%*
overlap/partial_overlap                            [  16.2 ns ...  16.0 ns ]      -1.09%*
overlap/full_containment                           [  14.4 ns ...  14.5 ns ]      +0.48%
point_containment/inside                           [   5.7 ns ...   5.4 ns ]      -5.42%*
point_containment/outside                          [   5.4 ns ...   5.7 ns ]      +5.84%*
point_containment/boundary                         [   5.4 ns ...   5.4 ns ]      +0.36%

Comparing to d8cd11d

@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 40.91%. Comparing base (8d9680c) to head (9990b05).
⚠️ Report is 1 commits behind head on main.

@@            Coverage Diff             @@
##             main     #984      +/-   ##
==========================================
- Coverage   41.01%   40.91%   -0.11%     
==========================================
  Files         226      239      +13     
  Lines       24181    25697    +1516     
  Branches      957     1004      +47     
==========================================
+ Hits         9917    10513     +596     
- Misses      14049    14966     +917     
- Partials      215      218       +3     

see 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@andrewgazelka
andrewgazelka merged commit 8cbc65f into main Jul 27, 2026
10 of 11 checks passed
@andrewgazelka
andrewgazelka deleted the fix/linux-release-ice branch July 27, 2026 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ICE]: thread 'rustc' (2584) panicked at ... compiler/rustc_codegen_ssa/src/mir/operand.rs:291:18:

1 participant