Skip to content

bazel: drop coverage workarounds, bump llvm to 0.8.14 - #10986

Merged
eder-matheus merged 2 commits into
The-OpenROAD-Project:masterfrom
The-OpenROAD-Project-staging:bazel-coverage-hermetic-llvm-0814
Jul 27, 2026
Merged

bazel: drop coverage workarounds, bump llvm to 0.8.14#10986
eder-matheus merged 2 commits into
The-OpenROAD-Project:masterfrom
The-OpenROAD-Project-staging:bazel-coverage-hermetic-llvm-0814

Conversation

@openroad-ci

Copy link
Copy Markdown
Member

What

Bumps hermetic-llvm (llvm) 0.8.11 → 0.8.14 and drops most of the
--config=coverage workaround that came in with #3608.

.bazelrc loses 5 of its 8 coverage lines:

-coverage:coverage --copt=-fprofile-instr-generate
-coverage:coverage --copt=-fcoverage-mapping
-coverage:coverage --linkopt=-fprofile-instr-generate
-coverage:coverage --strategy=TestRunner=local
-coverage:coverage --test_env=LLVM_PROFDATA=external/llvm++llvm_toolchain_minimal+llvm-toolchain-minimal-22.1.8-linux-amd64/bin/llvm-profdata
-coverage:coverage --test_env=LLVM_COV=external/llvm++llvm_toolchain_minimal+llvm-toolchain-minimal-22.1.8-linux-amd64/bin/llvm-cov
+coverage:coverage --noexperimental_fetch_all_coverage_outputs

test/downstream/MODULE.bazel and the docs/user/Bazel.md snippet pin the same
toolchain for downstream consumers; they move to 0.8.14 too so the repo shows
one version everywhere.

Why

#10913 landed bazel coverage support against a toolchain that had no coverage
support, so the config had to instrument by hand, point the collector at the
LLVM tools via --test_env, and run test actions unsandboxed. Two consequences
we called out at the time:

  • the --test_env paths embed the LLVM release
    (llvm-toolchain-minimal-22.1.8), so every llvm bump silently breaks the
    nightly coverage job;
  • test actions leave the sandbox, weakening hermeticity for exactly the CI job
    we care about.

We filed hermeticbuild/hermetic-llvm#675 with the analysis, and it was fixed by
hermeticbuild/hermetic-llvm#677 (merged upstream, first released in v0.8.14):

  1. the cc_toolchain's coverage features are enabled — coverage_stub,
    llvm_coverage_map_format, gcc_coverage_map_format were commented out
    behind a TODO: Re-enable once coverage is supported, so bazel coverage
    applied zero instrumentation flags and produced a structurally empty report
    (SF: entries, no DA: lines) with no error — a silent failure;
  2. bin/gcov, bin/llvm-cov and bin/llvm-profdata are now declared as
    toolchain files, so they are staged into the test sandbox (previously:
    GCov does not exist at the given path) and Bazel exports
    LLVM_COV / LLVM_PROFDATA from the tool paths for rules_cc's
    collect_cc_coverage.sh;
  3. upstream also added a bazel coverage lcov e2e test to their CI on Linux and
    macOS, so this path is now covered against regressions.

Instrumentation, tool staging and the env vars therefore all come from the
toolchain, and the hand-rolled flags plus the version-coupled paths go away.
The remaining two --experimental_* flags are not workarounds: they select
clang source-based coverage instead of gcov, which is the only viable path here
(gcov-style -fprofile-arcs objects fail to link, ld.lld: error: .eh_frame: relocation is not in any piece, as of LLVM 22.1.8).

The one new flag

--noexperimental_fetch_all_coverage_outputs is a rules_cc bug, not a
toolchain one: collect_cc_coverage.sh's init_gcov creates a gcov symlink
in each test's raw _coverage tree and only the gcov branch removes it, so in
llvm mode the tree keeps a dangling link and Bazel's output validation rejects
it:

ERROR: error while validating output tree artifact
  src/utl/test/TestAlgorithms/_coverage: child gcov is a dangling symbolic link

We only consume the merged lcov report, so the raw per-test outputs are skipped.
bazelbuild/rules_cc#795 fixes it upstream but is still open (latest release is
0.2.22); the flag can be dropped once rules_cc >= 0.2.23 is available.

Risk

0.8.14 still pins LLVM 22.1.8, identical to 0.8.11 — this is a module-level
bump, not a compiler bump. MODULE.bazel.lock churn is 4 lines. Nothing in the
repo referenced the sonameset_soname feature rename that also landed in
this range.

Verification

  • bazelisk coverage --config=coverage -- //src/utl/test:cpp_tests: 4/4 tests
    pass, sandboxed (processwrapper-sandbox, no --strategy override).
    Combined report: 1464 DA: lines across 57 files, 580 covered / 884
    uncovered, zero non-src/ entries.
  • genhtml renders it: 39.6% lines, 44.2% functions — identical to the numbers
    recorded in #10913, so the CI report contract is unchanged.
  • bazelisk build //:openroad succeeds on 0.8.14 (clean, 6936 actions).
  • Confirmed --noexperimental_fetch_all_coverage_outputs is still required:
    re-running with --experimental_fetch_all_coverage_outputs reproduces the
    dangling-symlink validation error above.
  • test/downstream could only be verified through module resolution
    (bazelisk mod graph in test/downstream resolves llvm@0.8.14). Analysis
    in that module currently fails for an unrelated, pre-existing reason —
    rules_pycross rejects the module graph with Unknown Python version: 3.8,
    before any target is configured. Reproduced on a pristine master worktree
    at 0.8.11, so it is not caused by this PR; worth a separate look.

Follow-up (not in this PR)

Drop --noexperimental_fetch_all_coverage_outputs when rules_cc >= 0.2.23
lands.

hermetic-llvm 0.8.14 enables the cc_toolchain coverage features and declares
gcov / llvm-cov / llvm-profdata as toolchain files (hermeticbuild/hermetic-llvm
The-OpenROAD-Project#675, fixed by The-OpenROAD-Project#677), so instrumentation, tool staging into the test sandbox
and LLVM_COV/LLVM_PROFDATA all come from the toolchain.

Remove the hand-rolled instrumentation copts/linkopts, the two --test_env
paths that hardcoded the LLVM release (they broke on every module bump) and
--strategy=TestRunner=local: coverage tests run sandboxed again.

Add --noexperimental_fetch_all_coverage_outputs to work around a rules_cc bug
(dangling `gcov` symlink in each test's raw _coverage tree in llvm mode);
drop it once rules_cc >= 0.2.23 (bazelbuild/rules_cc#795) is released.

0.8.14 still pins LLVM 22.1.8, so the compiler is unchanged.

Verified: `bazelisk coverage --config=coverage -- //src/utl/test:cpp_tests`
passes 4/4 sandboxed, combined lcov report has 1464 DA: lines across 57 files
(all under src/), genhtml renders 39.6% lines / 44.2% functions -- identical
to the numbers from the original coverage config. `bazelisk build //:openroad`
succeeds.

Signed-off-by: SombraSoft <sombrio@sombrasoft.dev>
Keep the toolchain pin consistent across the repo: the downstream-consumer
test module and the equivalent snippet in docs/user/Bazel.md tracked the
version we bumped in the previous commit.

Note that test/downstream currently fails analysis regardless of the llvm
version -- rules_pycross rejects the module graph with "Unknown Python
version: 3.8" -- so this bump is verified only through module resolution
(`bazelisk mod graph` shows llvm@0.8.14). The pycross failure reproduces
unchanged on master at 0.8.11 and is tracked separately.

Signed-off-by: SombraSoft <sombrio@sombrasoft.dev>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request upgrades the llvm Bazel dependency from version 0.8.11 to 0.8.14 across the workspace, downstream tests, and documentation. It also simplifies the .bazelrc coverage configuration by removing manual instrumentation flags, local test runner strategies, and hardcoded tool paths, as these are now natively supported by the updated toolchain. Additionally, it introduces the --noexperimental_fetch_all_coverage_outputs flag to bypass a known issue with dangling symlinks in rules_cc. There are no review comments, and I have no additional feedback to provide.

@sombraSoft
sombraSoft marked this pull request as ready for review July 25, 2026 22:42
@sombraSoft
sombraSoft requested a review from a team as a code owner July 25, 2026 22:42
@sombraSoft
sombraSoft requested a review from eder-matheus July 25, 2026 22:42
@eder-matheus
eder-matheus merged commit f8a7673 into The-OpenROAD-Project:master Jul 27, 2026
17 checks passed
@eder-matheus
eder-matheus deleted the bazel-coverage-hermetic-llvm-0814 branch July 27, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants