fix(bench): point the linker at the C runtime, and surface the error a tail buries

Two things the last matrix taught, one fixed and one made diagnosable.

1. cmake/linux/gcc still died in its own compiler probe with `cannot find
   crt1.o`. My previous attempt — only pass `--sysroot` when crt1.o is under it
   — was correct and changed nothing, and the CI log says why: neither
   `--sysroot=` nor the STATUS line the other branch would have printed appears,
   so `registry/subos/default` does not exist on the runner at all. The payload
   gcc then fell back to a built-in prefix that is not there either.

   The stable anchor is the PACKAGE: `subos/default/lib/crt1.o` is a symlink
   into `xpkgs/xim-x-glibc/<ver>/lib/`, and xpkgs is where the compiler itself
   was found, so it exists by construction — `xim-x-glibc` is present in the
   failing job's own log. `--sysroot` is gone entirely; `-B` and `-L` ADD to the
   search instead of replacing it, which is the property this needed from the
   start.

   HONEST LIMIT: I could not reproduce the runner's failure locally. A fake
   MCPP_HOME with no subos reproduces the missing directory, but this machine's
   payload gcc finds crt1.o by itself, so the pre-fix code configures there too.
   What is verified is that the arm still configures with and without a subos,
   and that the directory the fix names exists on the runner. Whether that is
   sufficient, CI will say.

2. `xmake/clang` failed with `seed build exited 255` and the captured tail was
   twenty lines of `generating.module.deps`. A tail is the wrong shape for a
   tool that prints a line per translation unit: the cause is printed once,
   hundreds of lines earlier. That cell cost a full matrix cycle and taught
   nothing.

   Failures now report the lines that LOOK like a cause, pulled from anywhere in
   the log, with the tail after them as context. A keyword sieve rather than
   per-engine parsing — approximately right for four different diagnostic
   formats beats exactly right until one changes its wording. Verified against a
   fake engine that prints an error and then 300 progress lines: the error is
   surfaced, where before only progress was.

Author: speak-agent

bench/projects/common/cmake/hermetic_payload.cmake

@@ -259,40 +259,55 @@ function(bench_hermetic_payload_preproject)
   if(_binutils)
     string(APPEND _add " -B${_binutils}/bin")
   endif()
-  # ⚠️ THE TEST IS "DOES IT HOLD A C RUNTIME", NOT "DOES THE DIRECTORY EXIST".
+  # ── The C runtime: FIND crt1.o, then ADD its directory. No --sysroot. ──────
   #
-  # `--sysroot` does not ADD a search path, it REPLACES gcc's default one. Point
-  # it at a directory that exists but has no libc and every link dies at the
-  # first object:
+  # ⚠️ `--sysroot` does not ADD a search path, it REPLACES gcc's default one, so
+  # pointing it anywhere that lacks a libc removes the C runtime entirely:
   #
   #     ld: cannot find crt1.o: No such file or directory
   #     ld: cannot find crti.o: No such file or directory
   #     ld: cannot find -lm: No such file or directory
   #
-  # which is a message about the C runtime and says nothing about the flag that
-  # caused it. `IS_DIRECTORY` passed on the runners because mcpp creates
-  # `registry/subos/default` whether or not anything has been installed into it,
-  # so all five cmake cells of the linux/gcc bench failed at cmake's own compiler
-  # probe — before a single line of the project was configured.
+  # — a message about the C runtime that names neither the flag nor the cause.
+  # All five cmake cells of the linux/gcc bench died there, inside cmake's own
+  # compiler probe, before a line of the project was configured.
   #
-  # This repository has been bitten by the same flag before (an `install()`
-  # source package lost its libc headers exactly this way). Existence was the
-  # wrong predicate then too.
-  set(_crt "")
-  foreach(_d lib lib64 usr/lib usr/lib64 usr/lib/x86_64-linux-gnu)
-    if(EXISTS "${_sysroot}/${_d}/crt1.o")
-      set(_crt "${_sysroot}/${_d}/crt1.o")
-      break()
+  # TWO WRONG ANSWERS PRECEDED THIS ONE, and both looked right:
+  #   * `IS_DIRECTORY "${_sysroot}"` — true on the runners, because that path is
+  #     created whether or not anything was installed into it.
+  #   * then "only pass it when crt1.o is under the sysroot" — correct as far as
+  #     it went, and it changed nothing: on the runner that directory does not
+  #     exist at all, so neither branch ran, no --sysroot was passed, and the
+  #     payload gcc fell back to a built-in prefix that is not there either.
+  #     Diagnosed from the CI log by the ABSENCE of both `--sysroot=` on the
+  #     command line and the STATUS line the second branch would have printed.
+  #
+  # The stable anchor is the PACKAGE. `subos/default/lib/crt1.o` is a symlink
+  # into `xpkgs/xim-x-glibc/<ver>/lib/`, and xpkgs is where the compiler itself
+  # was found, so it exists by construction. `-B` (startup files) and `-L`
+  # (`-lm`) ADD to the search rather than replacing it, which is the property
+  # this needed all along.
+  set(_crtdir "")
+  bench_newest_package("${_xpkgs}" "xim-x-glibc" _glibc)
+  foreach(_root "${_sysroot}" "${_glibc}")
+    if(_crtdir OR NOT _root)
+      continue()
     endif()
+    foreach(_d lib lib64 usr/lib usr/lib64 usr/lib/x86_64-linux-gnu)
+      if(EXISTS "${_root}/${_d}/crt1.o")
+        set(_crtdir "${_root}/${_d}")
+        break()
+      endif()
+    endforeach()
   endforeach()
-  if(_crt)
-    string(APPEND _add " --sysroot=${_sysroot}")
-  elseif(IS_DIRECTORY "${_sysroot}")
-    # Say so. A payload build that silently falls back to the host's libc is a
-    # different measurement from the one this file claims to set up, and the
-    # only way to notice is if it announces itself.
-    message(STATUS "bench: ${_sysroot} has no crt1.o — NOT passing --sysroot; "
-                   "this arm links against the host C runtime")
+  if(_crtdir)
+    string(APPEND _add " -B${_crtdir} -L${_crtdir}")
+  else()
+    # Say so. Falling back to the host's C runtime is a different measurement
+    # from the one this file claims to set up, and the only way anyone notices
+    # is if it announces itself.
+    message(STATUS "bench: no crt1.o under ${_sysroot} or ${_glibc} — this arm "
+                   "links against the host C runtime")
   endif()
   if(_add STREQUAL "")
     return()

bench/src/platform.cppm

@@ -107,6 +107,40 @@ inline bool log_mentions(const std::filesystem::path& p,
     return false;
 }
 
+// The lines anywhere in `p` that look like a cause, not a progress report.
+//
+// A tail cannot answer "why did this fail" for a tool that prints a line per
+// translation unit: the error scrolled past hundreds of lines ago and the last
+// 20 are all `[ 2%]: generating.module.deps ...`. That is exactly how an
+// `xmake exited 255` cell reached CI with nothing to diagnose it by.
+//
+// Deliberately a keyword sieve rather than per-engine parsing: every engine
+// here is a different program with a different diagnostic format, and one that
+// is merely APPROXIMATELY right on all of them beats four that are exactly
+// right until a tool changes its wording. False positives cost a line of noise;
+// a false negative costs a matrix cycle.
+inline std::string log_grep(const std::filesystem::path& p,
+                            std::initializer_list<std::string_view> markers,
+                            std::size_t max = 12) {
+    std::ifstream in(p, std::ios::binary);
+    if (!in) return {};
+    std::string out, line;
+    std::size_t kept = 0;
+    while (kept < max && std::getline(in, line)) {
+        if (!line.empty() && line.back() == '\r') line.pop_back();
+        bool hit = false;
+        for (const auto m : markers)
+            if (line.find(m) != std::string::npos) { hit = true; break; }
+        if (!hit) continue;
+        // Long lines here are usually a whole compiler command line; the cause
+        // is at the front of them.
+        if (line.size() > 400) { line.resize(400); line += " …"; }
+        out += "    "; out += line; out += '\n';
+        ++kept;
+    }
+    return out;
+}
+
 inline std::string tail_of(const std::filesystem::path& p, std::size_t lines = 20) {
     std::ifstream in(p, std::ios::binary);
     if (!in) return {};

bench/src/runner.cppm

@@ -293,6 +293,29 @@ public:
             // that crash is still undiagnosed.
             const auto crashed = platform::log_mentions(
                 job.log_path, {"PLEASE submit a bug report", "Stack dump"});
+
+            // ⚠️ A TAIL IS THE WRONG SHAPE WHEN THE TOOL IS CHATTY. Every build
+            // engine here prints a progress line per translation unit, so 20
+            // lines of tail is 20 lines of `generating.module.deps ...` and the
+            // error that actually stopped it — printed once, hundreds of lines
+            // earlier — is gone. That is not hypothetical: `xmake/clang` failed
+            // with `seed build exited 255` and the captured tail contained
+            // nothing but progress, so the cell could not be diagnosed from CI
+            // at all and cost a full matrix cycle to learn nothing.
+            //
+            // So the lines that LOOK like an error are pulled out first, from
+            // anywhere in the file, and the tail follows as context. Cheap, and
+            // it is the difference between "exited 255" and a cause.
+            if (const auto why = platform::log_grep(
+                    job.log_path,
+                    {"error:", "error :", "ERROR:", " error ", "fatal",
+                     "not found", "No such file", "cannot find", "undefined",
+                     "failed to", "Assertion", "abort"},
+                    /*max=*/12);
+                !why.empty())
+                report(std::format("--- error lines from {} ---\n{}",
+                                   job.log_path.filename().string(), why));
+
             if (const auto tail = platform::tail_of(job.log_path, crashed ? 80 : 20);
                 !tail.empty())
                 report(std::format("--- last lines of {} ---\n{}",