Skip to content

Build against NumPy 2.x and harden the meson/CI build#118

Open
brmather wants to merge 9 commits into
masterfrom
numpy2-build-robustness
Open

Build against NumPy 2.x and harden the meson/CI build#118
brmather wants to merge 9 commits into
masterfrom
numpy2-build-robustness

Conversation

@brmather

@brmather brmather commented Jul 23, 2026

Copy link
Copy Markdown
Member

Summary

PyPI's latest wheels (2.3.3) are compiled against NumPy 1.x and crash at import under NumPy 2.x (A module that was compiled using NumPy 1.x cannot be run in NumPy 2.0). This PR builds stripy against NumPy 2.x — which produces binaries that run on both 1.x and 2.x — and fixes the chain of related blockers that were preventing a working 2.3.4 wheel from ever reaching PyPI, plus several build/CI robustness issues.

Addresses #114 and #115 · Closes #117 · Refs #116

What's in here

NumPy 2.x (the core fix)

  • pyproject.toml: build-requires numpy>=2.0; runtime relaxed to numpy>=1.19.3 (the 2.0 back-compat floor) so 1.x environments still work; added py3.13; dropped the dead [tool.setuptools] block.
  • Replaced NumPy aliases removed in 1.24 (np.int, np.bool) that crashed at runtime under NumPy 2.x.
  • Removed the obsolete numpy.distutils setup.py/setup.cfg (dead under the meson-python backend) and the now-broken in-repo conda/ recipe (superseded by the conda-forge feedstock).

Release/CI pipeline

  • pypi.yml: fixed the actions/upload-artifact@v4 duplicate-name failure — all matrix jobs uploaded under the same name, which fails the run; now unique per-job names + download-artifact … pattern/merge-multiple. Bumped actions, cibuildwheel → 2.21.3 (cp313), added scipy to the wheel test deps. macos-14 (arm64) was already added, so releases now ship Apple-silicon wheels.
  • CI.yml: fixed a latent bug where pytest ran from the repo root and imported the ./stripy source (no .so) instead of the installed package → now runs from a neutral dir. Added a py3.9–3.13 × numpy{<2, ≥2} matrix that exercises exactly the "numpy-2-built wheel on numpy 1.x" path.
  • Modernised the doc workflows and the Fortran-setup composite action.

meson robustness

  • meson.build: removed the fragile os.chdir("..") numpy-include hack; fixed malformed rpath link args; excluded editor cruft from install_subdir.
  • Build issue with flang 21 #116: the per-platform link flags (-static, -Wl,-rpath) are gfortran/GCC conventions — now gated on meson.get_compiler('fortran').get_id() == 'gcc' so LLVM flang / MSVC toolchains don't get bogus flags. Because both host_machine.system() and get_compiler('fortran') resolve to the host/target, cross-compiled gfortran builds (conda-forge linux-aarch64/osx-arm64) still get the correct flags. (Note: this removes stripy's contribution to the Build issue with flang 21 #116 log; the fatal flang_rt.runtime.dynamic.lib error there is a flang-21 toolchain issue on the feedstock side, not stripy.)

#117 — the ier=-2 error message described only one of the two collinear cases the Fortran can return (first-three-nodes-collinear vs. a later ADDNOD collinearity). Extended the message in both spherical.py and cartesian.py (verified against stripack.f90/tripack.f90).

Hygiene — widened .gitignore (f2py output + local data caches), removed stray build artifacts and dead conda-CI env files, refreshed the README workflow badges/conda channel, added a Changelog entry.

Validation

Built locally in a clean venv (pip install ., PEP 517 isolation → compiled against numpy 2.5.1) and ran the suite under numpy 2.5.1:

46 passed, 2 warnings

All five compiled extensions import, and both test_linear_interpolation variants pass — the failure tracked in #115. A separate clean no-cache rebuild confirmed the meson #116 change compiles and links on the gfortran path with no behaviour change.

Not included / follow-ups

  • Publishing to PyPI still requires cutting a v2.3.4 GitHub release — that's what triggers the (now-fixed) wheel upload.
  • The conda-forge feedstock is intentionally untouched: the autotick bot opens the version-bump PR automatically once the PyPI sdist exists (the new sha256 can't be computed before then).
  • Pre-existing, out of scope: the permute retry loop in spherical.py (_update_triangulation) raises even if the final attempt succeeds (if niter >= 5 should be if ierr == -2). Happy to fix in a follow-up.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FFBEqxMhk9Ut6vVm5jfV8d


Update — CI is green; wheel-coverage changes to note

The first real CI runs (the old workflows were effectively not running tests — pytest ran from the repo root and imported the un-built source tree) surfaced several issues, now fixed on this branch:

  • test_linear_interpolation used exact float == on interpolated endpoints — platform-dependent for the permuted mesh (failed on Linux x64, passed on macOS arm64). Now np.isclose (fixes test_linear_interpolation is failing for NumPy 2.x #115).
  • macOS wheels: set MACOSX_DEPLOYMENT_TARGET per runner so delocate accepts the bundled Homebrew libgfortran.
  • Linux wheels: PIP_ONLY_BINARY=scipy (stop source-building scipy in the container) and build on manylinux_2_28 (GCC 12) — the old manylinux2014 image couldn't build modern numpy/scipy from source (NumPy requires GCC >= 10.3).
  • Added concurrency (cancel superseded runs) and stopped feature-branch pushes from redundantly building the full wheel matrix.
  • Trimmed the test matrix (9 → 3 jobs): kept the unique numpy-1.x ABI path (py3.9/3.12) plus one numpy-2.x smoke test (py3.13); the numpy-2.x-across-all-pythons grid was already covered by the per-OS wheel tests.

Wheel platform coverage changed — please note for the release:

  • macOS: Apple-silicon (arm64) only, minimum macOS 14. No Intel-mac (x86_64) PyPI wheel — Intel users are served by the conda-forge package. A portable low-target gfortran could restore lower/Intel targets later.
  • Linux: glibc ≥ 2.28 (manylinux_2_28; RHEL8 / Ubuntu 18.10+), up from the old manylinux2014 (glibc 2.17) baseline.

All checks green: 3 test jobs + sdist + ubuntu / windows / macos-14 wheels.

Ben Mather and others added 9 commits July 23, 2026 11:44
PyPI wheels were compiled against NumPy 1.x and crash at import under
NumPy 2.x. Build against NumPy 2.x (binaries then run on both 1.x and
2.x) and fix the surrounding release/build machinery.

- Replace NumPy aliases removed in 1.24 (np.int/np.bool)
- pyproject: build-requires numpy>=2.0, runtime numpy>=1.19.3; drop dead
  setuptools config; add py3.13
- Remove obsolete numpy.distutils setup.py/setup.cfg build path
- meson.build: drop the os.chdir numpy-include hack, fix malformed rpath
  link args, exclude cruft from install, accurate f2py comments
- pypi.yml: fix actions/upload-artifact@v4 duplicate-name failure (unique
  per-job names + download pattern/merge-multiple); bump actions;
  cibuildwheel 2.21.3; add scipy to wheel test deps
- CI.yml: matrix over py3.9-3.13 x numpy{<2,>=2}; run pytest from a
  neutral dir so the installed package (not the source shadow) is tested
- Modernise doc workflows and the Fortran-setup composite action
- Remove the stale in-repo conda recipe (superseded by the conda-forge
  feedstock), dead CI env files and stray build artifacts; widen
  .gitignore; refresh README badges/channel; add a changelog entry

Verified locally: builds against numpy 2.5.1 and all 46 tests pass under
numpy 2.5.1, including test_linear_interpolation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FFBEqxMhk9Ut6vVm5jfV8d
The per-platform link arguments (`-static` on Windows, `-Wl,-rpath,...`
on Unix) are gfortran/GCC conventions. LLVM flang and the MSVC-based
linkers warn about or silently ignore them (see gh-116, building with
flang 21 on Windows), so gate the whole block on
`meson.get_compiler('fortran').get_id() == 'gcc'`.

Both `host_machine.system()` and `get_compiler('fortran')` refer to the
host/target machine, so cross-compiled gfortran builds (e.g. conda-forge
linux-aarch64 / osx-arm64) still receive the correct per-target flags,
while non-GNU toolchains no longer get bogus flags. No behaviour change
for the existing gfortran Linux / macOS / MinGW (PyPI) builds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FFBEqxMhk9Ut6vVm5jfV8d
trmesh (spherical) and its planar counterpart both return ier=-2 in two
distinct situations: the first three nodes being collinear, and a later
ADDNOD call reporting that all nodes up to some node are collinear
(stripack.f90:59,5406 / tripack.f90:421,6433). The old message only
described the first case, giving misleading guidance for the second.

Extend the _ier_codes[-2] message in both spherical.py and cartesian.py
to cover both cases and note the data may be genuinely (near-)collinear
or degenerate. (Parsing the Fortran stdout to distinguish the two, as
suggested in the issue, isn't reliable through f2py, so a clearer single
message is the robust fix.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FFBEqxMhk9Ut6vVm5jfV8d
test_linear_interpolation compared interpolated endpoint values with
exact float equality, which is platform-dependent for the permuted
(random) mesh: it failed on Linux x86_64 under NumPy 2.x while passing on
macOS/arm64 (0 failures in 1000 local runs). The endpoints coincide with
input nodes, so linear interpolation recovers their coordinate only up to
floating-point rounding (~1e-16) -- the test's own comment already noted
"machine precision may differ". Compare with np.isclose and emit an
informative failure message. The monotonic `ascending` check is
unchanged. Applied to both the spherical and Cartesian tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FFBEqxMhk9Ut6vVm5jfV8d
The macos-13/14 wheel jobs failed in delocate with "Library dependencies
do not satisfy target MacOS version 11.0": Homebrew's gfortran targets
the runner's macOS version, so the libgfortran/libquadmath that delocate
bundles require that minimum, above the wheel's default 11.0 tag. Match
MACOSX_DEPLOYMENT_TARGET to the runner (13.0 / 14.0) via
CIBW_ENVIRONMENT_MACOS. Also set fail-fast: false so one OS failing no
longer cancels and masks the others.

Note: this raises the wheels' minimum macOS (arm64 -> 14, x86_64 -> 13);
a portable gfortran targeting an older macOS could lower it later.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FFBEqxMhk9Ut6vVm5jfV8d
Adding scipy to CIBW_TEST_REQUIRES caused the manylinux wheel test to
fail: for some interpreters pip resolved a scipy version with no matching
wheel (e.g. scipy 1.17.1 for cp311) and tried to build it from the sdist
inside the manylinux container, which has no OpenBLAS/build tooling
("Dependency OpenBLAS not found"). Set PIP_ONLY_BINARY=scipy on all three
platforms so pip only installs a scipy wheel, falling back to the newest
version that ships one for the target interpreter.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FFBEqxMhk9Ut6vVm5jfV8d
Add a concurrency group (cancel-in-progress) to both workflows so a new
push cancels the superseded run on the same ref instead of leaving stale
jobs queued behind the slow macOS runners. Also restrict the wheel-build
push trigger to master; feature-branch commits are already validated by
the pull_request event, so this stops every branch push from redundantly
building the full 4-OS wheel matrix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FFBEqxMhk9Ut6vVm5jfV8d
Run tests (CI.yml): 9 jobs -> 3. Its unique coverage is the NumPy-1.x
runtime ABI path (gh-114), so test that on the oldest and a recent Python
plus one NumPy-2.x source-build smoke test on the newest. Per-OS /
per-Python NumPy-2.x coverage is already provided by the wheel build+test
in pypi.yml, so the previous 9-way python x numpy grid was largely
redundant.

Build wheels: drop the macos-13 (x86_64) job and ship Apple-silicon
(arm64) wheels only. Intel-mac users are served by the conda-forge
package; x86_64 was also the slowest job to get a runner.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FFBEqxMhk9Ut6vVm5jfV8d
The default manylinux2014 image ships GCC 10.2. Recent NumPy (2.5+) only
publishes manylinux_2_28 wheels, so the PEP 517 build-isolation install
of `numpy>=2.0` fell back to a source build for cp312/cp313 and failed:
"NumPy requires GCC >= 10.3". Switch the Linux build images to
manylinux_2_28 / musllinux_1_2 (GCC 12), where numpy/scipy wheels are
available and any source build has a new enough compiler. This raises the
Linux wheels' glibc floor to 2.28 (RHEL8/Ubuntu 18.10+), the current
manylinux baseline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FFBEqxMhk9Ut6vVm5jfV8d
@brmather

Copy link
Copy Markdown
Member Author

Overhaul the rusty CI matrix, now we're getting green ticks...

@brmather
brmather requested a review from lmoresi July 23, 2026 04:27
@brmather

Copy link
Copy Markdown
Member Author

@lmoresi - if you're happy with this, then I'll merge and create a v2.4 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Please clarify this _ier_codes value for Exceptions from sTriangulation._update_triangulation(...) test_linear_interpolation is failing for NumPy 2.x

1 participant