Add permutedims! benchmark suite#71
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 🚀 New features to boost your workflow:
|
BenchmarkTools suite covering the HPTT/TTC reference cases, small sizes (including the strided-rs cases), and non-SIMD-aligned sizes, over element types and thread counts, configurable via ArgParse. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cases (and per-group sampling parameters) now live in benchmark/cases.toml; an alternative file can be passed with --cases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…trided views Cases now fix a shape (dimension ratios) and permutation and sweep a list of total lengths, so each can be normalized against a same-length memory copy (auto-generated `copy` group) for a machine-independent efficiency. Adds a `strided` group whose inputs/outputs are genuinely non-contiguous strided views (non-unit leading and second-leading strides), replacing the explicit-size hptt/small/unaligned groups. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Your PR no longer requires formatting changes. Thank you for your contribution! |
| end | ||
|
|
||
| # Column-major strides of a dense array of size `sz`. | ||
| colstrides(sz::NTuple{N, Int}) where {N} = ntuple(i -> prod(ntuple(j -> sz[j], i - 1)), N) |
There was a problem hiding this comment.
| colstrides(sz::NTuple{N, Int}) where {N} = ntuple(i -> prod(ntuple(j -> sz[j], i - 1)), N) | |
| colstrides(sz::NTuple{N, Int}) where {N} = (1, Base.front(cumprod(sz))...) |
| isempty(totals) && | ||
| error("$group: case uses `shape` but the group has no `totals`") | ||
| for total in totals | ||
| push!(insts, (; p, dims = shapedims(shape, total), min, mout, L = total, shape)) |
There was a problem hiding this comment.
Should this use L = prod(shapedims(shape, total)), i.e. the actual size. I think L is used for the comparison with plain copy!, correct?
|
Looks good; what is the plan with this? Can it be ran automatically upon PRs? Is CI infrastructure reliable for that? Are there comparisons with older versions available already? |
|
I think I still want to add some actual plotting code here and then indeed try to see if the stuff I did actually did improve/regress some parts of this. In principle we can try to run this on CI, but I think it might take too long so it might actually be better to do this manually, similar to how TensorKit has that |
2 participants
Adds a BenchmarkTools.jl suite at
benchmark/benchmarks.jl(definingSUITE) for trackingpermutedims!performance across revisions with AirSpeedVelocity.jl.Each case fixes a shape (dimension ratios) and a permutation and is swept over a list of total lengths, so its time can be divided by that of a same-length memory copy (the auto-generated
copygroup) for a machine-independent efficiency (≥ 1, approaching1when bandwidth-bound). Cases are defined incases.tomlin three groups, run forFloat64/ComplexF64and single/multi-threaded:balanced— equal dimensions across ranks 2–6skewed— one fat leading or trailing axis (the HPTT/TTC regime, as clean ratios)strided— genuinely non-contiguous strided views (non-unit leading and second-leading strides on input/output), which exercise the paths a dense array never hitsConfigurable through CLI options (ArgParse.jl) or a different
--casesTOML file; seebenchmark/README.mdfor usage.🤖 Generated with Claude Code