Add array_chunks by ronnodas · Pull Request #1023 · rust-itertools/itertools

ronnodas · 2025-03-02T18:42:28Z

Added an implementation of arrays that just repeatedly calls next_array(). Currently the N = 0 produces a post-monomorphization error, see the discussion in #1012.

The remainder() feature is not implemented, based on the discussion below, since this seems to require either:

gating this behind use_alloc, or
making the return type not Clone, or
more unsafe code implementing effectively more of arrayvec.

codecov · 2025-03-02T18:43:55Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.91%. Comparing base (6814180) to head (e923eb1).
⚠️ Report is 211 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1023      +/-   ##
==========================================
- Coverage   94.38%   93.91%   -0.48%     
==========================================
  Files          48       53       +5     
  Lines        6665     6707      +42     
==========================================
+ Hits         6291     6299       +8     
- Misses        374      408      +34

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ronnodas · 2025-03-02T23:53:01Z

Re the MSRV failure, I'm not sure what the idiomatic way is to trigger a PME on 1.63 based on N > 0.

jswrenn · 2025-03-03T00:07:13Z

Re the MSRV failure, I'm not sure what the idiomatic way is to trigger a PME on 1.63 based on N > 0.

You can do it like this: https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=233fb2f6ca3df52d15e3df96b8c4cb61

jswrenn · 2025-03-05T15:04:56Z

+/// Effectively assert!(N > 0) post-monomorphization
+fn assert_positive<const N: usize>() {
+    trait StaticAssert<const N: usize> {
+        const ASSERT: bool;
+    }
+
+    impl<const N: usize> StaticAssert<N> for () {
+        const ASSERT: bool = {
+            assert!(N > 0);
+            true
+        };
+    }
+
+    assert!(<() as StaticAssert<N>>::ASSERT);
+}


Rust doesn't provide stacktraces for monomorphization errors, so folks will only see that an error occured in assert_positive — and not the code that actually required the assertion.

Define this as a macro and call that instead.

Is the change in db21b2f what you meant?

jswrenn · 2025-03-05T15:05:48Z

+    /// assert_eq!(Some([]), it.next());
+    /// ```
+    #[cfg(feature = "use_alloc")]
+    fn arrays<const N: usize>(self) -> Arrays<Self, N>


IMO array_chunks is the better name, since it differentiates it from array_windows.

To add context, I was trying to avoid triggering the unstable-name-collisions lint, because of rust-lang/rust#100450. Should I change it back to array_chunks?

jswrenn · 2025-03-05T15:28:02Z

+pub struct Arrays<I: Iterator, const N: usize> {
+    iter: I,
+    partial: Vec<I::Item>,
+}


It strikes me as a code smell that N doesn't appear in the definition here. Shouldn't partial be an ArrayBuilder<T, N>?

Making it ArrayBuilder<T, N> a priori makes sense, but I had trouble with the bounds for the impl Clone in that case. Note that MaybeUninit<T>: Clone requires T: Copy.

Maybe it could also be array::IntoIter<T, N> or a hypothetical ArrayBuilder::IntoIter<T, N> (this is close to what the unstable Iterator::ArrayChunks does) but I think the unsafe code for that would need to be more careful.

The state you need to keep track of doesn't really depend on N (except partial.len() <= N but this is sort of incidental). However Arrays needs to have N as a parameter for Arrays::Item to be well-defined.

Suggestions?

Yes, should be ArrayBuilder, because this brings it closer to no_alloc.

Maybe this array builder could also be re-used in next.

Can you remember/share the exact problem?

@phimuemue If it holds an ArrayBuilder<T> then it won't be Clone without T : Copy. But we should have Arrays<I, N>: Clone when I: Clone and I::Item : Clone.

Ok. Then, let's implement it without partial/remaining for now.

(Because if we extend our ArrayBuilder further and further, we land at (a less tested version of) ArrayVec.)

king-11 · 2025-10-12T10:40:37Z

what are the thoughts about having an array_chunks which can fill the partially filled last item by fetching from a filler that the caller provides? I think it would be a very useful API in case people always want a certain length chunks

bend-n · 2025-11-11T07:18:52Z

im curious why you didnt want to just copy the previous stdlib implementation? you can simply implement this on top of a manual as_chunks?
returning an owned array and using a vec for remainder is kind of funky.

ronnodas · 2025-11-11T07:30:14Z

@bend-n Do you mean slice::as_chunks? That works for slices, but this is for a general iterator.

There's also a nightly version of this in std: Iterator::array_chunks. If you're asking about the implementation there, I don't think the code in this PR is that different, just trying to reuse things already in the crate. But TBF I haven't looked at either that implementation or this PR closely in a while.

bend-n · 2025-11-11T07:42:23Z

oh, my bad. i forgot that iterator::array_chunks was still unstable and assumed this was with regards to slice array chunks for some reason.

you could still copy the implementation of iterator::array_chunks, though?

ronnodas · 2025-12-23T03:57:18Z

you could still copy the implementation of iterator::array_chunks, though?

If you go through the source, Iterator::array_chunks is implemented in terms of (also unstable) Iterator::next_chunk, which is equivalent to Itertools::next_array. It made more sense to use the thing that's already in the crate rather than copy over something with duplicate functionality.

phimuemue · 2026-07-24T20:59:12Z

If it turns out to be too complicated, maybe go without partial in our first iteration?

Iterator::array_chunks exists on nightly and it's better to avoid the name collision especially if and when that's stabilized

ronnodas · 2026-07-25T09:53:00Z

@phimuemue I've removed partial/remainder() and thereby the gating behind use_alloc.

phimuemue · 2026-07-26T19:55:24Z

+    /// let it = (1..7).arrays::<3>();
+    /// itertools::assert_equal(it, vec![[1, 2, 3], [4, 5, 6]]);
+    ///
+    /// // you can also specify the complete type


Let's not encourage people to spell iterator types

Note that such an example is included in the docs for tuple_windows(), tuples(), tuple_combinations() and array_combinations(). Should I still remove this or keep it for consistency?

As you wish.

phimuemue · 2026-07-26T19:56:14Z


+    /// Return an iterator that groups the items in arrays of const generic size `N`.
+    ///
+    /// `N == 0` is a compile-time (but post-monomorphization) error.


Is it really important for users that it is post-monomorphization?

It won't be caught by cargo check, for example, which seems relevant to users.

phimuemue · 2026-07-26T20:00:18Z

@jswrenn regarding naming: what's our policy? IMHO, the best name is array_chunks, and I would use it and deprecate our function once it's also available in std.

ronnodas force-pushed the array-chunks branch from 12e26ae to d866b7d Compare March 2, 2025 23:19

ronnodas force-pushed the array-chunks branch from edaabbf to 6bede55 Compare March 3, 2025 00:16

jswrenn requested changes Mar 5, 2025

View reviewed changes

ronnodas requested a review from jswrenn June 20, 2025 18:47

ronnodas force-pushed the array-chunks branch from db21b2f to 013edf5 Compare March 5, 2026 07:12

bend-n reviewed Mar 5, 2026

View reviewed changes

Comment thread src/arrays.rs Outdated

ronnodas added 12 commits July 25, 2026 05:11

array_chunks

29080b4

copy-paste artefact

37a77b8

PME on array_chunks::<0>

e3af5fb

impl ExactSizeIterator

3aa4f60

exact size tests

add6f83

rename to arrays

d3db791

Iterator::array_chunks exists on nightly and it's better to avoid the name collision especially if and when that's stabilized

avoid const assert to keep msrv

e1d5072

make assert_positive a macro

eb82706

remove remainder()

1efe4a8

reuse existing next_array()

00b8c1e

don't require alloc

6a2f832

update TODO comment

a71605b

ronnodas force-pushed the array-chunks branch from 013edf5 to eb82706 Compare July 25, 2026 09:39

phimuemue reviewed Jul 25, 2026

View reviewed changes

Comment thread src/next_array.rs Outdated

revert changes to ArrayBuilder

41a68ac

phimuemue reviewed Jul 25, 2026

View reviewed changes

Comment thread src/arrays.rs Outdated

remove N = 0 check in size_hint()

66243aa

phimuemue reviewed Jul 25, 2026

View reviewed changes

Comment thread src/arrays.rs Outdated

phimuemue reviewed Jul 25, 2026

View reviewed changes

Comment thread tests/quick.rs Outdated

move exact size tests to quickcheck

bec8ee3

ronnodas force-pushed the array-chunks branch from 0d7e7cf to bec8ee3 Compare July 26, 2026 17:21

phimuemue reviewed Jul 26, 2026

View reviewed changes

Comment thread src/arrays.rs

assert N > 0 in size_hint()

e923eb1

Uh oh!

Conversation

ronnodas commented Mar 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Mar 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ronnodas commented Mar 2, 2025

Uh oh!

jswrenn commented Mar 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ronnodas Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

phimuemue Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

king-11 commented Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bend-n commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ronnodas commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bend-n commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ronnodas commented Dec 23, 2025

Uh oh!

Uh oh!

phimuemue commented Jul 24, 2026

Uh oh!

ronnodas commented Jul 25, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

phimuemue commented Jul 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ronnodas commented Mar 2, 2025 •

edited

Loading

codecov Bot commented Mar 2, 2025 •

edited

Loading

ronnodas Mar 5, 2025 •

edited

Loading

phimuemue Jul 24, 2026 •

edited

Loading

king-11 commented Oct 12, 2025 •

edited

Loading

bend-n commented Nov 11, 2025 •

edited

Loading

ronnodas commented Nov 11, 2025 •

edited

Loading

bend-n commented Nov 11, 2025 •

edited

Loading

phimuemue commented Jul 26, 2026 •

edited

Loading