[Parquet] Vectorise Pack Bits within BitWriter by sdf-jkl · Pull Request #10432 · apache/arrow-rs

sdf-jkl · 2026-07-24T22:05:04Z

Which issue does this PR close?

Closes Vectorised Pack Bits within BitWriter #2257.

Rationale for this change

Perf 🚀

What changes are included in this PR?

Renamed the FromBitpacked to BitPacking to store the Packing APIs
Added vectorised pack_batch matching unpack_batch kernels
Added unit tests
Added bench arms

Are these changes tested?

Yes!

Are there any user-facing changes?

No

sdf-jkl · 2026-07-25T00:35:33Z

run benchmark arrow_writer

adriangbot · 2026-07-25T00:38:31Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5075864703-1263-hj9x2 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing put_batch (f462d31) to f7dfcd2 (merge-base) diff
BENCH_NAME=arrow_writer
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_writer
BENCH_FILTER=
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-07-25T02:18:10Z

🤖 Arrow criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                              main                                   put_batch
-----                                              ----                                   ---------
bool/bloom_filter                                  1.01     13.0±0.06ms    19.3 MB/sec    1.00     12.8±0.10ms    19.5 MB/sec
bool/cdc                                           1.01     15.7±0.07ms    15.9 MB/sec    1.00     15.5±0.11ms    16.1 MB/sec
bool/default                                       1.01     10.8±0.06ms    23.1 MB/sec    1.00     10.7±0.10ms    23.4 MB/sec
bool/parquet_2                                     1.04     14.6±0.07ms    17.1 MB/sec    1.00     14.0±0.09ms    17.9 MB/sec
bool/zstd                                          1.02     11.3±0.04ms    22.1 MB/sec    1.00     11.2±0.10ms    22.4 MB/sec
bool/zstd_parquet_2                                1.04     15.0±0.05ms    16.7 MB/sec    1.00     14.3±0.10ms    17.4 MB/sec
bool_non_null/bloom_filter                         1.08      7.0±0.03ms    18.0 MB/sec    1.00      6.5±0.03ms    19.4 MB/sec
bool_non_null/cdc                                  1.09      6.8±0.05ms    18.4 MB/sec    1.00      6.2±0.03ms    20.1 MB/sec
bool_non_null/default                              1.13      4.2±0.03ms    29.9 MB/sec    1.00      3.7±0.02ms    34.0 MB/sec
bool_non_null/parquet_2                            1.11      8.9±0.03ms    14.0 MB/sec    1.00      8.0±0.03ms    15.6 MB/sec
bool_non_null/zstd                                 1.13      4.5±0.04ms    27.6 MB/sec    1.00      4.0±0.03ms    31.1 MB/sec
bool_non_null/zstd_parquet_2                       1.11      9.3±0.04ms    13.4 MB/sec    1.00      8.4±0.03ms    14.9 MB/sec
bool_ree/bloom_filter                              1.00     30.4±0.08ms     7.0 MB/sec    1.00     30.3±0.15ms     7.0 MB/sec
bool_ree/cdc                                       1.00     31.7±0.09ms     6.7 MB/sec    1.00     31.6±0.12ms     6.7 MB/sec
bool_ree/default                                   1.00     27.7±0.08ms     7.7 MB/sec    1.00     27.6±0.14ms     7.7 MB/sec
bool_ree/parquet_2                                 1.00     29.3±0.10ms     7.3 MB/sec    1.01     29.5±0.13ms     7.2 MB/sec
bool_ree/zstd                                      1.00     28.3±0.08ms     7.5 MB/sec    1.00     28.2±0.14ms     7.6 MB/sec
bool_ree/zstd_parquet_2                            1.00     29.6±0.09ms     7.2 MB/sec    1.01     29.8±0.15ms     7.1 MB/sec
decimal/bloom_filter                               1.00    146.5±0.60ms   409.6 MB/sec    1.01    148.0±0.78ms   405.5 MB/sec
decimal/cdc                                        1.00    131.6±0.43ms   456.1 MB/sec    1.01    132.3±0.21ms   453.4 MB/sec
decimal/default                                    1.00    107.0±0.48ms   560.5 MB/sec    1.00    107.3±0.35ms   559.3 MB/sec
decimal/parquet_2                                  1.01    272.3±0.91ms   220.4 MB/sec    1.00    269.0±0.68ms   223.0 MB/sec
decimal/zstd                                       1.00    118.2±0.40ms   507.7 MB/sec    1.00    118.1±0.40ms   508.1 MB/sec
decimal/zstd_parquet_2                             1.01    292.1±0.71ms   205.4 MB/sec    1.00    289.2±0.60ms   207.4 MB/sec
fixed_size_binary_ree/bloom_filter                 1.00     69.3±0.36ms    14.5 MB/sec    1.00     69.6±0.27ms    14.5 MB/sec
fixed_size_binary_ree/cdc                          1.00     72.8±0.15ms    13.8 MB/sec    1.00     73.0±0.18ms    13.8 MB/sec
fixed_size_binary_ree/default                      1.00     62.0±0.20ms    16.2 MB/sec    1.01     62.6±0.24ms    16.1 MB/sec
fixed_size_binary_ree/parquet_2                    1.00     79.2±0.51ms    12.7 MB/sec    1.02     80.8±0.31ms    12.5 MB/sec
fixed_size_binary_ree/zstd                         1.00     67.7±0.27ms    14.9 MB/sec    1.01     68.2±0.23ms    14.8 MB/sec
fixed_size_binary_ree/zstd_parquet_2               1.00     79.9±0.56ms    12.6 MB/sec    1.02     81.8±0.39ms    12.3 MB/sec
float_with_nans/bloom_filter                       1.00     92.2±0.22ms   151.8 MB/sec    1.01     92.7±0.26ms   151.0 MB/sec
float_with_nans/cdc                                1.00     81.0±0.21ms   172.8 MB/sec    1.01     82.0±0.17ms   170.8 MB/sec
float_with_nans/default                            1.00     73.8±0.20ms   189.7 MB/sec    1.01     74.5±0.13ms   188.0 MB/sec
float_with_nans/parquet_2                          1.00     94.3±0.22ms   148.5 MB/sec    1.02     96.6±0.25ms   144.9 MB/sec
float_with_nans/zstd                               1.00    111.9±0.16ms   125.2 MB/sec    1.01    112.6±0.20ms   124.3 MB/sec
float_with_nans/zstd_parquet_2                     1.00    131.4±0.34ms   106.5 MB/sec    1.02    133.8±0.27ms   104.6 MB/sec
fsb/bloom_filter                                   1.00      9.7±0.11ms  1654.5 MB/sec    1.00      9.7±0.11ms  1654.3 MB/sec
fsb/cdc                                            1.01     12.1±0.07ms  1328.5 MB/sec    1.00     12.0±0.06ms  1344.0 MB/sec
fsb/default                                        1.00      8.5±0.06ms  1896.2 MB/sec    1.00      8.5±0.05ms  1900.0 MB/sec
fsb/parquet_2                                      1.00     18.4±0.11ms   875.4 MB/sec    1.00     18.4±0.12ms   874.9 MB/sec
fsb/zstd                                           1.00     10.9±0.07ms  1478.0 MB/sec    1.02     11.2±0.10ms  1444.5 MB/sec
fsb/zstd_parquet_2                                 1.00     19.1±0.11ms   845.5 MB/sec    1.02     19.4±0.15ms   830.2 MB/sec
int32_ree/bloom_filter                             1.00     40.8±0.25ms    10.0 MB/sec    1.04     42.5±0.36ms     9.6 MB/sec
int32_ree/cdc                                      1.00     40.3±0.08ms    10.1 MB/sec    1.03     41.7±0.13ms     9.8 MB/sec
int32_ree/default                                  1.00     36.4±0.09ms    11.2 MB/sec    1.04     37.9±0.13ms    10.7 MB/sec
int32_ree/parquet_2                                1.00     36.4±0.09ms    11.2 MB/sec    1.04     37.9±0.13ms    10.7 MB/sec
int32_ree/zstd                                     1.00     37.0±0.07ms    11.0 MB/sec    1.04     38.5±0.14ms    10.6 MB/sec
int32_ree/zstd_parquet_2                           1.00     36.9±0.10ms    11.0 MB/sec    1.04     38.4±0.13ms    10.6 MB/sec
int32_ree_95pct_null/bloom_filter                  1.00     22.0±0.09ms    18.5 MB/sec    1.01     22.1±0.07ms    18.4 MB/sec
int32_ree_95pct_null/cdc                           1.01     24.2±0.05ms    16.8 MB/sec    1.00     23.9±0.05ms    17.0 MB/sec
int32_ree_95pct_null/default                       1.00     21.6±0.04ms    18.9 MB/sec    1.00     21.7±0.03ms    18.8 MB/sec
int32_ree_95pct_null/parquet_2                     1.00     21.6±0.03ms    18.8 MB/sec    1.01     21.7±0.06ms    18.7 MB/sec
int32_ree_95pct_null/zstd                          1.00     21.9±0.04ms    18.6 MB/sec    1.00     21.9±0.04ms    18.6 MB/sec
int32_ree_95pct_null/zstd_parquet_2                1.00     21.8±0.03ms    18.7 MB/sec    1.00     21.8±0.03ms    18.6 MB/sec
large_string_non_null/bloom_filter                 1.00     71.7±0.13ms     3.5 GB/sec    1.02     72.9±0.11ms     3.4 GB/sec
large_string_non_null/cdc                          1.00    243.2±1.17ms  1052.5 MB/sec    1.00    244.2±1.02ms  1048.2 MB/sec
large_string_non_null/default                      1.00     53.8±0.11ms     4.6 GB/sec    1.02     55.0±0.11ms     4.5 GB/sec
large_string_non_null/parquet_2                    1.00     53.7±0.10ms     4.7 GB/sec    1.03     55.2±0.12ms     4.5 GB/sec
large_string_non_null/zstd                         1.00     53.8±0.10ms     4.6 GB/sec    1.02     55.1±0.10ms     4.5 GB/sec
large_string_non_null/zstd_parquet_2               1.00     53.9±0.11ms     4.6 GB/sec    1.02     55.2±0.11ms     4.5 GB/sec
list_nested/bloom_filter                           1.00    148.3±0.21ms   194.5 MB/sec    1.00    148.6±0.23ms   194.2 MB/sec
list_nested/cdc                                    1.01    165.4±0.17ms   174.4 MB/sec    1.00    163.8±0.22ms   176.2 MB/sec
list_nested/default                                1.01    133.6±0.22ms   215.9 MB/sec    1.00    132.8±0.20ms   217.2 MB/sec
list_nested/parquet_2                              1.02    146.8±0.23ms   196.5 MB/sec    1.00    143.9±0.19ms   200.5 MB/sec
list_nested/zstd                                   1.00    142.8±0.18ms   202.1 MB/sec    1.00    142.3±0.42ms   202.8 MB/sec
list_nested/zstd_parquet_2                         1.02    159.6±0.19ms   180.8 MB/sec    1.00    156.7±0.23ms   184.2 MB/sec
list_primitive/bloom_filter                        1.00    282.0±1.19ms  1933.8 MB/sec    1.00    281.5±1.89ms  1937.2 MB/sec
list_primitive/cdc                                 1.01    315.4±0.96ms  1729.3 MB/sec    1.00    312.6±2.01ms  1744.7 MB/sec
list_primitive/default                             1.02    205.8±0.94ms     2.6 GB/sec    1.00    200.9±1.63ms     2.7 GB/sec
list_primitive/parquet_2                           1.03    227.9±0.48ms     2.3 GB/sec    1.00    221.6±0.43ms     2.4 GB/sec
list_primitive/zstd                                1.01    461.1±0.74ms  1182.7 MB/sec    1.00    457.5±2.78ms  1192.1 MB/sec
list_primitive/zstd_parquet_2                      1.01    450.8±1.04ms  1209.8 MB/sec    1.00    444.9±0.37ms  1225.7 MB/sec
list_primitive_non_null/bloom_filter               1.06    372.0±5.04ms  1463.1 MB/sec    1.00    350.7±4.58ms  1551.8 MB/sec
list_primitive_non_null/cdc                        1.00    395.4±6.87ms  1376.5 MB/sec    1.00    395.0±7.70ms  1378.0 MB/sec
list_primitive_non_null/default                    1.11    241.7±2.82ms     2.2 GB/sec    1.00    217.8±4.92ms     2.4 GB/sec
list_primitive_non_null/parquet_2                  1.11    272.8±0.65ms  1994.8 MB/sec    1.00    246.3±0.56ms     2.2 GB/sec
list_primitive_non_null/zstd                       1.03    665.3±2.64ms   818.1 MB/sec    1.00    647.5±4.98ms   840.5 MB/sec
list_primitive_non_null/zstd_parquet_2             1.03    649.6±8.85ms   837.7 MB/sec    1.00    631.0±3.47ms   862.4 MB/sec
list_primitive_sparse_99pct_null/bloom_filter      1.00     12.8±0.09ms     2.9 GB/sec    1.01     12.8±0.12ms     2.8 GB/sec
list_primitive_sparse_99pct_null/cdc               1.01     24.6±0.12ms  1520.0 MB/sec    1.00     24.5±0.10ms  1527.7 MB/sec
list_primitive_sparse_99pct_null/default           1.00     12.4±0.06ms     2.9 GB/sec    1.01     12.5±0.07ms     2.9 GB/sec
list_primitive_sparse_99pct_null/parquet_2         1.00     12.4±0.05ms     2.9 GB/sec    1.00     12.5±0.07ms     2.9 GB/sec
list_primitive_sparse_99pct_null/zstd              1.00     14.3±0.07ms     2.5 GB/sec    1.00     14.4±0.09ms     2.5 GB/sec
list_primitive_sparse_99pct_null/zstd_parquet_2    1.00     12.6±0.09ms     2.9 GB/sec    1.00     12.6±0.11ms     2.9 GB/sec
list_struct_with_list/bloom_filter                 1.00    283.2±0.46ms   161.0 MB/sec    1.01    284.8±0.52ms   160.1 MB/sec
list_struct_with_list/cdc                          1.00    324.2±0.46ms   140.6 MB/sec    1.00    323.3±0.52ms   141.0 MB/sec
list_struct_with_list/default                      1.00    257.9±0.39ms   176.8 MB/sec    1.00    258.5±0.38ms   176.4 MB/sec
list_struct_with_list/parquet_2                    1.01    275.5±0.37ms   165.5 MB/sec    1.00    273.3±0.39ms   166.9 MB/sec
list_struct_with_list/zstd                         1.00    280.4±0.88ms   162.6 MB/sec    1.00    280.7±0.45ms   162.4 MB/sec
list_struct_with_list/zstd_parquet_2               1.01    298.5±0.42ms   152.8 MB/sec    1.00    296.5±0.37ms   153.8 MB/sec
primitive/bloom_filter                             1.00    148.4±0.55ms   302.5 MB/sec    1.02    150.8±0.67ms   297.6 MB/sec
primitive/cdc                                      1.00    158.1±0.70ms   283.9 MB/sec    1.02    160.5±0.66ms   279.7 MB/sec
primitive/default                                  1.00    116.3±0.44ms   385.8 MB/sec    1.03    119.5±0.59ms   375.4 MB/sec
primitive/parquet_2                                1.01    131.5±0.50ms   341.2 MB/sec    1.00    130.5±0.66ms   343.9 MB/sec
primitive/zstd                                     1.00    146.8±0.66ms   305.7 MB/sec    1.02    149.4±0.70ms   300.4 MB/sec
primitive/zstd_parquet_2                           1.01    165.3±2.57ms   271.4 MB/sec    1.00    163.6±0.63ms   274.4 MB/sec
primitive_all_null/bloom_filter                    1.00    880.6±2.84µs    49.8 GB/sec    1.01    892.2±6.10µs    49.1 GB/sec
primitive_all_null/cdc                             1.01     19.1±0.36ms     2.3 GB/sec    1.00     18.9±0.32ms     2.3 GB/sec
primitive_all_null/default                         1.00    267.8±1.32µs   163.7 GB/sec    1.01    270.6±1.11µs   161.9 GB/sec
primitive_all_null/parquet_2                       1.00    267.9±1.25µs   163.6 GB/sec    1.00    269.2±1.40µs   162.8 GB/sec
primitive_all_null/zstd                            1.00    418.7±1.23µs   104.7 GB/sec    1.00    417.0±1.13µs   105.1 GB/sec
primitive_all_null/zstd_parquet_2                  1.00    366.1±1.32µs   119.7 GB/sec    1.00    367.5±1.15µs   119.3 GB/sec
primitive_non_null/bloom_filter                    1.00    106.6±0.34ms   412.9 MB/sec    1.00    107.0±0.41ms   411.2 MB/sec
primitive_non_null/cdc                             1.00     89.0±0.28ms   494.6 MB/sec    1.03     91.5±0.33ms   480.7 MB/sec
primitive_non_null/default                         1.00     67.0±0.22ms   656.7 MB/sec    1.01     67.9±0.25ms   648.3 MB/sec
primitive_non_null/parquet_2                       1.05     88.6±0.20ms   496.6 MB/sec    1.00     84.0±0.26ms   523.6 MB/sec
primitive_non_null/zstd                            1.00     97.9±0.19ms   449.2 MB/sec    1.01     98.6±0.24ms   446.3 MB/sec
primitive_non_null/zstd_parquet_2                  1.04    121.7±0.22ms   361.7 MB/sec    1.00    117.2±0.30ms   375.3 MB/sec
primitive_sparse_99pct_null/bloom_filter           1.00     12.2±0.18ms     3.6 GB/sec    1.02     12.4±0.16ms     3.5 GB/sec
primitive_sparse_99pct_null/cdc                    1.02     30.5±0.34ms  1471.3 MB/sec    1.00     30.0±0.35ms  1497.2 MB/sec
primitive_sparse_99pct_null/default                1.00     10.7±0.06ms     4.1 GB/sec    1.01     10.8±0.07ms     4.1 GB/sec
primitive_sparse_99pct_null/parquet_2              1.00     10.7±0.05ms     4.1 GB/sec    1.02     10.9±0.10ms     4.0 GB/sec
primitive_sparse_99pct_null/zstd                   1.00     14.1±0.13ms     3.1 GB/sec    1.01     14.3±0.10ms     3.1 GB/sec
primitive_sparse_99pct_null/zstd_parquet_2         1.00     12.6±0.08ms     3.5 GB/sec    1.01     12.7±0.10ms     3.4 GB/sec
short_string_non_null/bloom_filter                 1.02     27.2±0.05ms   441.1 MB/sec    1.00     26.6±0.13ms   451.9 MB/sec
short_string_non_null/cdc                          1.00     19.5±0.04ms   615.8 MB/sec    1.01     19.6±0.05ms   610.7 MB/sec
short_string_non_null/default                      1.00     15.4±0.04ms   778.5 MB/sec    1.01     15.6±0.07ms   768.4 MB/sec
short_string_non_null/parquet_2                    1.07     26.7±0.12ms   448.8 MB/sec    1.00     24.9±0.09ms   482.4 MB/sec
short_string_non_null/zstd                         1.00     35.6±0.11ms   337.1 MB/sec    1.00     35.8±0.10ms   335.6 MB/sec
short_string_non_null/zstd_parquet_2               1.07     29.6±0.11ms   405.2 MB/sec    1.00     27.8±0.06ms   432.2 MB/sec
string/bloom_filter                                1.03   230.3±24.68ms     2.2 GB/sec    1.00   223.5±23.54ms     2.3 GB/sec
string/cdc                                         1.00    202.3±1.01ms     2.5 GB/sec    1.00    202.8±0.96ms     2.5 GB/sec
string/default                                     1.13   136.8±25.68ms     3.7 GB/sec    1.00   120.9±20.19ms     4.2 GB/sec
string/parquet_2                                   1.00    101.9±6.29ms     5.0 GB/sec    1.11    113.1±0.32ms     4.5 GB/sec
string/zstd                                        1.00    430.1±2.55ms  1218.9 MB/sec    1.07   461.9±12.13ms  1134.9 MB/sec
string/zstd_parquet_2                              1.00    384.4±0.84ms  1363.7 MB/sec    1.00    383.1±3.30ms  1368.5 MB/sec
string_and_binary_view/bloom_filter                1.00     63.5±0.20ms   507.9 MB/sec    1.04     65.8±0.24ms   490.4 MB/sec
string_and_binary_view/cdc                         1.00     58.7±0.32ms   549.2 MB/sec    1.02     60.1±0.22ms   536.6 MB/sec
string_and_binary_view/default                     1.00     48.0±0.13ms   672.3 MB/sec    1.03     49.3±0.17ms   653.9 MB/sec
string_and_binary_view/parquet_2                   1.01     59.8±0.17ms   539.2 MB/sec    1.00     59.3±0.16ms   544.1 MB/sec
string_and_binary_view/zstd                        1.00     85.6±0.30ms   376.8 MB/sec    1.01     86.3±0.19ms   373.6 MB/sec
string_and_binary_view/zstd_parquet_2              1.01     73.9±0.29ms   436.6 MB/sec    1.00     73.1±0.18ms   441.0 MB/sec
string_dictionary/bloom_filter                     1.00     84.1±0.46ms     3.1 GB/sec    1.02     86.1±0.73ms     3.0 GB/sec
string_dictionary/cdc                              1.00     48.2±0.68ms     5.4 GB/sec    1.03     49.5±0.48ms     5.2 GB/sec
string_dictionary/default                          1.00     43.2±0.30ms     6.0 GB/sec    1.03     44.5±0.39ms     5.8 GB/sec
string_dictionary/parquet_2                        1.02     48.7±0.11ms     5.3 GB/sec    1.00     47.8±0.12ms     5.4 GB/sec
string_dictionary/zstd                             1.00    209.4±1.16ms  1261.5 MB/sec    1.00    209.5±0.59ms  1260.7 MB/sec
string_dictionary/zstd_parquet_2                   1.00    192.0±0.19ms  1375.9 MB/sec    1.00    192.3±0.27ms  1373.4 MB/sec
string_non_null/bloom_filter                       1.01   233.0±10.62ms     2.2 GB/sec    1.00   230.7±10.62ms     2.2 GB/sec
string_non_null/cdc                                1.01    238.6±1.30ms     2.1 GB/sec    1.00    236.8±1.20ms     2.2 GB/sec
string_non_null/default                            1.00   116.7±12.55ms     4.4 GB/sec    1.01    117.3±9.64ms     4.4 GB/sec
string_non_null/parquet_2                          1.18    122.3±2.32ms     4.2 GB/sec    1.00    104.0±1.33ms     4.9 GB/sec
string_non_null/zstd                               1.04    557.1±3.70ms   940.5 MB/sec    1.00    534.8±1.86ms   979.8 MB/sec
string_non_null/zstd_parquet_2                     1.04    505.6±7.04ms  1036.4 MB/sec    1.00    485.4±2.16ms  1079.5 MB/sec
string_ree/bloom_filter                            1.00    147.7±0.74ms   111.1 MB/sec    1.02    150.8±0.57ms   108.8 MB/sec
string_ree/cdc                                     1.00    218.8±0.85ms    75.0 MB/sec    1.01    221.2±0.83ms    74.2 MB/sec
string_ree/default                                 1.00    120.7±0.50ms   135.9 MB/sec    1.02    123.0±0.66ms   133.4 MB/sec
string_ree/parquet_2                               1.00    173.3±0.47ms    94.7 MB/sec    1.01    175.1±0.34ms    93.7 MB/sec
string_ree/zstd                                    1.00    180.7±0.68ms    90.8 MB/sec    1.01    181.9±0.50ms    90.2 MB/sec
string_ree/zstd_parquet_2                          1.00    190.0±0.45ms    86.4 MB/sec    1.00    190.0±0.46ms    86.4 MB/sec
string_ree_95pct_null/bloom_filter                 1.01     26.7±0.08ms    52.7 MB/sec    1.00     26.5±0.14ms    53.1 MB/sec
string_ree_95pct_null/cdc                          1.01     32.9±0.14ms    42.8 MB/sec    1.00     32.7±0.13ms    43.0 MB/sec
string_ree_95pct_null/default                      1.00     25.0±0.05ms    56.2 MB/sec    1.00     24.9±0.09ms    56.4 MB/sec
string_ree_95pct_null/parquet_2                    1.00     25.0±0.05ms    56.2 MB/sec    1.00     24.9±0.08ms    56.4 MB/sec
string_ree_95pct_null/zstd                         1.00     25.9±0.08ms    54.4 MB/sec    1.00     25.7±0.09ms    54.6 MB/sec
string_ree_95pct_null/zstd_parquet_2               1.00     25.8±0.06ms    54.5 MB/sec    1.00     25.8±0.07ms    54.5 MB/sec
struct_all_null/bloom_filter                       1.00    369.9±1.92µs    42.6 GB/sec    1.00    370.2±2.04µs    42.5 GB/sec
struct_all_null/cdc                                1.00      7.8±0.07ms     2.0 GB/sec    1.00      7.8±0.14ms     2.0 GB/sec
struct_all_null/default                            1.01    116.5±0.52µs   135.2 GB/sec    1.00    115.6±0.57µs   136.2 GB/sec
struct_all_null/parquet_2                          1.00    115.5±0.58µs   136.3 GB/sec    1.01    116.1±0.55µs   135.6 GB/sec
struct_all_null/zstd                               1.01    180.9±0.53µs    87.0 GB/sec    1.00    179.4±0.53µs    87.8 GB/sec
struct_all_null/zstd_parquet_2                     1.00    157.0±0.49µs   100.3 GB/sec    1.01    158.4±0.68µs    99.4 GB/sec
struct_non_null/bloom_filter                       1.00     45.0±0.13ms   355.2 MB/sec    1.03     46.5±0.15ms   344.4 MB/sec
struct_non_null/cdc                                1.00     45.0±0.18ms   355.7 MB/sec    1.02     46.0±0.18ms   348.0 MB/sec
struct_non_null/default                            1.00     31.4±0.11ms   509.6 MB/sec    1.04     32.7±0.14ms   488.6 MB/sec
struct_non_null/parquet_2                          1.03     40.1±0.11ms   398.8 MB/sec    1.00     39.1±0.12ms   408.8 MB/sec
struct_non_null/zstd                               1.00     40.0±0.11ms   400.0 MB/sec    1.03     41.2±0.15ms   388.0 MB/sec
struct_non_null/zstd_parquet_2                     1.01     53.8±0.14ms   297.4 MB/sec    1.00     53.0±0.14ms   301.7 MB/sec
struct_sparse_99pct_null/bloom_filter              1.00      6.5±0.07ms     2.4 GB/sec    1.01      6.6±0.07ms     2.4 GB/sec
struct_sparse_99pct_null/cdc                       1.03     13.9±0.13ms  1156.9 MB/sec    1.00     13.5±0.14ms  1196.7 MB/sec
struct_sparse_99pct_null/default                   1.00      6.0±0.05ms     2.6 GB/sec    1.00      6.0±0.03ms     2.6 GB/sec
struct_sparse_99pct_null/parquet_2                 1.00      6.0±0.05ms     2.6 GB/sec    1.01      6.0±0.03ms     2.6 GB/sec
struct_sparse_99pct_null/zstd                      1.00      7.4±0.06ms     2.1 GB/sec    1.00      7.4±0.03ms     2.1 GB/sec
struct_sparse_99pct_null/zstd_parquet_2            1.00      6.7±0.05ms     2.3 GB/sec    1.01      6.8±0.04ms     2.3 GB/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	2960.7s
Peak memory	3.0 GiB
Avg memory	2.7 GiB
CPU user	2877.4s
CPU sys	77.9s
Peak spill	0 B

branch

Metric	Value
Wall time	2950.6s
Peak memory	2.9 GiB
Avg memory	2.7 GiB
CPU user	2881.3s
CPU sys	67.2s
Peak spill	0 B

File an issue against this benchmark runner

sdf-jkl · 2026-07-27T21:47:22Z

run benchmark arrow_writer

adriangbot · 2026-07-27T21:49:58Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5097175518-1278-2jpd7 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing put_batch (704d990) to 25569a3 (merge-base) diff
BENCH_NAME=arrow_writer
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_writer
BENCH_FILTER=
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-07-27T23:28:39Z

🤖 Arrow criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                              main                                   put_batch
-----                                              ----                                   ---------
bool/bloom_filter                                  1.04     12.9±0.05ms    19.3 MB/sec    1.00     12.4±0.06ms    20.1 MB/sec
bool/cdc                                           1.04     15.8±0.08ms    15.8 MB/sec    1.00     15.2±0.06ms    16.4 MB/sec
bool/default                                       1.05     10.8±0.06ms    23.1 MB/sec    1.00     10.3±0.07ms    24.2 MB/sec
bool/parquet_2                                     1.20     14.6±0.06ms    17.1 MB/sec    1.00     12.2±0.08ms    20.6 MB/sec
bool/zstd                                          1.05     11.4±0.05ms    22.0 MB/sec    1.00     10.8±0.07ms    23.1 MB/sec
bool/zstd_parquet_2                                1.20     15.0±0.07ms    16.6 MB/sec    1.00     12.5±0.10ms    20.0 MB/sec
bool_non_null/bloom_filter                         1.10      7.1±0.03ms    17.7 MB/sec    1.00      6.5±0.03ms    19.4 MB/sec
bool_non_null/cdc                                  1.11      6.9±0.04ms    18.1 MB/sec    1.00      6.2±0.02ms    20.1 MB/sec
bool_non_null/default                              1.17      4.3±0.02ms    29.0 MB/sec    1.00      3.7±0.02ms    33.9 MB/sec
bool_non_null/parquet_2                            1.49      9.0±0.04ms    13.8 MB/sec    1.00      6.1±0.02ms    20.6 MB/sec
bool_non_null/zstd                                 1.16      4.7±0.02ms    26.7 MB/sec    1.00      4.0±0.02ms    31.0 MB/sec
bool_non_null/zstd_parquet_2                       1.45      9.4±0.04ms    13.2 MB/sec    1.00      6.5±0.02ms    19.3 MB/sec
bool_ree/bloom_filter                              1.02     30.5±0.08ms     7.0 MB/sec    1.00     30.0±0.10ms     7.1 MB/sec
bool_ree/cdc                                       1.01     31.8±0.12ms     6.7 MB/sec    1.00     31.4±0.13ms     6.8 MB/sec
bool_ree/default                                   1.02     27.8±0.10ms     7.7 MB/sec    1.00     27.3±0.11ms     7.8 MB/sec
bool_ree/parquet_2                                 1.04     29.4±0.09ms     7.2 MB/sec    1.00     28.2±0.31ms     7.6 MB/sec
bool_ree/zstd                                      1.02     28.4±0.09ms     7.5 MB/sec    1.00     27.9±0.15ms     7.6 MB/sec
bool_ree/zstd_parquet_2                            1.04     29.7±0.08ms     7.2 MB/sec    1.00     28.4±0.12ms     7.5 MB/sec
decimal/bloom_filter                               1.00    123.2±1.21ms   487.0 MB/sec    1.03    126.3±0.31ms   475.1 MB/sec
decimal/cdc                                        1.00    106.0±0.88ms   566.2 MB/sec    1.01    107.5±3.84ms   558.3 MB/sec
decimal/default                                    1.00     83.7±1.16ms   716.8 MB/sec    1.00     83.4±0.30ms   719.6 MB/sec
decimal/parquet_2                                  1.03    182.3±1.21ms   329.2 MB/sec    1.00    177.8±1.94ms   337.4 MB/sec
decimal/zstd                                       1.00     94.5±2.00ms   635.1 MB/sec    1.00     94.6±0.35ms   634.3 MB/sec
decimal/zstd_parquet_2                             1.02    201.3±1.76ms   298.0 MB/sec    1.00    196.6±1.27ms   305.2 MB/sec
fixed_size_binary_ree/bloom_filter                 1.01     59.1±0.29ms    17.0 MB/sec    1.00     58.6±1.07ms    17.2 MB/sec
fixed_size_binary_ree/cdc                          1.00     62.0±0.34ms    16.2 MB/sec    1.01     62.6±1.82ms    16.1 MB/sec
fixed_size_binary_ree/default                      1.01     52.2±0.32ms    19.3 MB/sec    1.00     52.0±1.49ms    19.4 MB/sec
fixed_size_binary_ree/parquet_2                    1.00     65.3±0.33ms    15.4 MB/sec    1.00     65.1±1.22ms    15.5 MB/sec
fixed_size_binary_ree/zstd                         1.01     58.0±0.28ms    17.4 MB/sec    1.00     57.3±0.18ms    17.6 MB/sec
fixed_size_binary_ree/zstd_parquet_2               1.00     65.9±0.30ms    15.3 MB/sec    1.00     65.9±1.11ms    15.3 MB/sec
float_with_nans/bloom_filter                       1.01     79.0±0.32ms   177.2 MB/sec    1.00     78.5±0.28ms   178.2 MB/sec
float_with_nans/cdc                                1.02     69.3±0.33ms   202.0 MB/sec    1.00     68.3±1.92ms   205.1 MB/sec
float_with_nans/default                            1.03     61.8±0.34ms   226.5 MB/sec    1.00     60.2±1.22ms   232.7 MB/sec
float_with_nans/parquet_2                          1.06     78.1±0.39ms   179.3 MB/sec    1.00     73.8±1.26ms   189.6 MB/sec
float_with_nans/zstd                               1.02    100.4±0.51ms   139.4 MB/sec    1.00     98.6±1.69ms   142.0 MB/sec
float_with_nans/zstd_parquet_2                     1.04    115.1±0.50ms   121.6 MB/sec    1.00    110.5±0.17ms   126.7 MB/sec
fsb/bloom_filter                                   1.00      8.1±0.04ms  1981.0 MB/sec    1.02      8.3±0.03ms  1942.1 MB/sec
fsb/cdc                                            1.00     10.5±0.06ms  1538.7 MB/sec    1.02     10.7±0.20ms  1502.4 MB/sec
fsb/default                                        1.00      7.0±0.03ms     2.2 GB/sec    1.01      7.1±0.04ms     2.2 GB/sec
fsb/parquet_2                                      1.00     13.2±0.09ms  1217.2 MB/sec    1.01     13.4±0.05ms  1204.4 MB/sec
fsb/zstd                                           1.00      9.5±0.04ms  1701.2 MB/sec    1.00      9.5±0.03ms  1695.6 MB/sec
fsb/zstd_parquet_2                                 1.00     14.1±0.07ms  1140.6 MB/sec    1.01     14.3±0.06ms  1126.2 MB/sec
int32_ree/bloom_filter                             1.00     40.6±0.24ms    10.0 MB/sec    1.01     40.9±0.26ms     9.9 MB/sec
int32_ree/cdc                                      1.00     40.3±0.13ms    10.1 MB/sec    1.01     40.5±0.14ms    10.0 MB/sec
int32_ree/default                                  1.00     36.5±0.08ms    11.2 MB/sec    1.01     36.7±0.14ms    11.1 MB/sec
int32_ree/parquet_2                                1.00     36.5±0.10ms    11.2 MB/sec    1.01     36.9±0.16ms    11.0 MB/sec
int32_ree/zstd                                     1.00     37.0±0.10ms    11.0 MB/sec    1.01     37.3±0.18ms    10.9 MB/sec
int32_ree/zstd_parquet_2                           1.00     37.0±0.11ms    11.0 MB/sec    1.01     37.3±0.18ms    10.9 MB/sec
int32_ree_95pct_null/bloom_filter                  1.00     21.9±0.04ms    18.6 MB/sec    1.01     22.1±0.07ms    18.4 MB/sec
int32_ree_95pct_null/cdc                           1.00     23.8±0.05ms    17.1 MB/sec    1.03     24.5±0.05ms    16.6 MB/sec
int32_ree_95pct_null/default                       1.00     21.5±0.03ms    18.9 MB/sec    1.00     21.6±0.04ms    18.8 MB/sec
int32_ree_95pct_null/parquet_2                     1.00     21.5±0.03ms    18.9 MB/sec    1.00     21.6±0.03ms    18.8 MB/sec
int32_ree_95pct_null/zstd                          1.00     21.8±0.03ms    18.7 MB/sec    1.00     21.8±0.03ms    18.6 MB/sec
int32_ree_95pct_null/zstd_parquet_2                1.00     21.7±0.02ms    18.7 MB/sec    1.00     21.8±0.03ms    18.7 MB/sec
large_string_non_null/bloom_filter                 1.00     72.8±0.24ms     3.4 GB/sec    1.01     73.7±0.13ms     3.4 GB/sec
large_string_non_null/cdc                          1.01    246.8±1.82ms  1037.3 MB/sec    1.00    245.3±0.89ms  1043.7 MB/sec
large_string_non_null/default                      1.00     54.8±0.13ms     4.6 GB/sec    1.02     55.7±0.12ms     4.5 GB/sec
large_string_non_null/parquet_2                    1.00     54.9±0.22ms     4.6 GB/sec    1.02     55.8±0.10ms     4.5 GB/sec
large_string_non_null/zstd                         1.00     55.1±0.26ms     4.5 GB/sec    1.01     55.8±0.08ms     4.5 GB/sec
large_string_non_null/zstd_parquet_2               1.01     56.6±1.45ms     4.4 GB/sec    1.00     55.9±0.10ms     4.5 GB/sec
list_nested/bloom_filter                           1.05    148.2±0.64ms   194.7 MB/sec    1.00    141.6±0.19ms   203.8 MB/sec
list_nested/cdc                                    1.05    164.1±0.23ms   175.9 MB/sec    1.00    155.6±0.17ms   185.5 MB/sec
list_nested/default                                1.08    133.8±0.70ms   215.7 MB/sec    1.00    124.2±0.16ms   232.3 MB/sec
list_nested/parquet_2                              1.09    146.6±0.65ms   196.8 MB/sec    1.00    134.4±0.36ms   214.7 MB/sec
list_nested/zstd                                   1.07    142.6±0.67ms   202.4 MB/sec    1.00    133.6±0.35ms   216.0 MB/sec
list_nested/zstd_parquet_2                         1.09    159.5±0.65ms   180.9 MB/sec    1.00    146.9±0.17ms   196.4 MB/sec
list_primitive/bloom_filter                        1.00    282.4±1.33ms  1930.9 MB/sec    1.10    311.3±8.03ms  1751.9 MB/sec
list_primitive/cdc                                 1.01    315.8±0.38ms  1726.9 MB/sec    1.00   314.0±10.59ms  1737.0 MB/sec
list_primitive/default                             1.00    203.3±2.13ms     2.6 GB/sec    1.10    224.6±8.32ms     2.4 GB/sec
list_primitive/parquet_2                           1.00    228.0±0.42ms     2.3 GB/sec    1.06    241.6±7.42ms     2.2 GB/sec
list_primitive/zstd                                1.00    456.7±2.03ms  1194.2 MB/sec    1.04    476.6±2.67ms  1144.2 MB/sec
list_primitive/zstd_parquet_2                      1.04    451.5±1.23ms  1208.0 MB/sec    1.00    433.6±7.97ms  1257.8 MB/sec
list_primitive_non_null/bloom_filter               1.00    358.1±3.79ms  1520.0 MB/sec    1.15   411.6±16.14ms  1322.3 MB/sec
list_primitive_non_null/cdc                        1.02    400.0±9.20ms  1360.7 MB/sec    1.00    393.3±9.25ms  1383.6 MB/sec
list_primitive_non_null/default                    1.00    226.8±3.10ms     2.3 GB/sec    1.22   277.5±10.86ms  1961.2 MB/sec
list_primitive_non_null/parquet_2                  1.04    258.9±1.59ms     2.1 GB/sec    1.00   248.9±27.18ms     2.1 GB/sec
list_primitive_non_null/zstd                       1.00    645.1±1.12ms   843.6 MB/sec    1.11    715.0±7.07ms   761.2 MB/sec
list_primitive_non_null/zstd_parquet_2             1.00    646.2±0.57ms   842.2 MB/sec    1.00   643.2±14.83ms   846.2 MB/sec
list_primitive_sparse_99pct_null/bloom_filter      1.00     12.7±0.06ms     2.9 GB/sec    1.02     12.9±0.06ms     2.8 GB/sec
list_primitive_sparse_99pct_null/cdc               1.00     24.2±0.08ms  1543.6 MB/sec    1.02     24.6±0.08ms  1520.8 MB/sec
list_primitive_sparse_99pct_null/default           1.00     12.4±0.05ms     3.0 GB/sec    1.02     12.6±0.05ms     2.9 GB/sec
list_primitive_sparse_99pct_null/parquet_2         1.00     12.4±0.04ms     3.0 GB/sec    1.02     12.6±0.05ms     2.9 GB/sec
list_primitive_sparse_99pct_null/zstd              1.00     14.2±0.05ms     2.6 GB/sec    1.01     14.4±0.05ms     2.5 GB/sec
list_primitive_sparse_99pct_null/zstd_parquet_2    1.00     12.5±0.05ms     2.9 GB/sec    1.01     12.7±0.06ms     2.9 GB/sec
list_struct_with_list/bloom_filter                 1.05    284.3±1.90ms   160.4 MB/sec    1.00    271.0±0.48ms   168.3 MB/sec
list_struct_with_list/cdc                          1.05    321.8±0.56ms   141.7 MB/sec    1.00    305.5±0.63ms   149.3 MB/sec
list_struct_with_list/default                      1.07    258.3±1.86ms   176.6 MB/sec    1.00    240.6±0.92ms   189.5 MB/sec
list_struct_with_list/parquet_2                    1.09    276.1±1.75ms   165.2 MB/sec    1.00    253.6±0.50ms   179.8 MB/sec
list_struct_with_list/zstd                         1.07    280.1±1.91ms   162.8 MB/sec    1.00    262.2±0.49ms   173.9 MB/sec
list_struct_with_list/zstd_parquet_2               1.08    299.1±1.87ms   152.4 MB/sec    1.00    276.6±0.48ms   164.8 MB/sec
primitive/bloom_filter                             1.10    157.8±2.42ms   284.4 MB/sec    1.00    143.7±0.49ms   312.2 MB/sec
primitive/cdc                                      1.05    161.4±1.29ms   278.1 MB/sec    1.00    153.1±0.48ms   293.1 MB/sec
primitive/default                                  1.08    120.4±1.23ms   372.7 MB/sec    1.00    111.6±0.41ms   402.2 MB/sec
primitive/parquet_2                                1.11    135.4±1.11ms   331.5 MB/sec    1.00    121.9±0.26ms   368.2 MB/sec
primitive/zstd                                     1.07    150.8±1.21ms   297.6 MB/sec    1.00    141.3±0.46ms   317.5 MB/sec
primitive/zstd_parquet_2                           1.09    168.4±0.90ms   266.5 MB/sec    1.00    154.8±0.43ms   289.9 MB/sec
primitive_all_null/bloom_filter                    1.00    879.2±2.59µs    49.8 GB/sec    1.00    879.4±4.27µs    49.8 GB/sec
primitive_all_null/cdc                             1.03     19.1±0.29ms     2.3 GB/sec    1.00     18.5±0.28ms     2.4 GB/sec
primitive_all_null/default                         1.06    266.7±1.09µs   164.3 GB/sec    1.00    252.4±1.68µs   173.6 GB/sec
primitive_all_null/parquet_2                       1.09    272.7±1.54µs   160.7 GB/sec    1.00    250.6±1.33µs   174.9 GB/sec
primitive_all_null/zstd                            1.02    414.5±1.21µs   105.7 GB/sec    1.00    405.8±1.26µs   108.0 GB/sec
primitive_all_null/zstd_parquet_2                  1.06    370.2±1.42µs   118.4 GB/sec    1.00    350.1±1.50µs   125.2 GB/sec
primitive_non_null/bloom_filter                    1.09    112.9±2.44ms   389.6 MB/sec    1.00    103.6±0.37ms   424.7 MB/sec
primitive_non_null/cdc                             1.02     89.5±0.30ms   491.5 MB/sec    1.00     87.4±0.36ms   503.3 MB/sec
primitive_non_null/default                         1.08     69.7±1.04ms   631.6 MB/sec    1.00     64.3±0.20ms   684.0 MB/sec
primitive_non_null/parquet_2                       1.14     91.1±1.07ms   483.0 MB/sec    1.00     80.0±0.29ms   550.2 MB/sec
primitive_non_null/zstd                            1.04    100.2±1.11ms   439.0 MB/sec    1.00     96.3±0.39ms   457.0 MB/sec
primitive_non_null/zstd_parquet_2                  1.08    122.5±0.74ms   359.1 MB/sec    1.00    113.1±0.30ms   389.1 MB/sec
primitive_sparse_99pct_null/bloom_filter           1.00     12.2±0.11ms     3.6 GB/sec    1.02     12.5±0.10ms     3.5 GB/sec
primitive_sparse_99pct_null/cdc                    1.00     29.7±0.32ms  1512.0 MB/sec    1.08     32.2±0.24ms  1395.7 MB/sec
primitive_sparse_99pct_null/default                1.00     10.7±0.05ms     4.1 GB/sec    1.02     10.9±0.04ms     4.0 GB/sec
primitive_sparse_99pct_null/parquet_2              1.00     10.7±0.05ms     4.1 GB/sec    1.02     10.9±0.03ms     4.0 GB/sec
primitive_sparse_99pct_null/zstd                   1.00     14.1±0.05ms     3.1 GB/sec    1.01     14.3±0.05ms     3.1 GB/sec
primitive_sparse_99pct_null/zstd_parquet_2         1.00     12.6±0.06ms     3.5 GB/sec    1.02     12.8±0.05ms     3.4 GB/sec
short_string_non_null/bloom_filter                 1.00     28.1±0.07ms   427.2 MB/sec    1.00     28.0±0.06ms   429.2 MB/sec
short_string_non_null/cdc                          1.01     19.8±0.05ms   607.2 MB/sec    1.00     19.5±0.06ms   614.5 MB/sec
short_string_non_null/default                      1.00     15.5±0.06ms   772.7 MB/sec    1.00     15.5±0.05ms   775.7 MB/sec
short_string_non_null/parquet_2                    1.11     26.7±0.10ms   448.9 MB/sec    1.00     24.0±0.05ms   499.7 MB/sec
short_string_non_null/zstd                         1.00     35.8±0.10ms   335.6 MB/sec    1.00     35.8±0.10ms   335.5 MB/sec
short_string_non_null/zstd_parquet_2               1.10     29.7±0.12ms   404.6 MB/sec    1.00     27.0±0.06ms   445.2 MB/sec
string/bloom_filter                                1.01    209.0±4.87ms     2.5 GB/sec    1.00   207.6±19.24ms     2.5 GB/sec
string/cdc                                         1.00   211.2±10.37ms     2.4 GB/sec    1.00   210.6±11.40ms     2.4 GB/sec
string/default                                     1.00   120.0±16.59ms     4.3 GB/sec    1.06   126.8±21.96ms     4.0 GB/sec
string/parquet_2                                   1.18    120.0±0.59ms     4.3 GB/sec    1.00    101.9±7.60ms     5.0 GB/sec
string/zstd                                        1.07   474.5±19.08ms  1104.9 MB/sec    1.00    442.4±6.48ms  1184.9 MB/sec
string/zstd_parquet_2                              1.17    458.4±0.60ms  1143.6 MB/sec    1.00    391.3±8.82ms  1339.7 MB/sec
string_and_binary_view/bloom_filter                1.01     65.4±0.23ms   493.3 MB/sec    1.00     64.9±0.25ms   497.1 MB/sec
string_and_binary_view/cdc                         1.03     59.4±0.20ms   543.3 MB/sec    1.00     57.8±0.18ms   557.9 MB/sec
string_and_binary_view/default                     1.04     49.0±0.56ms   658.8 MB/sec    1.00     47.3±0.16ms   682.2 MB/sec
string_and_binary_view/parquet_2                   1.08     60.8±0.20ms   530.1 MB/sec    1.00     56.3±0.23ms   572.9 MB/sec
string_and_binary_view/zstd                        1.02     85.8±0.22ms   375.8 MB/sec    1.00     84.1±0.18ms   383.3 MB/sec
string_and_binary_view/zstd_parquet_2              1.07     74.6±0.16ms   432.2 MB/sec    1.00     70.0±0.16ms   460.4 MB/sec
string_dictionary/bloom_filter                     1.02     88.0±0.33ms     2.9 GB/sec    1.00     86.1±0.49ms     3.0 GB/sec
string_dictionary/cdc                              1.02     48.6±0.28ms     5.3 GB/sec    1.00     47.6±0.69ms     5.4 GB/sec
string_dictionary/default                          1.06     44.6±0.21ms     5.8 GB/sec    1.00     42.2±1.15ms     6.1 GB/sec
string_dictionary/parquet_2                        1.04     49.2±0.15ms     5.2 GB/sec    1.00     47.1±0.26ms     5.5 GB/sec
string_dictionary/zstd                             1.01    209.5±0.49ms  1260.5 MB/sec    1.00    207.2±1.37ms  1274.7 MB/sec
string_dictionary/zstd_parquet_2                   1.01    192.5±0.30ms  1371.9 MB/sec    1.00    191.2±0.71ms  1381.6 MB/sec
string_non_null/bloom_filter                       1.02   250.6±21.81ms     2.0 GB/sec    1.00   245.4±22.35ms     2.1 GB/sec
string_non_null/cdc                                1.09    256.1±3.58ms  2046.0 MB/sec    1.00    235.9±1.79ms     2.2 GB/sec
string_non_null/default                            1.08   137.4±25.23ms     3.7 GB/sec    1.00   127.4±20.03ms     4.0 GB/sec
string_non_null/parquet_2                          1.01    118.3±5.26ms     4.3 GB/sec    1.00    116.7±6.21ms     4.4 GB/sec
string_non_null/zstd                               1.02    553.1±2.46ms   947.3 MB/sec    1.00    542.9±1.17ms   965.3 MB/sec
string_non_null/zstd_parquet_2                     1.04    506.0±2.98ms  1035.6 MB/sec    1.00    487.4±5.61ms  1075.1 MB/sec
string_ree/bloom_filter                            1.00    156.6±0.52ms   104.8 MB/sec    1.00    156.0±2.03ms   105.2 MB/sec
string_ree/cdc                                     1.00    225.5±1.37ms    72.8 MB/sec    1.00    225.4±0.63ms    72.8 MB/sec
string_ree/default                                 1.02    128.7±0.53ms   127.5 MB/sec    1.00    126.1±2.11ms   130.1 MB/sec
string_ree/parquet_2                               1.01    180.8±0.43ms    90.8 MB/sec    1.00    179.4±0.51ms    91.5 MB/sec
string_ree/zstd                                    1.00    187.9±0.68ms    87.3 MB/sec    1.00    188.5±2.68ms    87.1 MB/sec
string_ree/zstd_parquet_2                          1.01    196.3±0.52ms    83.6 MB/sec    1.00    194.5±0.79ms    84.3 MB/sec
string_ree_95pct_null/bloom_filter                 1.00     26.5±0.09ms    53.2 MB/sec    1.00     26.3±0.14ms    53.4 MB/sec
string_ree_95pct_null/cdc                          1.00     32.3±0.07ms    43.6 MB/sec    1.00     32.4±0.09ms    43.4 MB/sec
string_ree_95pct_null/default                      1.01     24.9±0.06ms    56.5 MB/sec    1.00     24.7±0.09ms    57.0 MB/sec
string_ree_95pct_null/parquet_2                    1.01     24.9±0.06ms    56.6 MB/sec    1.00     24.7±0.09ms    56.9 MB/sec
string_ree_95pct_null/zstd                         1.01     25.7±0.07ms    54.7 MB/sec    1.00     25.5±0.10ms    55.1 MB/sec
string_ree_95pct_null/zstd_parquet_2               1.00     25.5±0.08ms    55.1 MB/sec    1.00     25.6±0.15ms    55.0 MB/sec
struct_all_null/bloom_filter                       1.00    366.5±2.12µs    43.0 GB/sec    1.01    368.4±6.18µs    42.7 GB/sec
struct_all_null/cdc                                1.03      7.8±0.09ms     2.0 GB/sec    1.00      7.5±0.09ms     2.1 GB/sec
struct_all_null/default                            1.05    114.4±0.52µs   137.6 GB/sec    1.00    108.9±0.81µs   144.6 GB/sec
struct_all_null/parquet_2                          1.08    116.5±0.64µs   135.2 GB/sec    1.00    107.8±0.65µs   146.1 GB/sec
struct_all_null/zstd                               1.03    178.4±0.49µs    88.3 GB/sec    1.00    173.3±0.74µs    90.9 GB/sec
struct_all_null/zstd_parquet_2                     1.05    158.5±0.61µs    99.4 GB/sec    1.00    151.0±0.55µs   104.3 GB/sec
struct_non_null/bloom_filter                       1.00     44.6±0.11ms   359.1 MB/sec    1.04     46.5±0.13ms   344.4 MB/sec
struct_non_null/cdc                                1.00     44.6±0.13ms   358.6 MB/sec    1.02     45.7±1.69ms   350.3 MB/sec
struct_non_null/default                            1.00     31.3±0.10ms   511.1 MB/sec    1.00     31.2±0.10ms   512.2 MB/sec
struct_non_null/parquet_2                          1.07     40.0±0.08ms   399.6 MB/sec    1.00     37.5±0.12ms   426.4 MB/sec
struct_non_null/zstd                               1.00     39.8±0.09ms   402.0 MB/sec    1.00     39.9±0.10ms   400.7 MB/sec
struct_non_null/zstd_parquet_2                     1.05     53.9±0.21ms   297.1 MB/sec    1.00     51.3±0.19ms   311.7 MB/sec
struct_sparse_99pct_null/bloom_filter              1.00      6.5±0.03ms     2.4 GB/sec    1.03      6.6±0.05ms     2.4 GB/sec
struct_sparse_99pct_null/cdc                       1.00     13.4±0.10ms  1202.1 MB/sec    1.10     14.7±0.09ms  1097.1 MB/sec
struct_sparse_99pct_null/default                   1.00      5.9±0.02ms     2.7 GB/sec    1.03      6.1±0.02ms     2.6 GB/sec
struct_sparse_99pct_null/parquet_2                 1.00      5.9±0.02ms     2.7 GB/sec    1.02      6.1±0.03ms     2.6 GB/sec
struct_sparse_99pct_null/zstd                      1.00      7.4±0.04ms     2.1 GB/sec    1.01      7.5±0.08ms     2.1 GB/sec
struct_sparse_99pct_null/zstd_parquet_2            1.00      6.8±0.03ms     2.3 GB/sec    1.01      6.8±0.03ms     2.3 GB/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	2940.6s
Peak memory	2.9 GiB
Avg memory	2.7 GiB
CPU user	2844.3s
CPU sys	89.8s
Peak spill	0 B

branch

Metric	Value
Wall time	2900.6s
Peak memory	2.8 GiB
Avg memory	2.6 GiB
CPU user	2779.5s
CPU sys	119.0s
Peak spill	0 B

File an issue against this benchmark runner

simdize bit_packing and move it under the same trait as unpack

aac9013

github-actions Bot added the parquet Changes to the parquet crate label Jul 24, 2026

fmt

f462d31

sdf-jkl added 2 commits July 27, 2026 17:46

restructure RLE to support put_batch

a276281

Merge branch 'main' of https://github.com/apache/arrow-rs into put_batch

704d990

fmt

978803c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Parquet] Vectorise Pack Bits within BitWriter - #10432

[Parquet] Vectorise Pack Bits within BitWriter#10432
sdf-jkl wants to merge 5 commits into
apache:mainfrom
sdf-jkl:put_batch

sdf-jkl commented Jul 24, 2026

Uh oh!

sdf-jkl commented Jul 25, 2026

Uh oh!

adriangbot commented Jul 25, 2026

Uh oh!

adriangbot commented Jul 25, 2026

Uh oh!

sdf-jkl commented Jul 27, 2026

Uh oh!

adriangbot commented Jul 27, 2026

Uh oh!

adriangbot commented Jul 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

sdf-jkl commented Jul 24, 2026

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

sdf-jkl commented Jul 25, 2026

Uh oh!

adriangbot commented Jul 25, 2026

Uh oh!

adriangbot commented Jul 25, 2026

Uh oh!

sdf-jkl commented Jul 27, 2026

Uh oh!

adriangbot commented Jul 27, 2026

Uh oh!

adriangbot commented Jul 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants