Skip to content

Add parquet benchmark for output_dict_columns option - #23596

Open
y2kiran wants to merge 1 commit into
NVIDIA:mainfrom
y2kiran:ykiran-dict-transcode-bench
Open

Add parquet benchmark for output_dict_columns option#23596
y2kiran wants to merge 1 commit into
NVIDIA:mainfrom
y2kiran:ykiran-dict-transcode-bench

Conversation

@y2kiran

@y2kiran y2kiran commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Description

This PR adds benchmarks to test the newly added output_dict_columns options for the Parquet reader, which was introduced in this PR

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@y2kiran
y2kiran requested review from a team as code owners August 7, 2026 22:08
@github-actions github-actions Bot added libcudf Affects libcudf (C++/CUDA) code. CMake CMake build issue labels Aug 7, 2026
@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Summary by CodeRabbit

  • New Features
    • Added a Parquet dictionary-transcoding benchmark.
    • Compares string materialization, dictionary encoding, and direct dictionary transcoding.
    • Reports timing, memory usage, throughput, file size, and direct-transcoding speedups across configurable data and I/O scenarios.

Walkthrough

Changes

Parquet dictionary benchmark

Layer / File(s) Summary
Benchmark modes and result reporting
cpp/benchmarks/io/parquet/parquet_reader_dict.cpp
Defines three read modes, dictionary-width estimation, run configuration, timing collection, and comparison output.
Fixture generation and mode execution
cpp/benchmarks/io/parquet/parquet_reader_dict.cpp
Generates configurable dictionary-encoded Parquet data and runs string, dictionary-encoded, and direct-transcoding reads.
Validation, metrics, and target wiring
cpp/benchmarks/io/parquet/parquet_reader_dict.cpp, cpp/benchmarks/CMakeLists.txt
Validates output types, skips plain-encoded direct-transcode cases, records performance metrics, configures benchmark axes, and adds the source to the NvBench target.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

  • rapidsai/cudf#22532: Introduces the same Parquet dictionary reader benchmark implementation and CMake wiring.

Suggested labels: improvement

Suggested reviewers: pointkernel, mhaseeb123, pmattione-nvidia, davidwendt, devavret

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title accurately identifies the main change as adding a Parquet benchmark, although it does not specify the reader dictionary-transcoding focus.
Description check ✅ Passed The description directly explains that the pull request adds benchmarks for the Parquet reader's output_dict_columns options.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@vuule vuule left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add this to parquet_reader_options.cpp and follow the existing benchmark patterns there? Since this measures output_dict_columns, keeping it with the other reader-options benchmarks would make it easier to find and maintain.
Could we also make the PR title more specific, for example: “Add benchmark for Parquet output_dict_columns”?

@vuule vuule added tests Unit testing for project improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Aug 12, 2026
@y2kiran y2kiran changed the title Add parquet benchmark Add parquet benchmark for output_dict_columns option Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CMake CMake build issue improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change tests Unit testing for project

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants