Skip to content

[rust][tests] Add DynamicWriteBatchSizeEstimator integration tests#3588

Open
Prajwal-banakar wants to merge 3 commits into
apache:mainfrom
Prajwal-banakar:rust-integration-tests
Open

[rust][tests] Add DynamicWriteBatchSizeEstimator integration tests#3588
Prajwal-banakar wants to merge 3 commits into
apache:mainfrom
Prajwal-banakar:rust-integration-tests

Conversation

@Prajwal-banakar

@Prajwal-banakar Prajwal-banakar commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #fluss-rust/issues/539

Adds integration tests for DynamicWriteBatchSizeEstimator to verify end-to-end behavior through a real cluster. These tests exercise the estimator's adaptation logic — not just data integrity — by exposing the current estimated batch size via a feature-gated accessor.

Brief change log

  • Added crates/fluss/tests/integration/dynamic_batch_size.rs with 4 integration tests
  1. small_rows_shrink_batch_size — writes 200 tiny rows, then asserts estimated_batch_size_for_table() dropped below the initial max (256 KB), proving the estimator actually shrunk
  2. shrink_then_grow_batch_size — Phase 1 shrinks with tiny rows, Phase 2 sends ~200 KB rows to trigger growth; all records from both phases are read back to verify data integrity
  3. disabled_keeps_static_batch_size — writes 100 rows with writer_dynamic_batch_size_enabled = false; verifies all records round-trip correctly
  4. concurrent_appends_share_estimator_without_corruption — 4 tokio tasks share a single Arc from one connection, exercising concurrent access to the shared RecordAccumulator and estimator; all 200 records are read back
  • Added #[cfg(any(test, feature = "integration_tests"))] accessor estimated_batch_size() on RecordAccumulator
  • Added #[cfg(feature = "integration_tests")] method estimated_batch_size_for_table() on WriterClient and FlussConnection, wiring the estimator value up to integration test code without touching the public API
  • Registered the new module in crates/fluss/tests/test_fluss.rs

Tests

cargo fmt --all
cargo clippy --all-targets --fix --allow-dirty --allow-staged - BUILD SUCCESS
cargo build --features integration_tests - BUILD SUCCESS
cargo test --workspace - BUILD SUCCESS
cargo test --features integration_tests --workspace - BUILD SUCCESS

API and Format

No changes to public API or storage format.

Documentation

No new feature introduced. This PR only adds integration tests for an existing feature

@Prajwal-banakar

Copy link
Copy Markdown
Contributor Author

Hi @fresh-borzoni @charlesdong1991 i've moved this Pull Request from fluss-rust repo to here by addressing review comments, PTAL. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[rust][tests] Add DynamicWriteBatchSizeEstimator integration tests

1 participant