feat(storage): add iteration metrics and CSV reporting to appendable upload benchmark

[vsharonlynn]

This PR comes after PR #6149.

Issue #5716.

[gemini-code-assist]

Code Review

This pull request introduces a new benchmark crate, storage-benchmark-appendable-object, to measure steady-state append performance on appendable objects. Feedback focuses on ensuring the scenarios and source modules are properly gated under the google_cloud_unstable_storage_bidi flag to prevent compilation failures, relaxing the minimum iteration limit for easier local testing, correcting the percentile index calculation, ensuring the output directory is created automatically, and removing an unnecessary clone of remainder_chunk.

[gemini-code-assist]

The scenarios and source modules depend on APIs that are only available when the google_cloud_unstable_storage_bidi configuration flag is enabled. Unconditionally compiling these modules will cause workspace compilation failures (e.g., during cargo check or cargo build) when the flag is not set. Gating these module declarations ensures the benchmark crate compiles successfully in all environments.

Suggested change

	mod scenarios;
	mod source;
	#[cfg(google_cloud_unstable_storage_bidi)]
	mod scenarios;
	#[cfg(google_cloud_unstable_storage_bidi)]
	mod source;

[gemini-code-assist]

Enforcing a hard minimum of 100 measured iterations makes local testing, dry-runs, or debugging of the benchmark extremely slow and expensive (since each iteration uploads 100 MiB by default, resulting in 10 GiB of data transfer). Consider relaxing or removing this restriction to allow developers to run quick sanity checks with fewer iterations.

Suggested change

	if args.measured_iterations < 100 {
	anyhow::bail!("Minimum number of measured iterations is 100");
	}
	if args.measured_iterations == 0 {
	anyhow::bail!("Number of measured iterations must be greater than 0");
	}

[gemini-code-assist]

Using (len as f64 * percentage).floor() to index percentiles can lead to inaccurate results. For example, with 100 elements, the 99th percentile (p99) is indexed at 100 * 0.99 = 99, which is the 100th element (the absolute maximum, i.e., p100). Using the standard formula ((len - 1) as f64 * percentage).round() provides a much more accurate percentile estimation.

Suggested change

	let p50 = latencies[(latencies.len() as f64 * 0.50).floor() as usize];
	let p90 = latencies[(latencies.len() as f64 * 0.90).floor() as usize];
	let p99 = latencies[(latencies.len() as f64 * 0.99).floor() as usize];
	let len = latencies.len();
	let p50 = latencies[((len - 1) as f64 * 0.50).round() as usize];
	let p90 = latencies[((len - 1) as f64 * 0.90).round() as usize];
	let p99 = latencies[((len - 1) as f64 * 0.99).round() as usize];

[gemini-code-assist]

If the specified output_dir does not exist, File::create will fail with a NotFound I/O error. Creating the directory automatically using std::fs::create_dir_all makes the benchmark more robust and user-friendly.

Suggested change

	let path = Path::new(&args.output_dir).join(format!(
	"scenario1_s{}_c{}_{}.csv",
	args.object_size, args.chunk_size, timestamp
	));
	let mut file = File::create(&path)?;
	let output_dir = Path::new(&args.output_dir);
	if !args.output_dir.is_empty() {
	std::fs::create_dir_all(output_dir)?;
	}
	let path = output_dir.join(format!(
	"scenario1_s{}_c{}_{}.csv",
	args.object_size, args.chunk_size, timestamp
	));
	let mut file = File::create(&path)?;

[gemini-code-assist]

The remainder_chunk is an owned Option<Bytes> that is not used anywhere else after this block. You can consume it directly without cloning the underlying Bytes.

Suggested change

	if let Some(ref rc) = remainder_chunk {
	writer.append(rc.clone()).await?;
	}
	if let Some(rc) = remainder_chunk {
	writer.append(rc).await?;
	}

References

1. Scrutinize expensive or unnecessary uses of clone(). Can we move the data instead? ^(link)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.57%. Comparing base (b3a930b) to head (0bba9b1).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #6170   +/-   ##
=======================================
  Coverage   96.57%   96.57%           
=======================================
  Files         264      264           
  Lines       66577    66577           
=======================================
  Hits        64297    64297           
  Misses       2280     2280           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.