Add CLI option to set target-time parameter for batch measurement loop - #439
Open
oleksandr-pavlyk wants to merge 12 commits into
Open
Add CLI option to set target-time parameter for batch measurement loop#439oleksandr-pavlyk wants to merge 12 commits into
oleksandr-pavlyk wants to merge 12 commits into
Conversation
The option specifies target accumulated GPU time for batched measurements. This option is used to determine batch size and to decide when batched measurements loop terminates. (total accummulated time must exceed the target time and the number of launches executed should >= min_samples).
If time estimate results in batch size smaller than m_min_samples, use that instead so that batch does not degenerated into single launch and one does not observe the benefit of using warm L2-cache. Setting it this way permits batched measurement to complete in a single iteration of out loop. Also, update batch_size only after exit condition has been checked, and we know that m_batch_target_time is greater than m_total_cuda_time. Handle the possibility of `m_total_cuda_time` being zero.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
📝 WalkthroughSummary by CodeRabbit
WalkthroughAdds configurable batch target time for GPU measurements through C++ and Python APIs, CLI parsing, JSON serialization, hot-measurement scheduling, documentation, and tests. ChangesBatched measurements
Assessment against linked issues
Comment |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
When an unknown CLI option is encountered, we try it as a stopping criterion parameter. The error message now states the option that tripped the error, active stopping criterion and its parameters
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Remove dedicated validation in option parser, a rely on validation in benchmark setter method instead.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
The floor existed to prevent batch-size from degenerating to 1, which we now cap at 4 launches, so use of time floor is unnecessary. Removed no-cold measurements overwrite of batch target time, as it leads to non-transparent to the user significant increase in the target time used (say target time is default at 0.5 seconds, and timeout is default at 15 seconds, the overwrite would bump target time to 2.5 seconds). Present day NVBench only starts measure_hot after measure_cold has completed by default. If test is skipped/encountered errors during cold measurement loop, the measure_hot is not reached. The only way for a user to do batched measurements with cold measurments unavailable would be to use state::exec(nvbench::exec_tag::hot, launchable); But even in this case we want to honor requested `--batch-target-time` value.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
oleksandr-pavlyk
marked this pull request as ready for review
July 28, 2026 15:29
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Also test --batch-target-time nan
oleksandr-pavlyk
force-pushed
the
support-batch-target-time
branch
from
July 28, 2026 16:01
8708b47 to
ece54e4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
new
--batch-target-timeCLI optionThis PR adds support for
--batch-target-timeCLI option:This option controls batch size in batch measurement loop by specifying its target duration.
Batch measurement loop termination condition did not change: both the number of launches executed and the time executing them should exceed thresholds:
num_samples >= m_min_samples && total_cuda_time >= m_batch_target_time.Improved error on unrecognized CLI option
An error handling of unrecognized CLI options is improved:
Change in behavior of batch measurements when no prior cold measurements exist
We used to overwrite the value like so:
m_min_time = std::max( std::midpoint(m_min_time, m_timeout), m_min_time * 5);.In default settings (min-time of 0.5 seconds and timeout of 15 seconds) this would change value to 2.5 seconds.
Given then user may hit this branch by using
state::exec(exec_tag::hot, launchable);in the benchmark generator, this behavior is very surprising when--batch-target-timeis explicitly specified.For this reason, and because the option is now explicit, this overwrite has been removed.
Relationship to other work
This PR is an alternative to #424. The advantages of this solutions are:
--batch-target-timeoption is global, i.e., it is valid irrespective of which stopping criterion is considered active by option parser.--min-timeoption continues on as stdrel-criterion parameter.Closes #408