Skip to content

feat(expr): evaluate prefix predicates with strict metrics - #2889

Open
fallintoplace wants to merge 2 commits into
apache:mainfrom
fallintoplace:feat-strict-prefix-metrics
Open

feat(expr): evaluate prefix predicates with strict metrics#2889
fallintoplace wants to merge 2 commits into
apache:mainfrom
fallintoplace:feat-strict-prefix-metrics

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

N/A. Found while comparing StrictMetricsEvaluator with the Java reference implementation and the sibling metrics evaluators.

What changes are included in this PR?

Strict metrics evaluation answers whether every row in a data file must match a predicate. It previously returned false for startsWith and notStartsWith even when file bounds proved the result.

This PR adds conservative proofs for both predicates:

  • startsWith returns true only when null metrics prove there are no nulls and both string bounds start with the requested prefix
  • notStartsWith returns true when the column is entirely null, or when a truncated lower or upper bound proves the requested prefix is outside the possible value range
  • missing or inconclusive metrics still return false
  • non-string literals or bounds return an error instead of being evaluated as valid string metrics

The bound comparisons use iterators and do not allocate temporary truncated strings.

Are these changes tested?

Yes. Unit tests cover matching and non-matching bounds, missing bounds, null-containing files, all-null files, prefix overlap, and invalid non-string bounds.

Validated with:

  • cargo test -p iceberg expr::visitors::strict_metrics_evaluator::test --lib
  • cargo clippy -p iceberg --lib -- -D warnings

@fallintoplace fallintoplace changed the title feat(expr): evaluate prefix predicates in strict metrics feat(expr): evaluate startsWith and notStartsWith in strict metrics Jul 25, 2026
@fallintoplace fallintoplace changed the title feat(expr): evaluate startsWith and notStartsWith in strict metrics feat(expr): evaluate prefix predicates with strict metrics Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant