Skip to content

[AURON #2005] Upgrade orc to fix incorrect filter pushdown#2097

Open
Flyangz wants to merge 1 commit into
apache:masterfrom
Flyangz:bugfix/upgrade-orc
Open

[AURON #2005] Upgrade orc to fix incorrect filter pushdown#2097
Flyangz wants to merge 1 commit into
apache:masterfrom
Flyangz:bugfix/upgrade-orc

Conversation

@Flyangz

@Flyangz Flyangz commented Mar 17, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #2005 #2042

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

How was this patch tested?

unit and local test

@Flyangz Flyangz changed the title [AURON #2005 #2042] upgrade orc to fix incorrect filter pushdown [AURON #2005] upgrade orc to fix incorrect filter pushdown Mar 17, 2026
@SteNicholas

Copy link
Copy Markdown
Member

@Flyangz, could you please resolve conflicts?

test("test not null filter for orc table") {
withTable("orc_string_filter") {
sql("create table orc_string_filter(id int, b string) using orc")
sql("insert into orc_string_filter values (1, 'abc'), (2, null), (3, 'def')")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test reproduces #2005 (the Not / is not null row-group path), but the short values here ('abc', 'def') stay well under ORC's string-statistics truncation threshold, so they don't exercise the #2042 fix (#79, truncated string stats) — that path needs min/max stats long enough to get truncated. Since this PR closes #2042 as well, would it be worth adding a second case with long string values and a comparison/equality predicate, so the truncated-stats fix is locked in against regressions the same way the not-null case is?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added unit test for #2042

@SteNicholas SteNicholas changed the title [AURON #2005] upgrade orc to fix incorrect filter pushdown [AURON #2005] Upgrade orc to fix incorrect filter pushdown Jun 22, 2026
@Flyangz Flyangz force-pushed the bugfix/upgrade-orc branch from 0839a06 to bb779c0 Compare July 1, 2026 08:49
@Flyangz

Flyangz commented Jul 1, 2026

Copy link
Copy Markdown
Contributor Author

@Flyangz, could you please resolve conflicts?

@SteNicholas done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect Not predicate evaluation in ORC filtering pushdown

5 participants