feat(isthmus): map execution context variables to/from Calcite#976
Open
nielspardon wants to merge 1 commit into
Open
feat(isthmus): map execution context variables to/from Calcite#976nielspardon wants to merge 1 commit into
nielspardon wants to merge 1 commit into
Conversation
Add Isthmus (Substrait <-> Apache Calcite) mappings for the three execution context variables introduced in core by substrait-io#867: current_timestamp, current_date and current_timezone. Substrait -> Calcite (ExpressionRexConverter): three visit overrides build niladic RexCalls whose return type is forced from the Substrait type so time-zone semantics, precision and required-nullability round-trip. current_timestamp maps to CURRENT_TIMESTAMP typed TIMESTAMP_WITH_LOCAL_TIME_ZONE(p) (Calcite's operator would otherwise infer a timezone-less TIMESTAMP), current_date to CURRENT_DATE, and current_timezone to a new operator (below). Calcite -> Substrait: a new EXECUTION_CONTEXT_VARIABLE CallConverter matches these operators by identity and produces the corresponding Substrait expressions, registered in both CallConverters.defaults() and ConverterProvider.getCallConverters(). Session timezone: Calcite has no built-in operator for it, so add CurrentTimezoneFunction, a niladic SqlBaseContextVariable (modeled on CURRENT_USER/CURRENT_ROLE). It is registered in SubstraitOperatorTable via a separate scalar-operator table so it parses from SQL text without parentheses (through SqlValidatorImpl.makeNullaryCall) while avoiding polluting OVERRIDE_KINDS with its generic OTHER_FUNCTION kind. Tests: bidirectional expression<->rex round-trips for all three variables, plus SQL round-trips (including that current_timezone parses and is emitted back as the bare keyword).
5188814 to
36e5012
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Isthmus (Substrait ⇄ Apache Calcite) mappings for the three execution context variables introduced in
:coreby #867:current_timestamp,current_dateandcurrent_timezone. Previously any plan containing one of these failed to convert to/from Calcite.Changes
ExpressionRexConverter): threevisitoverrides build niladicRexCalls whose return type is forced from the Substrait type so time-zone semantics, precision and required-nullability round-trip.CurrentTimestamp(p)→CURRENT_TIMESTAMPtypedTIMESTAMP_WITH_LOCAL_TIME_ZONE(p)— Substrait'scurrent_timestampis aprecision_timestamp_tz, whereas Calcite's operator would otherwise infer a timezone-lessTIMESTAMP.CurrentDate→CURRENT_DATEtypedDATE.CurrentTimezone→ the newCurrentTimezoneFunctiontyped non-nullableVARCHAR.CallConverters.EXECUTION_CONTEXT_VARIABLEmatches these operators by identity and produces the corresponding Substrait expressions. Registered in bothCallConverters.defaults()andConverterProvider.getCallConverters().CurrentTimezoneFunction— a niladicSqlBaseContextVariable(modeled on Calcite's ownCURRENT_USER/CURRENT_ROLE). It is registered inSubstraitOperatorTablevia a separate scalar-operator table so that:SELECT current_timezoneparses from SQL text without parentheses (via Calcite'sSqlValidatorImpl.makeNullaryCall), andOTHER_FUNCTIONkind does not polluteOVERRIDE_KINDS(which would otherwise filter every standardOTHER_FUNCTIONoperator out of the operator list).Tests
ExecutionContextVariableConversionTest: bidirectional expression ⇄ Rex round-trips for all three variables.Substrait2SqlTest: full SQL round-trips forcurrent_timestamp,current_date, andcurrent_timezone, including asserting thatcurrent_timezoneparses and is emitted back as the bare keyword (no Calcite dialect changes required).Verification
:core:spotlessCheck,:isthmus:spotlessCheck, the full:isthmus:testsuite, and compilation of:spark:spark-3.5_2.12and:examples:substrait-sparkall pass.🤖 Generated with AI