Skip to content

TCP N5a: Add the read-side mirror of WritableElementTypes to IColumnCodec - #548

Draft
alex-clickhouse wants to merge 2 commits into
tcp/epic-n1-clientfrom
tcp/epic-n5-poco
Draft

TCP N5a: Add the read-side mirror of WritableElementTypes to IColumnCodec#548
alex-clickhouse wants to merge 2 commits into
tcp/epic-n1-clientfrom
tcp/epic-n5-poco

Conversation

@alex-clickhouse

@alex-clickhouse alex-clickhouse commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #462 (tcp/epic-n1-client) — review that first; this PR's diff is the last commit only.

First step of the Branch 2 POCO epic (N5a/N5/N9), landed on its own because it changes an internal interface every codec implements and nothing consumes it yet.

Why

A codec already declares what it accepts on the write path — WritableElementTypes (preference-ordered), CanWrite as the membership test, NullPlaceholderAs, and the type switch inside WriteColumn. DateTimeColumnCodec even already holds the TimeZoneInfo and the ToUtc/ToUnixSeconds helpers.

The read path had no counterpart, because a codec decodes to exactly one canonical type — and for the date/time family that type is the raw wire count:

type canonical IColumn<T>
DateTime uint (epoch seconds)
DateTime64(s) long (raw count at scale)
Time / Time64(s) int / long

So public DateTime CreatedAt { get; set; } — the most common POCO member there is — has no match at all. Rather than invent a third home for knowledge the write path already keeps on the codec, this adds the mirror.

What

  • ReadableElementTypes — defaults to [ElementType], mirroring WritableElementTypes.
  • ProjectRead(Expression value, Type targetType) — mirrors WriteColumn's type switch. An Expression, not a Func<,>, so the compiled per-column read loop the POCO reader will emit can inline the conversion; a delegate would cost an indirect call per row. Scale and timezone are embedded as constants, so the result closes over nothing.

Overridden by the four raw-count codecs. Nullable and LowCardinality delegate to their inner codec and lift through Nullable<> where the surface wrapped the inner type.

The conversions move into a new ColumnValueProjections that DateTimeColumn, DateTime64Column, TimeColumn and Time64Column now share instead of each keeping a private copy, so a raw count has one calendar reading whichever surface asks for it.

Deliberately excluded

Enum. EnumColumnCodec<T> does not override WritableElementTypes either — write takes only the raw ordinal — so offering a string label on read would be a read-only asymmetry rather than a mirror. Pinned by a test so adding it later is a visible choice. Label mapping stays possible (the label maps are reachable only from the codec) but is now its own decision.

Composites. Array/Map/Tuple/Nested do not lift their children's readable types, so Array(DateTime) reads only as uint[]. This is faithful: their write contracts do not lift either (ArrayColumnCodec.CanWrite is column is IColumn<TElement[]> over the child's canonical ElementType), so the gap is symmetric today. Lifting it is not a small extension of this contract — the readable set of a composite is the cartesian product of its children's, so Tuple of seven DateTime64s would enumerate 3^7 entries. See the discussion below.

DateTime Kind semantics

PresentAsDateTime follows the HTTP driver's AbstractDateTimeType.ToDateTime: a zero offset yields Kind=Utc, any other offset the wall clock in the column's zone as Kind=Unspecified.

For a bare DateTime/DateTime64 the two clients deliberately differ: HTTP presents it in UTC because its type object carries no zone, whereas this client resolves session_timezone and so agrees with what the server itself would display. That was weighed and chosen — keeping the whole TCP read path honoring session_timezone beat cross-client parity, at the cost of a POCO moved from the HTTP client shifting by the session offset on bare columns. Columns whose type names a timezone match HTTP exactly. Recorded in the PresentAsDateTime doc comment so it is not later "fixed" toward HTTP.

Adjacent and pre-existing: reads use the session timezone while writes use ResolveContext.ForWrite, which carries none, so ToUtc reads an Unspecified DateTime as a UTC wall clock. A round trip through a bare column on a non-UTC session is therefore not the identity.

Tests

Nothing consumes this yet, so the tests carry the PR.

  • Unit (ColumnReadProjectionTests) — expected values derived independently, not from the implementation: 1700000000 = 2023-11-14T22:13:20Z, Berlin +01:00 in November and +02:00 for a July instant (a fixed-base-offset implementation passes the first and fails the second), scale-9 …123456789 truncating to .1234567, Time64(9) -1000000001-00:00:01 showing truncation toward zero.
  • Invariant sweep over 36 registered types: every codec leads its readable list with ElementType, lists no duplicate, and can actually project each type it advertises — so a codec cannot advertise a projection it does not have.
  • Single evaluation — the nullable lift splices its source twice (HasValue, Value), so it binds a local first. Verified by mutation: inlining the source twice fails this test.
  • Integration (ColumnReadProjectionIntegrationTests) — projections checked against a real server's own timezone and scale handling, including a DST instant and LowCardinality(Nullable(DateTime)).

Coverage: every changed file ≥89.8%, ColumnValueProjections and IColumnCodec at 100%. Full TCP suite 1394/1394 green.

Review note

A review pass caught a latent bug, fixed here: both wrappers computed the advertised type per read type but decided whether to lift by testing the inner codec's canonical type. Those agree only while every read type shares its value/reference-ness with ElementType — so the moment a reference-typed inner gained a value-typed reading (FixedString(16)Guid being the plausible one), Nullable(FixedString(16)) would advertise Guid? and return a Guid-typed expression, silently turning a null row into default(Guid). The discriminator is now the type being projected to.

Being straight about its test status: mutating that condition back leaves all tests passing, because no current type pair distinguishes the two. The guard test documents the invariant for when such a type is added rather than catching a live bug.

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds read-side type projections to TCP codecs, preparing POCO reads on top of #462.

Changes:

  • Adds ReadableElementTypes and expression-based ProjectRead.
  • Adds shared date/time projections and nullable/low-cardinality lifting.
  • Adds unit and server integration coverage.

The documented reference-source-to-nullable-value projection case remains broken and requires correction.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
Types/IColumnCodec.cs Defines the projection contract.
Types/ColumnValueProjections.cs Centralizes projection logic.
Types/DateTimeColumn.cs Reuses shared conversion.
Types/DateTime64Column.cs Reuses shared scaled conversion.
Types/TimeColumn.cs Reuses shared duration conversion.
Types/Time64Column.cs Reuses shared scaled conversion.
Codecs/DateTimeColumnCodec.cs Advertises and builds date projections.
Codecs/DateTime64ColumnCodec.cs Adds scaled date projections.
Codecs/TimeColumnCodec.cs Adds TimeSpan projection.
Codecs/Time64ColumnCodec.cs Adds scaled TimeSpan projection.
Codecs/NullableColumnCodec.cs Lifts inner readable types.
Codecs/LowCardinalityColumnCodec.cs Propagates inner projections.
ColumnReadProjectionTests.cs Tests contracts and expressions.
ColumnReadProjectionIntegrationTests.cs Validates projections against ClickHouse.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +178 to +180
return innerTarget.IsValueType
? ColumnValueProjections.ProjectNullable(value, inner, innerTarget, targetType)
: inner.ProjectRead(value, innerTarget);
Comment on lines +382 to +384
return nullable && innerTarget.IsValueType
? ColumnValueProjections.ProjectNullable(value, inner, innerTarget, targetType)
: inner.ProjectRead(value, innerTarget);
@codecov

codecov Bot commented Aug 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

A codec already declares which CLR types it accepts on the write path
(WritableElementTypes, CanWrite, NullPlaceholderAs), and converts them
inside WriteColumn. The read path had no counterpart: a codec decodes to
exactly one canonical type, and for the date/time family that type is the
raw wire count (DateTime is IColumn<uint>, DateTime64 is IColumn<long>).
So the most common POCO member type has no match at all.

Add the mirror rather than a third home for the same knowledge:

- ReadableElementTypes, defaulting to [ElementType], mirroring
  WritableElementTypes.
- ProjectRead(Expression value, Type targetType), mirroring WriteColumn's
  type switch. An expression, not a delegate, so a caller compiling a
  per-column read loop can inline the conversion instead of paying an
  indirect call per row. Per-column state (scale, timezone) is embedded as
  a constant, so the result closes over nothing.

Overridden by the four codecs whose canonical type is a raw count:
DateTime, DateTime64, Time, Time64. Nullable and LowCardinality delegate to
their inner codec and lift through Nullable<> where the surface wrapped the
inner type. Enum is deliberately excluded: it does not override
WritableElementTypes either, so offering a label on read would be a
read-only asymmetry rather than a mirror.

The conversions move into ColumnValueProjections, which the four columns
now share instead of each keeping a private copy, so a raw count has one
calendar reading whichever surface asks. TimeColumn's projection changes
spelling (TimeSpan.FromSeconds to FromTicks) and is value-identical over
the type's whole range.

The DateTime projection follows the HTTP driver's Kind rule: a zero offset
yields Kind=Utc, any other offset the wall clock as Unspecified. The two
clients still diverge for a bare DateTime, where HTTP presents in UTC and
this client resolves the session timezone; that is documented, not changed.

Nothing consumes this yet, so the tests carry it: the codecs' behavior is
pinned against independently derived constants, against a real server's own
timezone and scale handling, and by an invariant sweep asserting every
registered codec leads its readable list with ElementType and can project
each type it advertises.
The divergence from the HTTP driver on a zone-less DateTime is deliberate:
this client resolves session_timezone, so it agrees with what the server
would display, at the cost of a POCO moved from the HTTP client shifting by
the session offset. Say so, so it is not later 'fixed' toward HTTP.

Comment only; the behavior is unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants