feat(schema): refactor UpdateSchemaAction for schema evolution in Iceberg - #2451
Open
TwinklerG wants to merge 1 commit into
Open
feat(schema): refactor UpdateSchemaAction for schema evolution in Iceberg#2451TwinklerG wants to merge 1 commit into
UpdateSchemaAction for schema evolution in Iceberg#2451TwinklerG wants to merge 1 commit into
Conversation
TwinklerG
changed the title
UpdateSchemaAction for schema evolution in Iceberg
TwinklerG
changed the title
UpdateSchemaAction for schema evolution in IcebergUpdateSchemaAction for schema evolution in Iceberg
Collaborator
|
I'm not sure if this is the correct direction: it changes the update_schema API, and I don't think it adds much value for users/devs Could you share more about your reasoning here? |
Contributor
Author
Well, I have been continuously tracking the Issues #2119 #697 and the related PRs. I noticed that the merged PR #2120 only added the capabilities of adding column and deleting column, but this is obviously only a small part of the schema evolution, and I don't think the original implementation was a good one. Therefore, I imitated the implementation of iceberg java and completed this refactoring |
nazq
added a commit
to nazq/iceberg-rust
that referenced
this pull request
Jun 2, 2026
Extends UpdateSchemaAction with a rename(RenameColumn) builder method,
completing the add/delete/rename triad for non-incompatible schema
evolution.
API shape:
action.rename(
RenameColumn::builder().name("old").new_name("new").build(),
);
Mirrors the typed-builder shape used by apache#2451's broader schema-evolution
refactor, so callers adopting this PR today can migrate to apache#2451 with
no source change once it lands.
Renames preserve field IDs — only the leaf name changes. RenameColumn::name
uses SCHEMA_NAME_DELIMITER for nested fields (e.g. "person.name");
RenameColumn::new_name must be unqualified.
Ordering in commit():
1. deletes validated
2. renames validated — runs after deletes so a rename can re-use a
name being deleted, before additions so an addition can re-use a
name being renamed away
3. additions validated + ID-assigned
4. schema tree rebuilt with renames threaded through
rebuild_fields / rebuild_field
Validation rules (match pyiceberg's UpdateSchema.rename_column):
- missing source field → PreconditionFailed
- source field also staged for deletion → PreconditionFailed
- new_name contains SCHEMA_NAME_DELIMITER → PreconditionFailed
- new_name collides with a non-deleted, non-renamed sibling →
PreconditionFailed
- same field renamed twice → last rename wins
Identifier-field handling falls out for free — Rust keys identifier
fields by ID, not name, so with_identifier_field_ids(base_schema
.identifier_field_ids()) propagates the set unchanged across rename.
Tests: 10 new tests covering simple root rename, nested rename,
missing field, delete-conflict, sibling-collision, dotted-name
rejection, identifier-field preservation, rename-frees-old-name
(combined with add), repeated-rename-last-wins, no-op self-rename.
Full iceberg lib suite: 1306/1306 passing. Clippy + rustfmt clean.
nazq
added a commit
to nazq/iceberg-rust
that referenced
this pull request
Jun 2, 2026
Extends UpdateSchemaAction with a rename(RenameColumn) builder method,
completing the add/delete/rename triad for non-incompatible schema
evolution.
API shape:
action.rename(
RenameColumn::builder().name("old").new_name("new").build(),
);
Mirrors the typed-builder shape used by apache#2451's broader schema-evolution
refactor, so callers adopting this PR today can migrate to apache#2451 with
no source change once it lands.
Renames preserve field IDs — only the leaf name changes. RenameColumn::name
uses SCHEMA_NAME_DELIMITER for nested fields (e.g. "person.name");
RenameColumn::new_name must be unqualified.
Ordering in commit():
1. deletes validated
2. renames validated — runs after deletes so a rename can re-use a
name being deleted, before additions so an addition can re-use a
name being renamed away
3. additions validated + ID-assigned
4. schema tree rebuilt with renames threaded through
rebuild_fields / rebuild_field
Validation rules (match pyiceberg's UpdateSchema.rename_column):
- missing source field → PreconditionFailed
- source field also staged for deletion → PreconditionFailed
- new_name contains SCHEMA_NAME_DELIMITER → PreconditionFailed
- new_name collides with a non-deleted, non-renamed sibling →
PreconditionFailed
- same field renamed twice → last rename wins
Identifier-field handling falls out for free — Rust keys identifier
fields by ID, not name, so with_identifier_field_ids(base_schema
.identifier_field_ids()) propagates the set unchanged across rename.
Tests: 10 new tests covering simple root rename, nested rename,
missing field, delete-conflict, sibling-collision, dotted-name
rejection, identifier-field preservation, rename-frees-old-name
(combined with add), repeated-rename-last-wins, no-op self-rename.
Full iceberg lib suite: 1306/1306 passing. Clippy + rustfmt clean.
nazq
added a commit
to nazq/iceberg-rust
that referenced
this pull request
Jun 2, 2026
Extends UpdateSchemaAction with a rename(RenameColumn) builder method,
completing the add/delete/rename triad for non-incompatible schema
evolution.
API shape:
action.rename(
RenameColumn::builder().name("old").new_name("new").build(),
);
Mirrors the typed-builder shape used by apache#2451's broader schema-evolution
refactor, so callers adopting this PR today can migrate to apache#2451 with
no source change once it lands.
Renames preserve field IDs — only the leaf name changes. RenameColumn::name
uses SCHEMA_NAME_DELIMITER for nested fields (e.g. "person.name");
RenameColumn::new_name must be unqualified.
Ordering in commit():
1. deletes validated
2. renames validated — runs after deletes so a rename can re-use a
name being deleted, before additions so an addition can re-use a
name being renamed away
3. additions validated + ID-assigned
4. schema tree rebuilt with renames threaded through
rebuild_fields / rebuild_field
Validation rules (match pyiceberg's UpdateSchema.rename_column):
- missing source field → PreconditionFailed
- source field also staged for deletion → PreconditionFailed
- new_name contains SCHEMA_NAME_DELIMITER → PreconditionFailed
- new_name collides with a non-deleted, non-renamed sibling →
PreconditionFailed
- same field renamed twice → last rename wins
Identifier-field handling falls out for free — Rust keys identifier
fields by ID, not name, so with_identifier_field_ids(base_schema
.identifier_field_ids()) propagates the set unchanged across rename.
Tests: 10 new tests covering simple root rename, nested rename,
missing field, delete-conflict, sibling-collision, dotted-name
rejection, identifier-field preservation, rename-frees-old-name
(combined with add), repeated-rename-last-wins, no-op self-rename.
Full iceberg lib suite: 1306/1306 passing. Clippy + rustfmt clean.
|
Just a heads-up that #2563 adds column rename to the existing |
nazq
added a commit
to nazq/iceberg-rust
that referenced
this pull request
Jun 8, 2026
Extends UpdateSchemaAction with a rename(RenameColumn) builder method,
completing the add/delete/rename triad for non-incompatible schema
evolution.
API shape:
action.rename(
RenameColumn::builder().name("old").new_name("new").build(),
);
Mirrors the typed-builder shape used by apache#2451's broader schema-evolution
refactor, so callers adopting this PR today can migrate to apache#2451 with
no source change once it lands.
Renames preserve field IDs — only the leaf name changes. RenameColumn::name
uses SCHEMA_NAME_DELIMITER for nested fields (e.g. "person.name");
RenameColumn::new_name must be unqualified.
Ordering in commit():
1. deletes validated
2. renames validated — runs after deletes so a rename can re-use a
name being deleted, before additions so an addition can re-use a
name being renamed away
3. additions validated + ID-assigned
4. schema tree rebuilt with renames threaded through
rebuild_fields / rebuild_field
Validation rules (match pyiceberg's UpdateSchema.rename_column):
- missing source field → PreconditionFailed
- source field also staged for deletion → PreconditionFailed
- new_name contains SCHEMA_NAME_DELIMITER → PreconditionFailed
- new_name collides with a non-deleted, non-renamed sibling →
PreconditionFailed
- same field renamed twice → last rename wins
Identifier-field handling falls out for free — Rust keys identifier
fields by ID, not name, so with_identifier_field_ids(base_schema
.identifier_field_ids()) propagates the set unchanged across rename.
Tests: 10 new tests covering simple root rename, nested rename,
missing field, delete-conflict, sibling-collision, dotted-name
rejection, identifier-field preservation, rename-frees-old-name
(combined with add), repeated-rename-last-wins, no-op self-rename.
Full iceberg lib suite: 1306/1306 passing. Clippy + rustfmt clean.
Contributor
Author
TwinklerG
force-pushed
the
twinklerg/update_schema
branch
2 times, most recently
from
July 21, 2026 09:07
3efcc67 to
625cc46
Compare
TwinklerG
force-pushed
the
twinklerg/update_schema
branch
from
July 24, 2026 08:02
ced3b46 to
164cd16
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
SchemaUpdatelogic to Iceberg-Rust #697 Add update_column_doc to UpdateSchemaAction (changing a column doc shouldn't reassign its field ID) #2742What changes are included in this PR?
Refactor
UpdateSchemaActionto support all schema evolution operations on Iceberg tables following the implementation of iceberg-java.Similar to PR #2120(only add column and delete column), but I refactor it to add more features.
Are these changes tested?
Yes. Besides, I add test cases which are ported from iceberg-java, three of them to be implemented because of the lack of support for
PrimitiveType::Unknown.