feat: add snapshot, snapshot_info, and table core utilities with tests#118
Open
lucasfang wants to merge 1 commit into
Open
feat: add snapshot, snapshot_info, and table core utilities with tests#118lucasfang wants to merge 1 commit into
lucasfang wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces new “core utilities” around snapshots and table creation, including a new public SnapshotInfo API, along with unit tests for snapshot/table behaviors.
Changes:
- Adds
Snapshotmodel with JSON (de)serialization and commit-kind helpers, plus snapshot unit tests. - Adds public
SnapshotInfoAPI (include/paimon/snapshot/snapshot_info.h) and implementsCommitKindToString. - Adds initial table core implementation (
Table::Create,FullName) and a bucket mode enum, plus table unit tests.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| src/paimon/core/table/table.cpp | Implements Table::Create and FullName |
| src/paimon/core/table/table_test.cpp | Adds gtest coverage for table creation and schema loading failure |
| src/paimon/core/table/bucket_mode.h | Introduces BucketMode enum and related constants |
| src/paimon/core/snapshot.h | Defines Snapshot structure, commit kind, and JSON contract |
| src/paimon/core/snapshot.cpp | Implements commit-kind mapping and JSON (de)serialization |
| src/paimon/core/snapshot_test.cpp | Adds snapshot tests for JSON round-trips and commit kinds |
| src/paimon/core/snapshot_info.cpp | Implements SnapshotInfo::CommitKindToString |
| include/paimon/snapshot/snapshot_info.h | Adds exported public SnapshotInfo API |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| * limitations under the License. | ||
| */ | ||
|
|
||
| #include "paimon/catalog/table.h" |
| * limitations under the License. | ||
| */ | ||
|
|
||
| #include "paimon/catalog/table.h" |
Comment on lines
+277
to
+280
| void Snapshot::FromJson(const rapidjson::Value& obj) noexcept(false) { | ||
| version_ = RapidJsonUtil::DeserializeKeyValue<int32_t>(obj, FIELD_VERSION, -1); | ||
| id_ = RapidJsonUtil::DeserializeKeyValue<int64_t>(obj, FIELD_ID); | ||
| schema_id_ = RapidJsonUtil::DeserializeKeyValue<int64_t>(obj, FIELD_SCHEMA_ID); |
Comment on lines
+238
to
+240
| obj.AddMember(rapidjson::StringRef(FIELD_TOTAL_RECORD_COUNT), | ||
| RapidJsonUtil::SerializeValue(total_record_count_.value(), allocator).Move(), | ||
| *allocator); |
Comment on lines
+241
to
+243
| obj.AddMember(rapidjson::StringRef(FIELD_DELTA_RECORD_COUNT), | ||
| RapidJsonUtil::SerializeValue(delta_record_count_.value(), allocator).Move(), | ||
| *allocator); |
Comment on lines
+122
to
+124
| default: | ||
| assert(false); | ||
| return "UNKNOWN"; |
Comment on lines
+137
to
+138
| assert(false); | ||
| return Unknown(); |
Comment on lines
+81
to
+85
| return version_ == other.version_ && id_ == other.id_ && schema_id_ == other.schema_id_ && | ||
| index_manifest_ == other.index_manifest_ && commit_user_ == other.commit_user_ && | ||
| commit_identifier_ == other.commit_identifier_ && commit_kind_ == other.commit_kind_ && | ||
| log_offsets_ == other.log_offsets_ && total_record_count_ == other.total_record_count_ && | ||
| delta_record_count_ == other.delta_record_count_ && |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: No linked issue
This change adds snapshot management, snapshot info, and table core utilities with comprehensive test coverage.
Included changes:
Snapshot module (
src/paimon/core/snapshot.h,src/paimon/core/snapshot.cpp):Snapshot Info module (
include/paimon/snapshot/snapshot_info.h,src/paimon/core/snapshot_info.cpp):Table module (
src/paimon/core/table/bucket_mode.h,src/paimon/core/table/table.cpp):Test coverage:
snapshot_test.cppwith comprehensive test cases for snapshot functionalitytable_test.cppwith test cases for table operationsTests
Not run. Local compile, CMake, and gtest environment checks are not part of this PR description.
Test coverage included in this change:
snapshot_test.cpptable_test.cppAPI and Format
This change adds public API in
include/paimon/snapshot/snapshot_info.h.No storage format or protocol changes.
Documentation
No documentation changes required.
Generative AI tooling
Migrate-by: Aone Copilot (Qwen3.7-Max)