HDFS-17850. Reject append when new generation stamp equals or is less… - #8334
HDFS-17850. Reject append when new generation stamp equals or is less…#8334deepujain wants to merge 2 commits into
Conversation
|
🎊 +1 overall
This message was automatically generated. |
|
CI is green on the current head, and this one still has not had a human review. @Hexiaoqiao @ZanderXu, when you have a moment, could you please take a look? Happy to make any follow-up changes. |
|
CI is still green on the current head, and this one is still waiting on human review. @hfutatzhanghb @Hexiaoqiao, when you have a moment, could you please take a look? Happy to make any follow-up changes. |
|
@deepujain Thanks for your work! Could you rebase this PR onto the latest trunk branch ? Will review later. Thanks. |
b40483d to
261c748
Compare
|
Rebased this onto the latest apache/trunk and force-pushed the refreshed single commit. Local validation passed with |
|
🎊 +1 overall
This message was automatically generated. |
|
The rebase request is addressed and CI is green on the current head. @hfutatzhanghb, when you have a moment, could you please take another look? Happy to make any follow-up changes. |
|
The requested rebase is already addressed and CI is still green on the current head. @hfutatzhanghb, when you have a moment, could you please take another look? Happy to make any follow-up changes. |
261c748 to
2f536d1
Compare
|
Rebased this on current trunk and force-pushed the refreshed head. Ran |
|
🎊 +1 overall
This message was automatically generated. |
|
Reran the existing |
|
Reran the existing workflow on the current head again. The latest fork-side rerun hit the shared lane, while and on this head are green. No code changes in this update. |
|
Reran the existing |
|
No code changes here. I reran the existing fork-side |
2f536d1 to
39d8560
Compare
|
🎊 +1 overall
This message was automatically generated. |
|
No code changes here. I reran the current head again on July 12, 2026 to refresh the fork-side red GitHub Actions lane. On that rerun, the only failing shard was If the patch looks reasonable, @hfutatzhanghb @Hexiaoqiao, could you please take a look when you have a moment? Happy to make any follow-up changes. |
|
No code changes here. |
|
No code changes here. I reran the existing fork-side |
|
No code changes here. I reran the fork-side |
|
No code changes here. |
39d8560 to
8886e2d
Compare
|
Rebased this branch onto current apache/trunk on August 4, 2026 and force-pushed the refreshed single commit. I also retried |
|
Pushed a no-code |
|
💔 -1 overall
This message was automatically generated. |
Summary
When using a third-party HDFS client, if the client does not update the generation stamp (GS) correctly on append, the DataNode previously accepted
newGS == currentGS, which can lead to corrupted replica state. This change requires the new generation stamp to be strictly greater than the replica's current generation stamp: append now rejects bothnewGS < currentGS(existing) andnewGS == currentGS(new), so misbehaving clients cannot silently corrupt replica state.Change
append(ExtendedBlock b, long newGS, long expectedBlockLen), change the validity check fromnewGS < b.getGenerationStamp()tonewGS <= b.getGenerationStamp(), so that equal generation stamps are also rejected with the same IOException message.testAppendRejectsSameOrLowerGenerationStamp(): create a file, get the finalized block, then assert thatappend(block, block.getGenerationStamp(), blockLen)andappend(block, block.getGenerationStamp() - 1, blockLen)throw IOException with "should be greater than the replica", and thatappend(block, block.getGenerationStamp() + 1, blockLen)succeeds (HDFS-17850).JIRA
Fixes HDFS-17850