[Subscription] Fix WAL replay after leader failover - #18474
Open
Caideyipi wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Problem
After an IoTConsensus leader failover, the new leader can replay WAL entries replicated from the former leader. These entries carry valid writer progress but keep
searchIndex = -1because they do not have a local search index.Replay-start lookup treated an uncovered follower entry before the first locally indexed entry as non-replayable and returned
LOCATE_MISS. This surfaced as subscription error 1904 when one DataNode went down in a replicated cluster.Fix
ProgressWALIterator; writer progress continues to deduplicate already committed entries.No client change is required for this case. With another provider available, the existing consumer poll path marks the failed provider unavailable and continues failover; an application may temporarily receive an empty poll result, but it does not need a special retry for error 1904.
Validation
mvn spotless:apply -pl iotdb-core/datanodemvn test -pl iotdb-core/datanode "-Dtest=ConsensusPrefetchingQueueTest,ProgressWALIteratorTest,ConsensusPrefetchingQueueWalBackpressureTest"mvn checkstyle:check -pl iotdb-core/datanodegit diff --checkThis PR has:
Key changed/added classes
ConsensusPrefetchingQueueConsensusPrefetchingQueueTestProgressWALIteratorTest