Skip to content

Add rewind support#121

Merged
berndverst merged 19 commits into
microsoft:mainfrom
andystaples:andystaples/add-orchestration-rewind
Jul 10, 2026
Merged

Add rewind support#121
berndverst merged 19 commits into
microsoft:mainfrom
andystaples:andystaples/add-orchestration-rewind

Conversation

@andystaples

@andystaples andystaples commented Mar 6, 2026

Copy link
Copy Markdown
Contributor

Adds rewind support to durabletask.
Requires the following changes:

  1. Add rewind action to protobuf: Adding a Rewind Action durabletask-protobuf#65
  2. Add rewind support to the DTS sidecar: https://msazure.visualstudio.com/One/_git/AAPT-DTMB/pullrequest/14935863
  3. Add rewind action support to DTS

Resolves #68

Comment thread durabletask/worker.py
@andystaples andystaples marked this pull request as ready for review July 9, 2026 21:18
Copilot AI review requested due to automatic review settings July 9, 2026 21:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds orchestration rewind support to the Durable Task Python SDK, including a new client API, worker-side history rewriting logic, and in-memory backend support, along with unit + E2E tests and a changelog entry.

Changes:

  • Added TaskHubGrpcClient.rewind_orchestration() and underlying gRPC plumbing to request rewinds.
  • Implemented worker-side rewind detection and clean-history generation via RewindOrchestrationAction.
  • Implemented rewind handling in the in-memory test backend and added comprehensive rewind test coverage (core + Azure Managed provider).

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tests/durabletask/test_rewind_e2e.py Adds core SDK end-to-end tests for rewind behavior against the in-memory backend.
tests/durabletask/test_build_rewind_result.py Adds unit tests for worker history rewriting semantics during rewind.
tests/durabletask-azuremanaged/test_dts_rewind_e2e.py Adds (currently skipped) DTS end-to-end rewind tests for azuremanaged.
durabletask/worker.py Adds rewind detection and history rewrite action creation in the worker.
durabletask/testing/README.md Updates in-memory backend limitations documentation to reflect rewind support.
durabletask/testing/in_memory_backend.py Implements rewind RPC and backend-side processing of RewindOrchestrationAction.
durabletask/internal/helpers.py Adds helper for executionCompleted history events.
durabletask/client.py Adds rewind_orchestration API to the client.
CHANGELOG.md Documents the new rewind client API and in-memory backend support under Unreleased.

Comment thread tests/durabletask/test_rewind_e2e.py Outdated
Comment thread durabletask/client.py Outdated
Comment thread durabletask/testing/in_memory_backend.py
Comment thread durabletask/testing/in_memory_backend.py Outdated
Comment thread durabletask/worker.py Outdated
Comment thread durabletask/worker.py Outdated

@berndverst berndverst left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewind support review (compared against durabletask-dotnet)

Thanks for this — overall it looks solid, and I appreciate that the SDK-side history rewriting in _build_rewind_result is clearly documented with the backend-contract warning. I compared it against dotnet's rewind support and have a few gaps/notes (most are inline).

🔴 Async client parity is missing (main gap). rewind_orchestration and the stub RewindInstance are added only to the sync TaskHubGrpcClient / _SyncTaskHubSidecarServiceStub. AsyncTaskHubGrpcClient and _AsyncTaskHubSidecarServiceStub don't get equivalents, so AsyncDurableTaskSchedulerClient in durabletask-azuremanaged can't rewind. Every other lifecycle op (terminate/suspend/resume/restart/purge) exists on both clients, and the generated orchestrator_service_pb2_grpc.py already exposes the RPC. See inline.

🟡 azuremanaged CHANGELOG. Only the core CHANGELOG.md was updated, but rewind is surfaced through the DTS clients and has a dedicated tests/durabletask-azuremanaged/test_dts_rewind_e2e.py. Per the repo's dual-changelog guideline, durabletask-azuremanaged/CHANGELOG.md should get an ## Unreleased entry. See inline.

🟡 Retry-policy / timer edge case in _build_rewind_result — see inline.

🟢 Minor: likely-dead branch around subOrchestrationInstanceFailed.taskScheduledId — see inline.

🟢 Error mapping (not blocking). dotnet maps gRPC status codes to typed exceptions (NotFound→ArgumentException, FailedPrecondition→InvalidOperationException, Unimplemented→NotImplementedException, Cancelled→OperationCanceledException). This PR lets grpc.RpcError propagate, which is consistent with every other Python client method here, so it's fine to leave — just flagging the UX difference from dotnet.

Architecture note (no action needed). dotnet keeps rewind a thin client→sidecar call and lets DurableTask.Core rewrite history (its worker only deserializes ExecutionRewound). This PR instead computes the clean history in the SDK worker. That's a reasonable divergence given the DTS backend design, but it creates a tight SDK↔backend contract: the worker detects a rewind via an executionCompleted event in committed history plus an executionRewound event in the new events. Since the DTS E2E tests are @skip until the emulator ships rewind, that contract is currently only exercised against the in-memory backend — worth a follow-up to un-skip once the emulator supports it.

Comment thread durabletask/client.py
Comment thread durabletask/client.py
Comment thread CHANGELOG.md
Comment thread durabletask/worker.py
Comment thread durabletask/worker.py Outdated
- Add async rewind_orchestration parity on AsyncTaskHubGrpcClient

- in-memory backend: stamp ExecutionRewound with current time; avoid duplicate executionRewound on post-rewind replay

- worker: fix rewind comments, guard parentInstance, drop dead sub-orchestration branch

- tests: free-port fixture, clarify docstrings, un-skip DTS rewind e2e (emulator now supports RewindInstance)

- azuremanaged CHANGELOG entry
Comment thread durabletask/worker.py Outdated
@berndverst berndverst merged commit e7b5f15 into microsoft:main Jul 10, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for orchestration rewind

4 participants