feat: data-links - recover from expired signing credentials during large data-link uploads - #652
Draft
georgi-seqera wants to merge 1 commit into
Draft
feat: data-links - recover from expired signing credentials during large data-link uploads#652georgi-seqera wants to merge 1 commit into
georgi-seqera wants to merge 1 commit into
Conversation
…rge data-link uploads
georgi-seqera
requested review from
sabulous,
t0randr and
weronikasosnowskaseqera
July 30, 2026 09:10
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Recover from expired signing credentials during large data-link uploads
Why
tw data-links uploadfails partway through large uploads withHTTP 400 ExpiredToken(e.g. a 108 GiB upload died at ~75% after ~60 minutes). The presigned URLs the CLI receives at the start of a multipart upload are signed with temporary credentials that expire after ~1 hour, and the CLI had no way to recover — it uploaded parts strictly sequentially, with no retry and no ability to obtain fresh URLs, so any upload outliving the credential lifetime failed outright.This is the client half of the fix; it pairs with the new Platform refresh endpoint introduced in
26.2.0(API version1.192.0).How
The CLI now detects an expired-credentials error mid-upload, asks Platform to re-sign the URLs for the parts it still needs, and retries them — so a long-running upload transparently continues instead of failing.
Alongside that, the upload path gains general resilience it was missing:
404), the CLI stops with a clear, actionable message rather than a confusing low-level error.Behavior differs per provider, matching what each backend can actually support:
Consuming the refresh endpoint requires a regenerated SDK, so this PR also bumps the Tower Java SDK version.
What it looks like
No new flags or usage changes — the same command now survives credential expiry:
Against a Platform too old to support refresh, it fails fast with a clear reason instead of a raw
ExpiredToken:Scope