fix(media): support IRSA/credential-chain S3 auth and configurable signing region#1406
Merged
Conversation
…gion The media S3 client always built static credentials from BUZZ_S3_ACCESS_KEY/BUZZ_S3_SECRET_KEY and hardcoded the SigV4 signing region to us-east-1. On EKS this blocks two things: the relay can't use its IRSA pod role (role/buzz) for media, forcing long-lived static IAM keys; and any non-us-east-1 deployment signs requests with the wrong credential scope, which AWS rejects. The underlying rust-s3 / aws-creds stack already supports the AWS default credential chain (env -> profile -> web-identity/IRSA -> container -> instance metadata) via Credentials::default(); the http-credentials feature is already enabled transitively through our tokio-rustls-tls feature. We just never called it. - MediaStorage::new: when both access/secret keys are non-empty, keep using them as static credentials (MinIO/local/dev unchanged); when both are empty, fall back to Credentials::default() so the pod's IAM role resolves via IRSA. Reject partial static credentials instead of silently switching auth modes. - Add MediaConfig.s3_region (env BUZZ_S3_REGION, falling back to AWS_REGION, default us-east-1) and use it for SigV4 signing instead of the hardcoded us-east-1. Local dev is unchanged: with the AWS key envs unset, the relay still defaults them to buzz_dev/buzz_dev_secret (static path). Production opts into IRSA by setting the key envs to empty strings (and dropping the s3-access-key/s3-secret-key ExternalSecret refs). Co-authored-by: Tyler Longwell <tlongwell@block.xyz> Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
tlongwell-block
force-pushed
the
fix/s3-irsa-auth
branch
from
797ffbb to
f050e0c
Compare
Ignored integration test proving the IRSA/credential-chain fallback did not regress hardcoded credentials: builds MediaStorage::new with static keys (buzz_dev/buzz_dev_secret) and round-trips put -> head -> get -> delete against the docker-compose MinIO. Opt-in via --ignored; reads config from BUZZ_S3_* env with MinIO defaults. Co-authored-by: Tyler Longwell <tlongwell@block.xyz> Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
…to git store GitStore::new had the same two bugs the media fix addressed: a hardcoded us-east-1 signing region and static-only credentials (Credentials::new with Some/Some), which short-circuits the AWS credential chain and never reaches web-identity/IRSA. The git store is wired from the same config.media.* fields, so under IRSA it would receive empty-string keys and fail. Mirror MediaStorage::new exactly: - take a region param, threaded from config.media.s3_region at the call site in state.rs (BUZZ_S3_REGION -> AWS_REGION -> us-east-1); - select credentials by (access_key.is_empty(), secret_key.is_empty()): both non-empty -> static; both empty -> Credentials::default() (chain); mixed -> StoreError::Config fail-fast, so a half-configured static deploy can't silently fall through to the chain. Add a StoreError::Config(String) variant for the fail-fast case. Update the two test call sites (store.rs probe, hydrate.rs live) to pass an explicit region. Add unit tests for the static-region and partial-key paths. Verified locally: the BUZZ_GIT_S3_PROBE live MinIO git pack round-trip (store -> hydrate -> clone) passes through the modified constructor, so the static path is unaffected. Co-authored-by: Tyler Longwell <tlongwell@block.xyz> Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Make the relay's S3 media client work with IRSA (EKS pod IAM role) and with non-
us-east-1regions, without long-lived static IAM keys.Two coupled bugs in
MediaStorage::new(crates/buzz-media/src/storage.rs) blocked the bb-block production deploy:Credentials::new(Some(access), Some(secret), …). That path short-circuits the AWS credential chain the moment a key is present (aws-credscredentials.rs:284), so the pod's IRSA role (role/buzz) — which the media bucket policy already trusts — was never used. The only way to run was to mint a long-lived IAM user + static keys.us-east-1.Region::Custom { region: "us-east-1", … }is the SigV4 credential scope (aws-regionregion.rs:204). Pointed athttps://s3.us-west-2.amazonaws.com, AWS rejects the mismatched scope.The key insight
The
rust-s3/aws-credsstack already supports the AWS default credential chain (env → profile → web-identity/IRSA → container → instance-metadata) viaCredentials::default(). Thehttp-credentialsfeature that gates the IRSAAssumeRoleWithWebIdentitypath is already enabled transitively through ourtokio-rustls-tlsfeature (aws-credsCargo.toml:47-49; confirmed viacargo tree -e featuresandattohttpcinCargo.lock). No new dependency, no feature flag. We just weren't calling it.Changes
MediaStorage::new: if boths3_access_keyands3_secret_keyare non-empty → use them as static credentials (MinIO/local/dev, any static-key deploy — unchanged). Otherwise →Credentials::default(), resolving IRSA/env/profile/metadata.MediaConfig.s3_region(new): envBUZZ_S3_REGION, falling back toAWS_REGION, defaultus-east-1. Used for SigV4 signing in place of the hardcode.Behavior
buzz_dev/buzz_dev_secret(static path), ands3_regiondefaults tous-east-1.BUZZ_S3_ACCESS_KEY=""/BUZZ_S3_SECRET_KEY=""(empty strings — an empty env var stays empty; only a missing var falls back to thebuzz_devdefault) and providingAWS_REGION/BUZZ_S3_REGION. The pod already injectsAWS_REGION,AWS_DEFAULT_REGION, and the IRSA web-identity env.Companion (bb-block, separate)
Once this is on the relay image, the bb-block manifest drops the
s3-access-key/s3-secret-keyExternalSecretdata refs and sets the twoBUZZ_S3_*key envs to empty. No IAM user, no static keys to rotate. (Context: bb-block #110 / Buzz deploy thread.)Testing
cargo test -p buzz-media— 41 passed (incl. newstatic_keys_build_client_with_configured_region).cargo test -p buzz-relay --lib config— 14 passed.cargo clippy -p buzz-media -p buzz-relay --all-targets— clean.