Skip to content

Secure log shipper docs: no public services or IPs#2420

Open
kcmartin wants to merge 4 commits into
mainfrom
fix-log-shipper-no-public-access
Open

Secure log shipper docs: no public services or IPs#2420
kcmartin wants to merge 4 commits into
mainfrom
fix-log-shipper-no-public-access

Conversation

@kcmartin

@kcmartin kcmartin commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

The Log Shipper guide told users to edit fly.toml and keep a [[services]] section on internal_port = 8686. That port serves Vector's GraphQL API (including an unauthenticated /playground), so following the docs exposed it publicly. This was reported on the community forum, where Fly staff recommended deploying with --no-public-ips.

Vector pulls logs from Fly.io's internal log stream and doesn't need to accept inbound connections, so this:

  • Removes the [[services]] / port 8686 block from the setup steps
  • Deploys with fly deploy --no-public-ips
  • Notes how to release IPs on already-exposed deployments (fly ips list / fly ips release)

Supersedes #930 (same fix, but that PR targeted a since-moved file and had gone stale).

The log shipper guide told users to keep a [[services]] section on
port 8686, which exposed Vector's unauthenticated GraphQL API publicly
(reported at community.fly.io/t/14836). Vector pulls logs from the
internal stream and needs no inbound access, so remove the service
section and deploy with --no-public-ips. Also note how to release IPs
on already-exposed deployments.

Supersedes #930.
@kcmartin

kcmartin commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

Tested this end to end against a real deployment (throwaway app in my personal org, destroyed after):

  • Followed the updated instructions verbatim: launched the shipper image, deleted the generated [http_service] section, deployed with fly deploy --no-public-ips. Result: no public IPs allocated, no DNS record created, port 8686 unreachable. flyctl confirms: "Your app is deployed but does not have a public or private IP address."
  • Logs confirmed Vector connects outbound to the internal NATS stream, so the shipper genuinely needs no inbound services.
  • Negative control: redeployed with the old-style public service config and allocated IPs. Vector's GraphQL playground was served publicly at https://<app>.fly.dev/playground, reproducing the exposure from the original community report.
  • Remediation path: fly ips release on both addresses cleared the DNS records immediately; the hostname stopped resolving entirely within a few minutes.

Two things surfaced during testing and were folded into the PR: fly launch generates [http_service] (single brackets), so the text now matches, and Datadog was added to the Vale vocabulary to clear a pre-existing lint failure on this file.

@kcmartin

kcmartin commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

Hold on merging for now: #1828 raises a related need (the shipper can wedge silently, and a health check was recommended by support). Since this PR removes the public service, the right approach is a top-level [checks] machine check against Vector's /health on 8686, which needs no service or public IPs. Plan is to add a short health check section here before this merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant