Skip to content

Lightsail as the default 1-click template #150

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lionello wants to merge 1 commit into from
Closed

Lightsail as the default 1-click template #150

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 57 additions & 40 deletions .github/workflows/publish-template.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ on:
branches: [ master ]
paths:
- 'aws/template.yaml'
- 'aws/lightsail-template.yaml'
- '.github/workflows/publish-template.yml'
workflow_dispatch:

Expand All @@ -27,7 +28,10 @@ jobs:
env:
AWS_REGION: ${{ vars.AWS_REGION || 'us-east-1' }}
BUCKET: ${{ vars.AGENT_BOX_BUCKET }}
OBJECT_KEY: template.yaml
# Both templates ship from the same bucket: lightsail-template.yaml is
# the default the README's Launch buttons point at, template.yaml is the
# EC2 alternative.
TEMPLATES: lightsail-template.yaml template.yaml
ROLE_ARN: ${{ vars.AWS_ROLE_ARN }}
steps:
- name: Sanity-check required variables
Expand All @@ -44,20 +48,21 @@ jobs:

- uses: cachix/install-nix-action@v31

- name: Validate template locally
- name: Validate templates locally
run: |
pip install --quiet cfn-lint
cfn-lint aws/template.yaml
cfn-lint aws/template.yaml aws/lightsail-template.yaml

- name: Inject pinned defaults for the 1-click copy
# The source template deliberately ships without Default: values for
- name: Inject pinned defaults for the 1-click copies
# The source templates deliberately ship without Default: values for
# AgentBoxRev / AgentBoxSha256 (see the params' Descriptions), and
# with empty Defaults for AgentNixpkgsUrl / AgentNixpkgsSha256 (so
# source-template launches like deploy-test fall back to the AMI
# source-template launches like deploy-test fall back to the base
# channel) — the 1-click Launch buttons need real values pre-filled,
# so we inject pinned values into only the S3 template copy: the
# so we inject pinned values into only the S3 template copies: the
# AgentBox pair gets a Default inserted, the AgentNixpkgs pair gets
# its empty Default replaced.
# its empty Default replaced. Both templates declare the four
# parameters with identical layout, so one awk serves both.
run: |
rev="${{ github.sha }}"
sha="sha256-$(curl -sSfL "https://raw.githubusercontent.com/${{ github.repository }}/${rev}/modules/agent-box.nix" \
Expand All @@ -69,37 +74,42 @@ jobs:
echo "Sha: $sha"
echo "AgentUrl: $agentUrl"
echo "AgentSha: $agentSha"
awk -v rev="$rev" -v sha="$sha" -v agentUrl="$agentUrl" -v agentSha="$agentSha" '
/^ AgentBoxRev:$/ { rf=1; print; next }
rf && /^ Type:/ { print; print " Default: " rev; rf=0; next }
/^ AgentBoxSha256:$/ { sf=1; print; next }
sf && /^ Type:/ { print; print " Default: " sha; sf=0; next }
/^ AgentNixpkgsUrl:$/ { sf2=1; print; next }
sf2 && /^ Default:/ { print " Default: " agentUrl; sf2=0; next }
/^ AgentNixpkgsSha256:$/ { sf3=1; print; next }
sf3 && /^ Default:/ { print " Default: " agentSha; sf3=0; next }
{ print }
' aws/template.yaml > aws/template.filled.yaml
if ! grep -qxF " Default: $rev" aws/template.filled.yaml \
|| ! grep -qxF " Default: $sha" aws/template.filled.yaml \
|| ! grep -qxF " Default: $agentUrl" aws/template.filled.yaml \
|| ! grep -qxF " Default: $agentSha" aws/template.filled.yaml; then
echo "::error::Default injection failed — expected lines not present in filled template."
diff aws/template.yaml aws/template.filled.yaml || true
exit 1
fi
cfn-lint aws/template.filled.yaml
for t in $TEMPLATES; do
awk -v rev="$rev" -v sha="$sha" -v agentUrl="$agentUrl" -v agentSha="$agentSha" '
/^ AgentBoxRev:$/ { rf=1; print; next }
rf && /^ Type:/ { print; print " Default: " rev; rf=0; next }
/^ AgentBoxSha256:$/ { sf=1; print; next }
sf && /^ Type:/ { print; print " Default: " sha; sf=0; next }
/^ AgentNixpkgsUrl:$/ { sf2=1; print; next }
sf2 && /^ Default:/ { print " Default: " agentUrl; sf2=0; next }
/^ AgentNixpkgsSha256:$/ { sf3=1; print; next }
sf3 && /^ Default:/ { print " Default: " agentSha; sf3=0; next }
{ print }
' "aws/$t" > "aws/filled-$t"
if ! grep -qxF " Default: $rev" "aws/filled-$t" \
|| ! grep -qxF " Default: $sha" "aws/filled-$t" \
|| ! grep -qxF " Default: $agentUrl" "aws/filled-$t" \
|| ! grep -qxF " Default: $agentSha" "aws/filled-$t"; then
echo "::error::Default injection failed for $t — expected lines not present in filled template."
diff "aws/$t" "aws/filled-$t" || true
exit 1
fi
cfn-lint "aws/filled-$t"
done

- name: Configure AWS credentials via OIDC
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ env.ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}

- name: Validate template with CloudFormation
run: aws cloudformation validate-template --template-body "file://aws/template.filled.yaml"
- name: Validate templates with CloudFormation
run: |
for t in $TEMPLATES; do
aws cloudformation validate-template --template-body "file://aws/filled-$t"
done

- name: Upsert bucket + public-read policy on template.yaml
- name: Upsert bucket + public-read policy on the templates
run: |
if aws s3api head-bucket --bucket "$BUCKET" 2>/dev/null; then
echo "Bucket $BUCKET already exists."
Expand All @@ -119,10 +129,12 @@ jobs:
--public-access-block-configuration \
'BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=false,RestrictPublicBuckets=false'

# Public-read on template.yaml only — CloudFormation's Launch Stack UI
# requires an S3 URL for templateURL. The module itself is fetched by
# the box direct from raw.githubusercontent.com (dual-stack), so this
# bucket carries no other public assets.
# Public-read on the two template objects only — CloudFormation's
# Launch Stack UI requires an S3 URL for templateURL. The module
# itself is fetched by the box direct from raw.githubusercontent.com
# (dual-stack), so this bucket carries no other public assets. One
# policy covers both objects (a single PutBucketPolicy replaces the
# whole document, so the two templates must not fight over it).
aws s3api put-bucket-policy --bucket "$BUCKET" --policy "$(cat <<POLICY
{
"Version": "2012-10-17",
Expand All @@ -131,15 +143,20 @@ jobs:
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::${BUCKET}/template.yaml"
"Resource": [
"arn:aws:s3:::${BUCKET}/lightsail-template.yaml",
"arn:aws:s3:::${BUCKET}/template.yaml"
]
}]
}
POLICY
)"

- name: Upload to S3
run: |
aws s3 cp aws/template.filled.yaml "s3://${BUCKET}/${OBJECT_KEY}" \
--content-type "application/x-yaml" \
--cache-control "no-cache, max-age=0"
echo "::notice::Published to https://${BUCKET}.s3.amazonaws.com/${OBJECT_KEY}"
for t in $TEMPLATES; do
aws s3 cp "aws/filled-$t" "s3://${BUCKET}/${t}" \
--content-type "application/x-yaml" \
--cache-control "no-cache, max-age=0"
echo "::notice::Published to https://${BUCKET}.s3.amazonaws.com/${t}"
done
2 changes: 1 addition & 1 deletion AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Keep the module self-contained: deployed boxes fetch `modules/agent-box.nix` as
- `nix build -L .#checks.x86_64-linux.multi-user` runs the quick module/configuration assertion.
- `nix build -L .#checks.x86_64-linux.module-single-file` verifies standalone module evaluation.
- `nix build -L .#checks.x86_64-linux.<name>` runs an individual VM test such as `sessions` or `settings-page`.
- `cfn-lint aws/template.yaml` validates the CloudFormation template.
- `cfn-lint aws/template.yaml aws/lightsail-template.yaml` validates the CloudFormation templates.

Prefer targeted checks over `nix flake check`; the intentionally filesystem-free VM configuration makes the latter unsuitable. Live browser tests require `E2E_BASE_URL` and `E2E_PASSWORD`; run `playwright test -c tests/e2e` after provisioning the nixpkgs Playwright browsers described in the config.

Expand Down
127 changes: 69 additions & 58 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,74 +18,58 @@ Supported agents:

## 1-click AWS launch

Provisions one EC2 instance (NixOS 25.11) with the module + a browser terminal
(Caddy -> ttyd) already wired up. First load takes ~2-3 minutes while the AMI
provisions, `nixos-rebuild switch` applies the config, and Caddy issues a
Let's Encrypt cert against `<eip>.sslip.io`.
Provisions one AWS Lightsail instance running NixOS with the module + a
browser terminal (Caddy -> ttyd) already wired up, priced as **one flat
monthly bundle** (compute + SSD + static IPv4 + a multi-TB transfer
allowance). Lightsail has no NixOS blueprint, so the box boots the Ubuntu
24.04 blueprint and converts itself to NixOS in-place on first boot with
[nixos-infect](https://github.com/elitak/nixos-infect); expect the first
launch to take ~10-20 minutes while the conversion and the first
`nixos-rebuild switch` run and Caddy issues a Let's Encrypt cert against
`<static-ip>.sslip.io`.

| Region | Launch |
| --- | --- |
| us-east-1 (N. Virginia) | [Launch stack →](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/quickcreate?stackName=agent-box&templateURL=https%3A%2F%2Fdefang-agent-box.s3.us-west-2.amazonaws.com%2Ftemplate.yaml) |
| us-west-2 (Oregon) | [Launch stack →](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/quickcreate?stackName=agent-box&templateURL=https%3A%2F%2Fdefang-agent-box.s3.us-west-2.amazonaws.com%2Ftemplate.yaml) |
| eu-central-1 (Frankfurt) | [Launch stack →](https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/quickcreate?stackName=agent-box&templateURL=https%3A%2F%2Fdefang-agent-box.s3.us-west-2.amazonaws.com%2Ftemplate.yaml) |
| eu-west-1 (Ireland) | [Launch stack →](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/quickcreate?stackName=agent-box&templateURL=https%3A%2F%2Fdefang-agent-box.s3.us-west-2.amazonaws.com%2Ftemplate.yaml) |
| us-east-1 (N. Virginia) | [Launch stack →](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/quickcreate?stackName=agent-box&templateURL=https%3A%2F%2Fdefang-agent-box.s3.us-west-2.amazonaws.com%2Flightsail-template.yaml) |
| us-west-2 (Oregon) | [Launch stack →](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/quickcreate?stackName=agent-box&templateURL=https%3A%2F%2Fdefang-agent-box.s3.us-west-2.amazonaws.com%2Flightsail-template.yaml) |
| eu-central-1 (Frankfurt) | [Launch stack →](https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/quickcreate?stackName=agent-box&templateURL=https%3A%2F%2Fdefang-agent-box.s3.us-west-2.amazonaws.com%2Flightsail-template.yaml) |
| eu-west-1 (Ireland) | [Launch stack →](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/quickcreate?stackName=agent-box&templateURL=https%3A%2F%2Fdefang-agent-box.s3.us-west-2.amazonaws.com%2Flightsail-template.yaml) |

Choose `Agent` (`claude` or `codex`), set a `WebPassword` (any 16&ndash;64
characters, including password-manager symbols), pick an instance size,
characters, including password-manager symbols), pick a bundle size,
launch. The stack reports
CREATE_COMPLETE only after the box phones home from its first successful
rebuild — a first boot that goes wrong (issue 106 has one way) rolls the
rebuild — a first boot that goes wrong rolls the
stack back visibly instead of leaving a green stack with a dead URL. The agent runs as the
`UserName` linux user (default `agent`). The template creates its own
IPv6-enabled VPC/subnet so nothing on the account has to be pre-configured. The
stack Outputs show `https://<v6-or-v4>.sslip.io/<UserName>/` - open it, sign in
`UserName` linux user (default `agent`). Lightsail manages the networking, so
nothing on the account has to be pre-configured. The
stack Outputs show `https://<ip>.sslip.io/<UserName>/` - open it, sign in
as the `UserName` with your `WebPassword`, complete the selected agent's
one-time sign-in, done. `<UserName>@<stack name>` is used as the Claude Remote
Control session name; rename the stack before launch if you want a friendlier
label in the Claude apps.

**Cost note (Feb-2024 AWS IPv4 pricing).** The default is **IPv6-only** to
avoid the ~$3.60/mo public-IPv4 charge that AWS bills for *every* public IPv4,
elastic or not. Works if your client has IPv6 connectivity (most consumer ISPs
in NA/EU do; corporate/coffee-shop nets often don't). If IPv6 isn't reachable
for you, set `PublicIpv4: true` at launch — allocates an EIP, adds $3.60/mo,
works everywhere. IPv6-only boxes still reach IPv4-only sites — notably
`github.com` — through a free public DNS64/NAT64 service
([nat64.net](https://nat64.net)), on by default. Traffic to IPv4-only hosts
transits the NAT64 operator's gateways (TLS and SSH stay end-to-end encrypted
and authenticated); set `Nat64: false` to opt out, at the price of the box
not reaching IPv4-only hosts.

Costs, all-in, running 24/7: the default is a **persistent Spot** instance
(`UseSpot`) — on a Spot interruption AWS stops and later restarts the *same*
instance, so the disk, IPv6 address, and TLS cert survive (a live tmux
session doesn't; RAM is lost on any stop). Spot for the default `t4g.medium`
(Graviton/aarch64, 2 vCPU / 4 GiB) is currently $0.018–0.024/hr across the
four launch regions — **~$16–20/mo** including the ~$2.40/mo default 30 GiB
gp3 root volume (`RootVolumeSize`). `t4g.small` on Spot lands around
**~$7–10/mo** all-in and works for a single light agent, though its 2 GiB is
tight during self-update rebuilds. Spot runs ~50–60% below on-demand for
these types (small Graviton instances don't see the deep 70–90% Spot
discounts); `UseSpot: false` gets you on-demand `t4g.medium` at ~$0.034/hr
(~$27/mo all-in). Networking is $0/mo in the default IPv6-only mode; with
`PublicIpv4: true` the Elastic IP adds ~$3.60/mo. Terminate the stack to
stop billing.

Out of disk anyway? Enlarge the volume from the EC2 console (Volumes ->
Modify) and reboot the instance — NixOS grows the partition and filesystem
on boot. The box also garbage-collects the nix store automatically.

**Root shell via SSM Session Manager.** The template ships no SSH key; the
browser terminal is an unprivileged `agent` user. For a root path onto the
box (e.g. to inspect `amazon-init` on a failed first boot, which is
journal-only and invisible to `get-console-output`), the default template
attaches an IAM instance profile with `AmazonSSMManagedInstanceCore`. Open
a shell via the AWS console (Systems Manager -> Session Manager) or
`aws ssm start-session --target <InstanceId>`, then `sudo -i`. This adds
one CAPABILITY_IAM checkbox to the Launch Stack form; opt out with
`EnableSsm=false` to skip it. See
[aws/README.md](./aws/README.md#root-access-via-ssm-session-manager) for
details.
**Cost.** The bundle price is the whole bill — no separate EBS, transfer, or
public-IPv4 line items. The default `small_3_0` (2 vCPU / 2 GiB / 60 GiB SSD)
is **$12/mo flat**; bundles range from `micro_3_0` (1 GiB, $7/mo) to
`xlarge_3_0` (16 GiB, $84/mo). 2 GiB is tight while `nixos-rebuild` evaluates
a self-update — the template adds a 3 GiB swap file to carry it, so updates
are slow rather than fatal; pick `medium_3_0` (4 GiB, $24/mo) for comfortable
rebuild headroom. The attached static IPv4 is included, and the URL survives
a stop/start (a live tmux session doesn't; RAM is lost on any stop). Delete
the stack to stop billing.

Out of disk? Lightsail bundles have a fixed SSD — snapshot the instance and
restore onto a larger bundle to grow. The box also garbage-collects the nix
store automatically.

**Root shell for debugging.** The browser terminal is an unprivileged `agent`
user. For a root path onto the box (e.g. to inspect a failed first-boot
conversion — logs land in `/var/log/agent-box-infect.log`), the template
leaves port 22 open by default (`DebugSsh`) for key-only root SSH with the
Lightsail default key: download it from the Lightsail console (Account ->
SSH keys), then `ssh -i <key> root@<static-ip>`. Password auth stays off;
set `DebugSsh=false` at launch to keep 22 closed.

**Changing the web password.** Open the settings page (the gear icon next to
the terminal), choose **Change password**, and enter the current password plus
Expand All @@ -108,9 +92,36 @@ Anything that is not a fast-forward of the running revision is refused.
Verifying releases against an offline signing key is tracked in
[issue 46](https://github.com/defangdevs/agent-box/issues/46).

Template source: [`aws/template.yaml`](./aws/template.yaml).
See [`aws/README.md`](./aws/README.md) for the region -> AMI refresh workflow
and the S3-hosting setup.
Template source: [`aws/lightsail-template.yaml`](./aws/lightsail-template.yaml).
See [`aws/README.md`](./aws/README.md) for design notes and the S3-hosting
setup.

### Alternative: EC2 template (Spot, IPv6-only)

The original EC2 template is still published, for accounts that want EC2's
flexibility: arbitrary instance types, Spot pricing, IaC-native VPC
networking, root access via SSM Session Manager, and an EBS root volume that
can grow in place.

| Region | Launch |
| --- | --- |
| us-east-1 (N. Virginia) | [Launch stack →](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/quickcreate?stackName=agent-box&templateURL=https%3A%2F%2Fdefang-agent-box.s3.us-west-2.amazonaws.com%2Ftemplate.yaml) |
| us-west-2 (Oregon) | [Launch stack →](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/quickcreate?stackName=agent-box&templateURL=https%3A%2F%2Fdefang-agent-box.s3.us-west-2.amazonaws.com%2Ftemplate.yaml) |
| eu-central-1 (Frankfurt) | [Launch stack →](https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/quickcreate?stackName=agent-box&templateURL=https%3A%2F%2Fdefang-agent-box.s3.us-west-2.amazonaws.com%2Ftemplate.yaml) |
| eu-west-1 (Ireland) | [Launch stack →](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/quickcreate?stackName=agent-box&templateURL=https%3A%2F%2Fdefang-agent-box.s3.us-west-2.amazonaws.com%2Ftemplate.yaml) |

It boots a NixOS 25.11 AMI directly (no conversion step; first load ~2-3
minutes) and defaults to a **persistent Spot** `t4g.medium` on an
**IPv6-only** network — ~$16-20/mo all-in, dodging AWS's ~$3.60/mo
public-IPv4 charge. Set `PublicIpv4: true` at launch if your client has no
IPv6 connectivity (corporate/coffee-shop networks often don't; adds the
$3.60/mo EIP), and `UseSpot: false` for on-demand (~$27/mo all-in, no
interruption risk). IPv6-only boxes reach IPv4-only hosts through a free
public DNS64/NAT64 service ([nat64.net](https://nat64.net)); set
`Nat64: false` to opt out. The full cost breakdown, the Spot
stop-not-terminate behavior, SSM root access, and the other design notes
live in [aws/README.md](./aws/README.md); template source:
[`aws/template.yaml`](./aws/template.yaml).

## Why

Expand Down
Loading