diff --git a/.agents/references/shared/terraform-module-standards.md b/.agents/references/shared/terraform-module-standards.md index 41d8575..1a60b5f 100644 --- a/.agents/references/shared/terraform-module-standards.md +++ b/.agents/references/shared/terraform-module-standards.md @@ -33,10 +33,23 @@ These standards apply to both primitive and reference architecture modules. - `examples/complete/README.md` must match `examples/complete/main.tf`, variables, and outputs. - Do not leave empty terraform-docs blocks. +## Toolchain managers + +Install pinned tools from `.tool-versions` with `mise install` (preferred) or `make configure` (mise or asdf). Keep the manager environment active so shims are on `PATH`, then invoke managed tools **by name** — the same way you would if they were natively installed. Do not prefix every command with `mise exec --` or `asdf exec`. + +Use extended manager features when they earn their keep: + +- `mise run ` — repo-defined tasks and orchestration +- `mise install` / `asdf install` — installing a pinned or alternate tool version +- version selection APIs — when the work needs a Terraform or other tool version that is **not** the repo default (see `.github/scripts/check-terraform-version-floor.sh`) + +If a tool is missing, install and activate the environment first; do not treat `mise exec --` as the default invocation style. + ## Validation -- Prefer `mise run ` when a suitable task exists. -- Otherwise use `mise exec -- ` for Terraform, Go, pre-commit, and related tools. +- Prefer Makefile targets (`make lint`, `make check`, `make test`) and `pre-commit run` when they cover the work. +- When creating a module with Terratest, set the Go version in `.tool-versions` and the `go` directive in `go.mod` to the latest stable version supported by Terratest and the repository tooling. Confirm the installed version with `go version`. +- From the module root, refresh the complete Go dependency graph with `go get -u ./...`, then run `go mod tidy`. Verify `go list -m -u all` reports no remaining available module upgrades; resolve incompatibilities in the Terratest code rather than retaining stale dependencies. - Run formatting and linting before broader test flows. - For examples, validate initialization, Terraform validation, and plan where credentials and backend constraints allow. - For Go tests, run `go mod tidy`, build or targeted tests, and broader Terratest only when the required cloud access is available. diff --git a/.agents/skills/primitive-module/SKILL.md b/.agents/skills/primitive-module/SKILL.md index 03a21e0..2c04a97 100644 --- a/.agents/skills/primitive-module/SKILL.md +++ b/.agents/skills/primitive-module/SKILL.md @@ -24,8 +24,9 @@ Use this skill for repositories named `tf--module_primitive- 4. Implement one primitive resource interface with explicit variable types, descriptions, validations, resource outputs, and provider constraints. 5. Build `examples/complete/` as the canonical secure usage example. 6. Keep `README.md` derived from `TEMPLATED_README.md`; replace the module-specific sections and preserve skeleton development boilerplate. -7. Add or update Terratest code so assertions verify specific expected values and security settings through provider APIs when applicable. -8. Run focused validation, then broader checks such as formatting, linting, Terraform init/validate/plan for the example, README generation, and Go test build or Terratest where practical. +7. Set the latest supported Go baseline and refresh the complete Go dependency graph as required by the shared Terraform module standards. +8. Add or update Terratest code so assertions verify specific expected values and security settings through provider APIs when applicable. +9. Run focused validation, then broader checks such as formatting, linting, Terraform init/validate/plan for the example, README generation, and Go test build or Terratest where practical. ## Completion Gate diff --git a/.agents/skills/reference-architecture/SKILL.md b/.agents/skills/reference-architecture/SKILL.md index 1875114..36a10d5 100644 --- a/.agents/skills/reference-architecture/SKILL.md +++ b/.agents/skills/reference-architecture/SKILL.md @@ -24,8 +24,9 @@ Use this skill for repositories named `tf--module_reference-= the root's. Terraform applies every +# required_version in the module tree and takes the maximum, so an example +# that declares less than the root is stating something untrue about itself. +# Declaring more is legitimate: an example may consume a module, or use a +# language feature, that genuinely needs a newer Terraform. +# +# * Each directory is then loaded with its own declared floor. +# +# In practice root and examples usually match; the >= relation exists so that a +# genuinely newer example does not force the root floor up with it. +# +# This target is CI-oriented. CI runs it after `make lint`, so generated example +# provider files and a lock file already exist. Running it locally is supported +# but may install a Terraform version you would not otherwise have. +# +# Usage: +# check-terraform-version-floor.sh # run the check +# check-terraform-version-floor.sh --print-floor # print the ROOT floor only +# +# --print-floor exists so CI can compute a cache key before installing the +# toolchain. It reports the root floor, which is always needed; an example +# declaring something higher is rare and installs uncached. + +set -euo pipefail + +MODE="${1:-check}" +VERSIONS_FILE="${VERSIONS_FILE:-versions.tf}" +# Keep our .terraform out of the way of the main lint pass, which inits the same +# directories with a different Terraform version. +FLOOR_TF_DATA_DIR="${FLOOR_TF_DATA_DIR:-.terraform-version-floor}" + +die() { echo "ERROR: $*" >&2; exit 1; } + +[[ -f "${VERSIONS_FILE}" ]] || die "no ${VERSIONS_FILE} found in $(pwd)" + +# ------------------------------------------------------------------------------ +# Resolve the lowest Terraform version a constraint admits. +# +# Only lower-bound operators contribute a floor: `~>`, `>=`, `=`, and a bare +# version. Upper bounds (`<`, `<=`) are ignored, and strict `>` / exclusions +# (`!=`) contribute nothing -- neither appears in fleet use today. Where several +# terms are comma-separated we take the highest of the minimums. +# ------------------------------------------------------------------------------ + +# `|| true` matters: under `set -e` with pipefail, a non-matching grep would abort +# the script here, making the "no required_version" diagnostics below unreachable. +read_constraint() { + { grep -oE 'required_version[[:space:]]*=[[:space:]]*"[^"]*"' "$1" 2>/dev/null || true; } \ + | head -1 | sed -E 's/.*"(.*)"$/\1/' +} + +resolve_floor() { + local constraint="$1" floor="" term raw maj min pat candidate + IFS=',' read -ra terms <<< "${constraint}" + for term in "${terms[@]}"; do + term="$(printf '%s' "${term}" | tr -d '[:space:]')" + case "${term}" in + '~>'*|'>='*|'='[0-9]*|[0-9]*) + raw="$(printf '%s' "${term}" | grep -oE '[0-9]+(\.[0-9]+)*' || true)" + [[ -n "${raw}" ]] || continue + IFS='.' read -r maj min pat <<< "${raw}" + candidate="${maj:-0}.${min:-0}.${pat:-0}" + if [[ -z "${floor}" ]] || + [[ "$(printf '%s\n%s\n' "${floor}" "${candidate}" | sort -V | tail -1)" == "${candidate}" ]]; then + floor="${candidate}" + fi + ;; + *) ;; + esac + done + printf '%s' "${floor}" +} + +# version_ge A B -> true when A >= B +version_ge() { + [[ "$(printf '%s\n%s\n' "$1" "$2" | sort -V | head -1)" == "$2" ]] +} + +constraint="$(read_constraint "${VERSIONS_FILE}")" +[[ -n "${constraint}" ]] || die "no required_version found in ${VERSIONS_FILE}" + +floor="$(resolve_floor "${constraint}")" +[[ -n "${floor}" ]] || die "could not resolve a lower bound from required_version = \"${constraint}\"" + +if [[ "${MODE}" == "--print-floor" ]]; then + printf '%s\n' "${floor}" + exit 0 +fi + +echo "==> Root required_version: ${constraint} (floor ${floor})" + +# ------------------------------------------------------------------------------ +# Each example must declare a floor at or above the root's. +# +# Checked before any binary is acquired, so this needs no Terraform. +# ------------------------------------------------------------------------------ + +example_dirs=() +while IFS= read -r d; do + [[ -n "$d" ]] && example_dirs+=("$d") +done < <( + find ./examples -path "*/.terraform" -prune -o -name main.tf -print 2>/dev/null \ + | xargs -n1 dirname 2>/dev/null | sort -u +) + +# Parallel arrays: bash 3.2 (macOS) has no associative arrays. +check_dirs=(".") +check_floors=("${floor}") +bad=0 + +for d in ${example_dirs[@]+"${example_dirs[@]}"}; do + if [[ ! -f "${d}/versions.tf" ]]; then + echo " ${d}: no versions.tf -- it must declare a required_version" >&2 + bad=1 + continue + fi + ec="$(read_constraint "${d}/versions.tf")" + if [[ -z "${ec}" ]]; then + echo " ${d}/versions.tf declares no required_version" >&2 + bad=1 + continue + fi + ef="$(resolve_floor "${ec}")" + if [[ -z "${ef}" ]]; then + echo " ${d}/versions.tf has no lower bound in \"${ec}\"" >&2 + bad=1 + continue + fi + if ! version_ge "${ef}" "${floor}"; then + echo " ${d}/versions.tf declares \"${ec}\" (floor ${ef}), below the root's ${floor}" >&2 + bad=1 + continue + fi + check_dirs+=("${d}") + check_floors+=("${ef}") +done + +if [[ "${bad}" -ne 0 ]]; then + cat >&2 </dev/null 2>&1; then + candidate="$(mise which terraform --version "${version}" 2>/dev/null || true)" + if [[ -n "${candidate}" && -x "${candidate}" ]]; then printf '%s' "${candidate}"; return 0; fi + fi + return 1 +} + +ensure_binary() { + local version="$1" bin + if bin="$(resolve_binary "${version}")"; then printf '%s' "${bin}"; return 0; fi + if command -v asdf >/dev/null 2>&1; then + echo "==> Installing Terraform ${version} via asdf" >&2 + asdf install terraform "${version}" >&2 \ + || die "asdf could not install Terraform ${version} (does a build exist for this platform?)" + elif command -v mise >/dev/null 2>&1; then + echo "==> Installing Terraform ${version} via mise" >&2 + mise install "terraform@${version}" >&2 \ + || die "mise could not install Terraform ${version} (does a build exist for this platform?)" + else + die "Terraform ${version} is not installed and neither asdf nor mise is available to install it" + fi + bin="$(resolve_binary "${version}")" || die "install reported success but Terraform ${version} was not found" + printf '%s' "${bin}" +} + +# ------------------------------------------------------------------------------ +# Load each directory with its own declared floor. +# ------------------------------------------------------------------------------ + +created_locks=() +cleanup() { + local d + for d in ${check_dirs[@]+"${check_dirs[@]}"}; do + [[ -n "${d}" ]] && rm -rf -- "${d:?}/${FLOOR_TF_DATA_DIR}" + done + # Only remove lock files this check created; never touch pre-existing ones. + for d in ${created_locks[@]+"${created_locks[@]}"}; do + [[ -n "${d}" ]] && rm -f -- "${d}/.terraform.lock.hcl" + done +} +trap cleanup EXIT + +check_dir() { + local dir="$1" version="$2" label="$3" bin out rc lock_args=() + bin="$(ensure_binary "${version}")" + + # Never rewrite a lock file produced by the earlier lint init at a different + # Terraform version; if none exists yet, note it so cleanup can remove ours. + if [[ -f "${dir}/.terraform.lock.hcl" ]]; then + lock_args=(-lockfile=readonly) + else + created_locks+=("${dir}") + fi + + set +e + # Guarded expansion: bash 3.2 (macOS) errors on an empty array under `set -u`. + out="$( cd "${dir}" && TF_DATA_DIR="${FLOOR_TF_DATA_DIR}" "${bin}" \ + init -backend=false -input=false ${lock_args[@]+"${lock_args[@]}"} 2>&1 )" + rc=$? + if [[ ${rc} -eq 0 ]]; then + out="$( cd "${dir}" && TF_DATA_DIR="${FLOOR_TF_DATA_DIR}" "${bin}" validate 2>&1 )" + rc=$? + fi + set -e + + if [[ ${rc} -ne 0 ]]; then + cat >&2 <= 1.3; nullable and moved need >= 1.1; a validation block's +error_message form and precondition/postcondition need >= 1.2; terraform_data +needs >= 1.4; check and import blocks need >= 1.5; removed needs >= 1.7; +strcontains/startswith/endswith and provider:: functions need >= 1.8; +templatestring needs >= 1.9. +EOF + exit 1 + fi + echo " ok: ${label} on Terraform ${version}" +} + +echo "==> Loading each directory at its declared floor" +i=0 +while [[ ${i} -lt ${#check_dirs[@]} ]]; do + d="${check_dirs[$i]}" + v="${check_floors[$i]}" + if [[ "${d}" == "." ]]; then + check_dir "${d}" "${v}" "the root module" + else + check_dir "${d}" "${v}" "${d}" + fi + i=$(( i + 1 )) +done + +echo "==> OK: root module and $(( ${#check_dirs[@]} - 1 )) example(s) load at their declared floors" diff --git a/AGENTS.md b/AGENTS.md index d79b9d8..fc92991 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,7 +11,7 @@ This repository follows the Launch Terraform module standards. Keep this file sm - Treat examples and tests as part of the public contract. Update them with implementation changes. - Do not preserve skeleton placeholders, TODOs, or copied template names in completed modules. - Do not introduce provider-specific guidance into shared rules unless it is clearly labeled by provider. -- Use `mise` for local tool execution when available. Prefer `mise run ` for configured tasks and `mise exec -- ` otherwise. +- **Toolchain managers (mise / asdf):** install pinned tools from `.tool-versions` (`mise install` preferred, or `make configure` / asdf when mise is unavailable), activate the environment so shims are on `PATH`, then run managed tools **by name** (`terraform`, `go`, `pre-commit`, `make lint`) as if natively installed. Do not wrap every invocation in `mise exec --` or `asdf exec`. Use extended manager features only when they add value: `mise run ` for repo-defined tasks, installing tools, or selecting an **alternate version** (not the repo default). - Use SSH-based Git remotes or `gh` for GitHub repository operations. If SSH or `gh` is not working, stop and resolve that rather than silently switching to HTTPS Git remotes. - GitHub API access through `gh api` or `gh api graphql` is acceptable when repository metadata is needed. @@ -32,5 +32,6 @@ This repository follows the Launch Terraform module standards. Keep this file sm ## Validation Expectations - Run the narrowest useful validation first, then broaden when the change affects shared behavior. +- When creating a module with Terratest, update the Go baseline and dependency graph before considering the implementation complete; follow the shared Terraform module standards for the required commands and verification. - Before considering module creation complete, validate formatting, linting, Terraform initialization/validation for examples, README generation, and Terratest readiness where practical. - If full cloud-backed tests cannot be run, state what was validated and what remains unproven. diff --git a/Makefile b/Makefile index 8ab409c..e2d1cec 100644 --- a/Makefile +++ b/Makefile @@ -34,6 +34,12 @@ TERRAFORM ?= terraform RM ?= rm -rf TFLINT ?= tflint +# Every module and example directory inits separately, so without a shared cache +# each one downloads the same providers again. Terraform ignores this setting if +# the directory does not exist, so tfmodule/init creates it. +TF_PLUGIN_CACHE_DIR ?= $(HOME)/.terraform.d/plugin-cache +export TF_PLUGIN_CACHE_DIR + # ------------------------------------------------------------------------------ # Variables — Golang # ------------------------------------------------------------------------------ @@ -347,6 +353,7 @@ tfmodule/fmt: .PHONY: tfmodule/init tfmodule/init: + @mkdir -p "$(TF_PLUGIN_CACHE_DIR)" @$(foreach module,$(ALL_TF_MODULES),$(call init_terraform_module,$(module))) @$(foreach module,$(ALL_EXAMPLES),$(call init_terraform_module,$(module))) @@ -358,6 +365,13 @@ tfmodule/lint: tfmodule/init @$(foreach module,$(ALL_EXAMPLES),$(call tflint_terraform_module,$(module))) @$(foreach module,$(ALL_EXAMPLES),$(call validate_terraform_module,$(module))) +.PHONY: tfmodule/check-version-floor +# Deliberately not a prerequisite of tfmodule/lint: it may need to install an +# older Terraform, which would be a surprising cost on a local `make lint`. +# CI invokes it explicitly. +tfmodule/check-version-floor: + @bash .github/scripts/check-terraform-version-floor.sh + .PHONY: tfmodule/list tfmodule/list: @echo -n "Modules: "