From 90bf4cadbc31e0c0e75bce6579ef56225ebfa00e Mon Sep 17 00:00:00 2001 From: waqasm86 Date: Wed, 8 Jul 2026 19:47:19 +0500 Subject: [PATCH 1/6] Validate layered local NVIDIA workflow --- .gitignore | 9 +++ README.md | 20 +++++- docs/ARCHITECTURE.md | 34 ++++++++-- docs/COMMANDS.md | 7 ++ docs/LIVE-VALIDATION-2026-07-08.md | 26 ++++++++ internal/modules/infra/infra.go | 9 ++- .../modules/observability/observability.go | 64 +++++++++++++++---- .../observability/observability_test.go | 53 +++++++++++++++ internal/platform/layers.go | 54 ++++++++++++++++ internal/platform/layers_test.go | 32 ++++++++++ 10 files changed, 286 insertions(+), 22 deletions(-) create mode 100644 docs/LIVE-VALIDATION-2026-07-08.md create mode 100644 internal/platform/layers.go create mode 100644 internal/platform/layers_test.go diff --git a/.gitignore b/.gitignore index 3657520..72b6ca8 100644 --- a/.gitignore +++ b/.gitignore @@ -25,6 +25,10 @@ coverage.* *.bak *.swp *~ +local-run-output/ +validation-output/ +evidence/ +reports/ # Editor and OS metadata .DS_Store @@ -35,8 +39,13 @@ Thumbs.db # Kubernetes and Helm local artifacts kubeconfig *.kubeconfig +*.kubeconfig.yaml /.helm/ /.kube/ +*.tgz +*.tar +*.tar.gz +*.zip # Vendored dependencies are not committed unless intentionally enabled later. /vendor/ diff --git a/README.md b/README.md index 094944c..7ab320b 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ It coordinates two current layers: OpenTelemetry, Prometheus, Grafana, and related services. The CLI is intentionally NVIDIA-only. It does not include AMD, Intel, Apple -Silicon, or CPU-only accelerator abstractions. +Silicon, or CPU-first accelerator abstractions. ## Why this exists @@ -19,7 +19,8 @@ workloads. `edge-cli` provides one installable control plane so operators can ru preflight checks, installs, validation, status, logs, and safe uninstall flows without treating each repository as a disconnected CLI application. -Future repositories can be added as new Go modules under `internal/modules`. +Future repositories can be added as new layers in `internal/platform` and new Go +modules under `internal/modules`. ## Install @@ -99,6 +100,11 @@ edge status 5. Observability validation. 6. Next command and URL hints. +An empty local k3s cluster with only the default k3s system components is a +valid starting point. `edge install infra` is responsible for preparing the +NVIDIA runtime and GPU Operator layer before any GPU-backed LLM workload is +installed. + ## Infra Layer `edge install infra` uses the local `k3s-nvidia-edge` project as the @@ -122,6 +128,15 @@ edge uninstall infra --yes `edge install observability` uses the local `llm-observability-stack` chart. It validates the infra layer first, builds Helm dependencies, installs the chart, waits for Ollama, Open WebUI, and OpenTelemetry Collector, then runs validation. +For GPU profiles, `edge-cli` does not allow bypassing infra validation and +forcibly keeps `gpu-operator`, `nvidia-device-plugin`, and `dcgm-exporter` +disabled in the observability chart. Those components belong to +`k3s-nvidia-edge`. + +On single-GPU laptops, run `edge validate infra` before installing +observability. After Ollama is running and reserving `nvidia.com/gpu: 1`, +observability validation checks base readiness without launching another CUDA +pod that would contend for the only GPU. Useful commands: @@ -178,6 +193,7 @@ commands such as `kubectl`, `helm`, `apt-get`, `systemctl`, `k3s`, and ## Documentation - [Architecture](docs/ARCHITECTURE.md) +- [Live validation - 2026-07-08](docs/LIVE-VALIDATION-2026-07-08.md) - [Command reference](docs/COMMANDS.md) - [Configuration](docs/CONFIGURATION.md) - [Troubleshooting](docs/TROUBLESHOOTING.md) diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index a1c1af1..4e7f3e7 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -5,14 +5,16 @@ ## Layers -The platform is split into three layers: +The platform is split into ordered layers: -- `edge-cli`: command-line control plane, configuration, checks, workflows, and +- Layer 0, `edge-cli`: command-line control plane, configuration, checks, workflows, and module orchestration. -- `k3s-nvidia-edge`: infrastructure layer for Linux, k3s, NVIDIA Container +- Layer 1, `k3s-nvidia-edge`: infrastructure layer for Linux, k3s, NVIDIA Container Toolkit, GPU Operator, RuntimeClass, and CUDA validation. -- `llm-observability-stack`: LLMOps layer for Helm workloads such as Ollama, +- Layer 2, `llm-observability-stack`: LLMOps layer for Helm workloads such as Ollama, Open WebUI, OpenTelemetry Collector, Prometheus, Grafana, and related tools. +- Future Layer 3 repositories, such as a data/storage stack, should declare + their dependency on the previous layer before they can be installed. ## Module Model @@ -34,6 +36,28 @@ interface: - status/logs where relevant - safe uninstall behavior +Layer metadata lives in `internal/platform`. Install workflows run in catalog +order. Uninstall workflows run in reverse order where layers own resources that +depend on earlier layers. + +## Dependency Gates + +`llm-observability-stack` GPU profiles require a valid infra layer first: + +- `RuntimeClass/nvidia` exists +- nodes advertise allocatable `nvidia.com/gpu` +- GPU Operator/device plugin/DCGM are owned by `k3s-nvidia-edge` +- CUDA validation succeeds from the infra layer + +`edge-cli` rejects GPU observability installs that try to skip infra validation. +It also appends Helm overrides that keep `gpu-operator.enabled`, +`nvidia-device-plugin.enabled`, and `dcgm-exporter.enabled` false in the +observability chart. + +Full infra validation owns CUDA pod execution. Observability dependency checks +verify the ready base layer without launching a CUDA pod, so validation remains +usable after Ollama has reserved the only GPU on a low-VRAM laptop. + ## Execution Model The CLI is 100% Go. It does not create Bash scripts. Host and cluster operations @@ -61,5 +85,5 @@ gpu: vendor: nvidia ``` -AMD, Intel, Apple Silicon, and CPU-only accelerator abstractions are out of +AMD, Intel, Apple Silicon, and CPU-first accelerator abstractions are out of scope. diff --git a/docs/COMMANDS.md b/docs/COMMANDS.md index 6dc04df..2509dc8 100644 --- a/docs/COMMANDS.md +++ b/docs/COMMANDS.md @@ -44,6 +44,11 @@ edge install all --yes `edge install all` executes infra preflight, infra install, infra validation, observability install, and observability validation. +For GPU observability profiles, the CLI validates the infra layer first and does +not allow `llm-observability-stack` to install GPU Operator, NVIDIA device +plugin, or DCGM exporter. Start from an empty local k3s cluster by running the +infra install before observability. + ## Validate ```bash @@ -57,6 +62,8 @@ Operator pods, GPU allocatable resources, and a CUDA `nvidia-smi` validation pod `observability` validation checks infra first, then verifies the Helm release, namespace, Ollama, Open WebUI, OpenTelemetry Collector, optional Prometheus/Grafana services, pod readiness, and optional Ollama smoke behavior. +It does not launch a second CUDA pod after Ollama is running, which avoids GPU +contention on single-GPU workstations. ## Uninstall diff --git a/docs/LIVE-VALIDATION-2026-07-08.md b/docs/LIVE-VALIDATION-2026-07-08.md new file mode 100644 index 0000000..4cd49a2 --- /dev/null +++ b/docs/LIVE-VALIDATION-2026-07-08.md @@ -0,0 +1,26 @@ +# Live Validation - 2026-07-08 + +Validated on local Xubuntu 24, single-node k3s, NVIDIA GeForce 940M. + +Layer order tested: + +1. Empty/mostly-empty k3s baseline: CoreDNS and local-path-provisioner only. +2. `edge install observability --skip-infra-check --profile geforce-940m-k3s --yes` refused the GPU install. +3. `edge validate infra` failed before Layer 1 because `nvidia.com/gpu` was not allocatable. +4. `edge install infra --skip-base-package-install --skip-toolkit-install --skip-k3s-install --yes` installed/upgraded the `k3s-nvidia-edge` GPU Operator wrapper. +5. `edge validate infra` passed and ran a CUDA `nvidia-smi` pod. +6. `edge install observability --profile geforce-940m-k3s --yes` deployed Ollama, Open WebUI, Redis, and OpenTelemetry Collector. +7. Repeated install/validate upgraded the observability Helm release safely. +8. `edge uninstall observability --yes` removed Layer 2 while Layer 1 stayed healthy. +9. Reinstall and validation passed again. + +Observed ownership: + +- `k3s-nvidia-edge` Helm release lives in `gpu-operator`. +- GPU Operator, NVIDIA device plugin, DCGM exporter, NFD, RuntimeClass, and `nvidia.com/gpu` are provided by Layer 1. +- `llm-observability-stack` rendered no GPU Operator, NVIDIA device plugin, or DCGM exporter resources. +- `llm-observability-stack` package did not include the old substrate chart dependencies. + +Single-GPU note: + +`edge validate infra` runs CUDA validation before Layer 2 is installed. Once Ollama is running and reserving `nvidia.com/gpu: 1`, observability validation checks base readiness without launching another CUDA pod. diff --git a/internal/modules/infra/infra.go b/internal/modules/infra/infra.go index 6f8a1b1..70324c2 100644 --- a/internal/modules/infra/infra.go +++ b/internal/modules/infra/infra.go @@ -29,6 +29,7 @@ type Options struct { Verbose bool DryRun bool RequireHostCUDA bool + SkipCUDAValidation bool InstallK3sChannel string InstallK3sExec string } @@ -80,7 +81,7 @@ func Status(ctx context.Context, opts Options) error { r.Check(ctx, "nodes", execx.Command{Name: "kubectl", Args: []string{"get", "nodes", "-o", "wide"}}), r.Check(ctx, "runtime classes", execx.Command{Name: "kubectl", Args: []string{"get", "runtimeclass"}}), r.Check(ctx, "helm releases", execx.Command{Name: "helm", Args: []string{"list", "-A"}}), - r.Check(ctx, "GPU Operator values", execx.Command{Name: "helm", Args: []string{"get", "values", "gpu-operator", "-n", "gpu-operator", "-o", "yaml"}}), + r.Check(ctx, "GPU Operator values", execx.Command{Name: "sh", Args: []string{"-c", "helm get values gpu-operator -n gpu-operator -o yaml 2>/dev/null || helm get values k3s-nvidia-edge -n gpu-operator -o yaml"}}), r.Check(ctx, "GPU allocatable", execx.Command{Name: "kubectl", Args: []string{"get", "nodes", "-o", "custom-columns=NAME:.metadata.name,GPU:.status.allocatable.nvidia\\.com/gpu"}}), } return execx.PrintResults(results) @@ -139,6 +140,9 @@ func Validate(ctx context.Context, opts Options) error { if err := execx.PrintResults(results); err != nil { return err } + if opts.SkipCUDAValidation { + return nil + } return cudaValidation(ctx, r, opts) } @@ -250,6 +254,9 @@ func installK3s(ctx context.Context, r execx.Runner, opts Options) error { } func prepareKubeconfig(ctx context.Context, r execx.Runner) error { + if _, err := r.Output(ctx, execx.Command{Name: "kubectl", Args: []string{"cluster-info"}}); err == nil { + return nil + } home, err := os.UserHomeDir() if err != nil { return err diff --git a/internal/modules/observability/observability.go b/internal/modules/observability/observability.go index 947d7cd..fe6a8ca 100644 --- a/internal/modules/observability/observability.go +++ b/internal/modules/observability/observability.go @@ -30,6 +30,12 @@ type Options struct { SkipInfraCheck bool } +var forcedBaseLayerDisables = []string{ + "gpu-operator.enabled=false", + "nvidia-device-plugin.enabled=false", + "dcgm-exporter.enabled=false", +} + func DefaultOptions(repoPath, infraRepoPath, namespace string) Options { return Options{ RepoPath: repoPath, @@ -49,9 +55,7 @@ func Doctor(ctx context.Context, opts Options) error { } r := runner(opts) if !opts.SkipInfraCheck { - infraOpts := infra.DefaultOptions(opts.InfraRepoPath) - infraOpts.Verbose = opts.Verbose - infraOpts.DryRun = opts.DryRun + infraOpts := infraDependencyOptions(opts) if err := infra.Validate(ctx, infraOpts); err != nil { return fmt.Errorf("infra is not ready: %w", err) } @@ -86,10 +90,11 @@ func Install(ctx context.Context, opts Options) error { if err := ValidateRepo(opts.RepoPath); err != nil { return err } + if err := ValidateInstallOptions(opts); err != nil { + return err + } if !opts.SkipInfraCheck { - infraOpts := infra.DefaultOptions(opts.InfraRepoPath) - infraOpts.Verbose = opts.Verbose - infraOpts.DryRun = opts.DryRun + infraOpts := infraDependencyOptions(opts) if err := infra.Validate(ctx, infraOpts); err != nil { return fmt.Errorf("infra must validate before observability install: %w", err) } @@ -115,9 +120,7 @@ func Install(ctx context.Context, opts Options) error { func Validate(ctx context.Context, opts Options) error { if !opts.SkipInfraCheck { - infraOpts := infra.DefaultOptions(opts.InfraRepoPath) - infraOpts.Verbose = opts.Verbose - infraOpts.DryRun = opts.DryRun + infraOpts := infraDependencyOptions(opts) if err := infra.Validate(ctx, infraOpts); err != nil { return fmt.Errorf("infra validation failed: %w", err) } @@ -156,7 +159,7 @@ func Uninstall(ctx context.Context, opts Options) error { return errors.New("uninstall requires --yes; namespace and user data are kept unless deletion is requested by flags") } r := runner(opts) - if err := r.Run(ctx, execx.Command{Name: "helm", Args: []string{"uninstall", opts.Release, "-n", opts.Namespace, "--wait"}, Mutates: true}); err != nil { + if err := r.Run(ctx, execx.Command{Name: "helm", Args: []string{"uninstall", opts.Release, "-n", opts.Namespace, "--wait", "--ignore-not-found"}, Mutates: true}); err != nil { fmt.Printf("observability uninstall returned: %v\n", err) } if !opts.KeepNamespace { @@ -216,10 +219,31 @@ func ProfileValuesFile(profile string) string { } } +func GPUProfile(profile string) bool { + file := ProfileValuesFile(profile) + name := profile + " " + file + return !contains(name, "cpu") +} + +func ValidateInstallOptions(opts Options) error { + if opts.SkipInfraCheck && GPUProfile(opts.Profile) { + return errors.New("cannot skip infra validation for GPU observability profiles; run edge install infra or edge validate infra first") + } + return nil +} + func runner(opts Options) execx.Runner { return execx.Runner{Verbose: opts.Verbose, DryRun: opts.DryRun} } +func infraDependencyOptions(opts Options) infra.Options { + infraOpts := infra.DefaultOptions(opts.InfraRepoPath) + infraOpts.Verbose = opts.Verbose + infraOpts.DryRun = opts.DryRun + infraOpts.SkipCUDAValidation = true + return infraOpts +} + func optional(ctx context.Context, r execx.Runner, label string, cmd execx.Command) execx.Result { res := r.Check(ctx, label+" (optional)", cmd) if res.Err != nil { @@ -237,9 +261,23 @@ func helmInstallArgs(opts Options) []string { for _, set := range opts.SetValues { args = append(args, "--set", set) } + if GPUProfile(opts.Profile) { + for _, set := range forcedBaseLayerDisables { + args = append(args, "--set", set) + } + } return args } +func contains(value, needle string) bool { + for i := 0; i+len(needle) <= len(value); i++ { + if value[i:i+len(needle)] == needle { + return true + } + } + return needle == "" +} + func applyOptionalCRDs(ctx context.Context, r execx.Runner, repoPath string) error { crdDir := filepath.Join(repoPath, "charts", "kube-prometheus-stack", "charts", "crds", "crds") matches, err := filepath.Glob(filepath.Join(crdDir, "*.yaml")) @@ -247,10 +285,8 @@ func applyOptionalCRDs(ctx context.Context, r execx.Runner, repoPath string) err return err } for _, file := range matches { - if err := r.Run(ctx, execx.Command{Name: "kubectl", Args: []string{"create", "--save-config=false", "-f", file}, Mutates: true}); err != nil { - if applyErr := r.Run(ctx, execx.Command{Name: "kubectl", Args: []string{"apply", "--server-side", "-f", file}, Mutates: true}); applyErr != nil { - return applyErr - } + if err := r.Run(ctx, execx.Command{Name: "kubectl", Args: []string{"apply", "--server-side", "-f", file}, Mutates: true}); err != nil { + return err } } return nil diff --git a/internal/modules/observability/observability_test.go b/internal/modules/observability/observability_test.go index d0ec976..57ccf7d 100644 --- a/internal/modules/observability/observability_test.go +++ b/internal/modules/observability/observability_test.go @@ -15,3 +15,56 @@ func TestProfileValuesFile(t *testing.T) { } } } + +func TestGPUInstallCannotSkipInfraCheck(t *testing.T) { + opts := DefaultOptions("/tmp/obs", "/tmp/infra", "llm-observability") + opts.SkipInfraCheck = true + if err := ValidateInstallOptions(opts); err == nil { + t.Fatal("ValidateInstallOptions accepted --skip-infra-check for GPU profile") + } +} + +func TestCPUInstallCanSkipInfraCheck(t *testing.T) { + opts := DefaultOptions("/tmp/obs", "/tmp/infra", "llm-observability") + opts.Profile = "cpu-k3s" + opts.SkipInfraCheck = true + if err := ValidateInstallOptions(opts); err != nil { + t.Fatalf("ValidateInstallOptions rejected CPU skip infra check: %v", err) + } +} + +func TestGPUHelmInstallForcesBaseLayerChartsDisabled(t *testing.T) { + opts := DefaultOptions("/tmp/obs", "/tmp/infra", "llm-observability") + opts.SetValues = []string{"nvidia-device-plugin.enabled=true"} + args := helmInstallArgs(opts) + for _, want := range forcedBaseLayerDisables { + if lastSetValue(args, want[:len(want)-len("=false")]) != want { + t.Fatalf("last override for %q was not forced false in %#v", want, args) + } + } +} + +func TestRepoValidationRequiresChartAndTemplates(t *testing.T) { + dir := t.TempDir() + if err := ValidateRepo(dir); err == nil { + t.Fatal("ValidateRepo accepted incomplete repo") + } +} + +func TestObservabilityDependencyCheckDoesNotConsumeGPU(t *testing.T) { + opts := DefaultOptions("/tmp/obs", "/tmp/infra", "llm-observability") + infraOpts := infraDependencyOptions(opts) + if !infraOpts.SkipCUDAValidation { + t.Fatal("observability dependency check should skip CUDA pod validation") + } +} + +func lastSetValue(args []string, key string) string { + last := "" + for i := 0; i < len(args)-1; i++ { + if args[i] == "--set" && len(args[i+1]) > len(key) && args[i+1][:len(key)+1] == key+"=" { + last = args[i+1] + } + } + return last +} diff --git a/internal/platform/layers.go b/internal/platform/layers.go new file mode 100644 index 0000000..8df079b --- /dev/null +++ b/internal/platform/layers.go @@ -0,0 +1,54 @@ +package platform + +type Operation string + +const ( + Doctor Operation = "doctor" + Install Operation = "install" + Validate Operation = "validate" + Status Operation = "status" + Logs Operation = "logs" + Uninstall Operation = "uninstall" +) + +type Layer struct { + Name string + RepoName string + DependsOn []string + Operations []Operation + UninstallRev bool +} + +func Catalog() []Layer { + return []Layer{ + { + Name: "infra", + RepoName: "k3s-nvidia-edge", + Operations: []Operation{Doctor, Install, Validate, Status, Uninstall}, + }, + { + Name: "observability", + RepoName: "llm-observability-stack", + DependsOn: []string{"infra"}, + Operations: []Operation{Doctor, Install, Validate, Status, Logs, Uninstall}, + UninstallRev: true, + }, + } +} + +func InstallOrder() []string { + layers := Catalog() + order := make([]string, 0, len(layers)) + for _, layer := range layers { + order = append(order, layer.Name) + } + return order +} + +func UninstallOrder() []string { + install := InstallOrder() + for left, right := 0, len(install)-1; left < right; left, right = left+1, right-1 { + install[left], install[right] = install[right], install[left] + } + return install +} diff --git a/internal/platform/layers_test.go b/internal/platform/layers_test.go new file mode 100644 index 0000000..fd45253 --- /dev/null +++ b/internal/platform/layers_test.go @@ -0,0 +1,32 @@ +package platform + +import "testing" + +func TestLayerDependencyOrder(t *testing.T) { + got := InstallOrder() + want := []string{"infra", "observability"} + if len(got) != len(want) { + t.Fatalf("InstallOrder length = %d, want %d", len(got), len(want)) + } + for i := range want { + if got[i] != want[i] { + t.Fatalf("InstallOrder[%d] = %q, want %q", i, got[i], want[i]) + } + } + + uninstall := UninstallOrder() + if uninstall[0] != "observability" || uninstall[1] != "infra" { + t.Fatalf("UninstallOrder = %#v, want reverse layer order", uninstall) + } +} + +func TestCatalogKeepsFutureLayersAdditive(t *testing.T) { + for _, layer := range Catalog() { + if layer.Name == "" || layer.RepoName == "" { + t.Fatalf("layer must declare name and repo: %#v", layer) + } + if len(layer.Operations) == 0 { + t.Fatalf("layer %q must declare supported operations", layer.Name) + } + } +} From 171a07afb79ce6daaf083ce13bf3e2ebf16fa2a8 Mon Sep 17 00:00:00 2001 From: waqasm86 Date: Fri, 17 Jul 2026 23:25:38 +0500 Subject: [PATCH 2/6] Add automatic CPU and NVIDIA orchestration --- README.md | 31 +++++-- docs/ACCELERATOR-MODES.md | 43 +++++++++ docs/LIVE-VALIDATION-2026-07-17.md | 63 ++++++++++++++ docs/LOCAL-DEPENDENCY-INVENTORY.md | 87 +++++++++++++++++++ internal/app/app.go | 4 +- internal/cli/accelerator_test.go | 12 +++ internal/cli/root.go | 84 +++++++++++++++--- internal/modules/infra/infra.go | 72 +++++++++++---- .../modules/observability/observability.go | 61 ++++++++++++- .../observability/observability_test.go | 34 +++++++- internal/nvidia/nvidia.go | 26 ++++++ internal/nvidia/nvidia_test.go | 24 +++++ 12 files changed, 497 insertions(+), 44 deletions(-) create mode 100644 docs/ACCELERATOR-MODES.md create mode 100644 docs/LIVE-VALIDATION-2026-07-17.md create mode 100644 docs/LOCAL-DEPENDENCY-INVENTORY.md create mode 100644 internal/cli/accelerator_test.go create mode 100644 internal/nvidia/nvidia_test.go diff --git a/README.md b/README.md index 7ab320b..4489727 100644 --- a/README.md +++ b/README.md @@ -9,8 +9,8 @@ It coordinates two current layers: - `llm-observability-stack`: LLMOps workloads, Helm charts, Open WebUI, Ollama, OpenTelemetry, Prometheus, Grafana, and related services. -The CLI is intentionally NVIDIA-only. It does not include AMD, Intel, Apple -Silicon, or CPU-first accelerator abstractions. +The CLI supports automatic NVIDIA/CPU selection. It does not currently install +AMD, Intel, or Apple Silicon accelerator runtimes. ## Why this exists @@ -52,8 +52,8 @@ Default configuration: ```yaml repos: - k3sNvidiaEdge: /media/waqasm86/External1/Waqas-Projects/Project-Edge-Computing-LLM/k3s-nvidia-edge - llmObservabilityStack: /media/waqasm86/External1/Waqas-Projects/Project-Edge-Computing-LLM/llm-observability-stack + k3sNvidiaEdge: /media/waqasm86/External1/Waqas-Projects/Project-Linux-Kubernetes-Nvidia/Project-Edge-Computing-LLM/k3s-nvidia-edge + llmObservabilityStack: /media/waqasm86/External1/Waqas-Projects/Project-Linux-Kubernetes-Nvidia/Project-Edge-Computing-LLM/llm-observability-stack cluster: kubeconfig: "" defaultNamespace: llm-observability @@ -86,16 +86,26 @@ edge repo doctor ## Full Install ```bash -edge doctor -edge install all --yes +edge install all --accelerator auto --yes edge status ``` +`auto` deploys the NVIDIA infrastructure layer when `nvidia-smi` detects a +working GPU. Without NVIDIA hardware it skips the toolkit and GPU Operator, +installs or validates basic k3s, and selects `values.cpu-k3s.yaml`. + +Explicit modes are useful in automation: + +```bash +edge install all --accelerator nvidia --yes +edge install all --accelerator cpu --yes +``` + `edge install all` runs: -1. Infra diagnostics. -2. Linux/k3s/NVIDIA GPU infrastructure install. -3. Infra validation and CUDA validation pod. +1. Host accelerator detection. +2. Linux/k3s installation and accelerator-specific infrastructure setup. +3. Basic k3s validation, plus NVIDIA and CUDA validation in NVIDIA mode. 4. `llm-observability-stack` Helm install. 5. Observability validation. 6. Next command and URL hints. @@ -193,6 +203,9 @@ commands such as `kubectl`, `helm`, `apt-get`, `systemctl`, `k3s`, and ## Documentation - [Architecture](docs/ARCHITECTURE.md) +- [Accelerator selection](docs/ACCELERATOR-MODES.md) +- [Local dependency and repository inventory](docs/LOCAL-DEPENDENCY-INVENTORY.md) +- [Live validation - 2026-07-17](docs/LIVE-VALIDATION-2026-07-17.md) - [Live validation - 2026-07-08](docs/LIVE-VALIDATION-2026-07-08.md) - [Command reference](docs/COMMANDS.md) - [Configuration](docs/CONFIGURATION.md) diff --git a/docs/ACCELERATOR-MODES.md b/docs/ACCELERATOR-MODES.md new file mode 100644 index 0000000..6e331a3 --- /dev/null +++ b/docs/ACCELERATOR-MODES.md @@ -0,0 +1,43 @@ +# Accelerator selection + +`edge install all` and `edge install observability` accept +`--accelerator auto|nvidia|cpu`. + +## Automatic mode + +For the full layered installation, `auto` probes the host with `nvidia-smi` before +making changes: + +- NVIDIA detected: install Ubuntu prerequisites, NVIDIA Container Toolkit, k3s, + `k3s-nvidia-edge`, and the NVIDIA observability profile. +- NVIDIA not detected: skip NVIDIA toolkit and GPU Operator components, install or + validate basic k3s, and use `values.cpu-k3s.yaml`. + +For an observability-only installation, `auto` inspects Kubernetes allocatable +resources. It selects NVIDIA only when at least one node advertises a positive +`nvidia.com/gpu` value; otherwise it selects CPU. + +## Explicit modes + +Use `--accelerator nvidia` where the GPU layer must be present and a missing GPU +should fail validation. Use `--accelerator cpu` to guarantee that no NVIDIA runtime +class, GPU request, GPU Operator, device plugin, or DCGM dependency is required by +the application layer. + +An explicit `--profile` on `edge install observability` takes precedence over +accelerator-based profile selection. + +## Detection boundaries + +Host detection answers whether NVIDIA infrastructure should be installed. Cluster +detection answers whether a workload can request a Kubernetes GPU. A host may pass +`nvidia-smi` while Kubernetes still reports no GPU; that means the GPU layer needs +installation or repair rather than that a GPU workload is ready. + +## Interrupted first installs + +If a first Helm client times out while large images continue pulling, Kubernetes +may later become healthy while Helm remains `pending-install`. With `--yes`, the +observability installer detects that exact state, reconciles the recorded revision, +and then proceeds with the normal idempotent upgrade. Other pending Helm states are +not rewritten automatically. diff --git a/docs/LIVE-VALIDATION-2026-07-17.md b/docs/LIVE-VALIDATION-2026-07-17.md new file mode 100644 index 0000000..ea7438e --- /dev/null +++ b/docs/LIVE-VALIDATION-2026-07-17.md @@ -0,0 +1,63 @@ +# Live validation — 2026-07-17 + +## Host + +- Ubuntu 24.04.3 LTS +- k3s v1.36.2+k3s1 with containerd 2.3.2-k3s2 +- NVIDIA GeForce 940M, 1 GiB VRAM +- NVIDIA driver 580.95.05, reported CUDA capability 13.0 +- Helm v4.2.3 +- Go 1.26.5 +- Python 3.11.15 at `/usr/local/bin/python3.11` + +## Accelerator orchestration + +The new `--accelerator auto` cluster probe selected CPU before the NVIDIA device +plugin advertised `nvidia.com/gpu`, then selected NVIDIA after Layer 1 completed. +Explicit CPU dry-run rendered `values.cpu-k3s.yaml` and skipped NVIDIA toolkit, +GPU Operator, RuntimeClass, DCGM, and GPU scheduling checks. + +## NVIDIA layer + +`edge install infra --yes` installed NVIDIA Container Toolkit 1.19.1 and the local +`k3s-nvidia-edge` GPU Operator chart v26.3.3. GPU Operator, Node Feature Discovery, +the toolkit, device plugin, validator, and DCGM Exporter became healthy. Kubernetes +advertised one `nvidia.com/gpu` resource. + +The CUDA validation pod used `nvidia/cuda:12.8.1-base-ubuntu24.04`, completed, and +printed the host GeForce 940M through `nvidia-smi`. + +## Repository tests + +- `go test ./...`: pass +- `go vet ./...`: pass +- Go builds for all three CLIs: pass +- Helm lint and default/CPU/NVIDIA renders: pass +- Python 3.11 smoke suite: 17 passed +- Frontend typecheck and production build: pass +- CPU full-install dry-run: pass +- Live NVIDIA infra validation with `--skip-cuda` while the GPU was reserved: pass + +The frontend build reports a non-failing Vite warning for a JavaScript chunk over +500 kB. It is a performance optimization opportunity, not a correctness failure. + +## Operational observations + +First-time Ollama and Open WebUI pulls are multi-gigabyte operations on this node. +The initial application Helm wait reached 15 minutes while those pulls continued. +Ollama took approximately 25 minutes and Open WebUI approximately 33 minutes to +pull. Redis, OpenTelemetry Collector, PVC provisioning, GPU scheduling, and +services were already healthy. + +After the pulls completed, Helm revision 1 still recorded `pending-install` +because the original client had timed out. `helm rollback ... 1` reconciled the +already healthy resources as deployed, and unchanged idempotent installs then +completed successfully in under 20 seconds. Final Helm revision 4 is `deployed`. + +Final live application checks passed: + +- all four application pods Ready with zero restarts; +- Open WebUI `/health` returned `{"status":true}`; +- Ollama loaded `gemma3-1b-it-gguf-local` and returned exactly `validation ok`; +- Ollama selected CUDA on the GeForce 940M and offloaded 23 of 27 layers; +- `ollama ps` reported a 57% CPU / 43% GPU split for the 1.2 GB loaded model. diff --git a/docs/LOCAL-DEPENDENCY-INVENTORY.md b/docs/LOCAL-DEPENDENCY-INVENTORY.md new file mode 100644 index 0000000..c708506 --- /dev/null +++ b/docs/LOCAL-DEPENDENCY-INVENTORY.md @@ -0,0 +1,87 @@ +# Local dependency and repository inventory + +Validated on 2026-07-17 against +`/media/waqasm86/External1/Waqas-Projects/Project-Linux-Kubernetes-Nvidia`. +The scan found 58 local Git worktrees. This document distinguishes the platform's +direct source/runtime dependencies from incidental links in vendored charts, +lockfiles, generated package metadata, and documentation. + +## Layer ownership + +| Layer | Organization repository | Responsibility | +|---|---|---| +| Control | `Edge-Computing-LLM/edge-cli` | Detect the accelerator, orchestrate installation, and expose status and validation commands. | +| Infrastructure | `Edge-Computing-LLM/k3s-nvidia-edge` | Install or validate k3s and, conditionally, NVIDIA Container Toolkit, GPU Operator, RuntimeClass, device plugin, and DCGM. | +| Application | `Edge-Computing-LLM/llm-observability-stack` | Deploy CPU- or GPU-profiled Ollama, Open WebUI, OpenTelemetry, Prometheus, and Grafana workloads. | +| Dashboard | `Edge-Computing-LLM/Frontend-Edge-LLM-Observability` | Present LLM, Kubernetes, and accelerator telemetry. | + +The NVIDIA infrastructure layer is conditional. CPU hosts skip its GPU-specific +components and deploy the application layer with `values.cpu-k3s.yaml`. + +## Direct upstream repositories available locally + +The following major upstreams used by installation, charts, runtime images, or +platform operations are present under the scanned parent directory: + +- Kubernetes/k3s: `k3s-io/k3s`, `kubernetes/kubectl`, + `kubernetes/client-go`, `rancher/local-path-provisioner`, + `kubernetes-sigs/metrics-server`, `kubernetes-sigs/node-feature-discovery`, + `coredns/coredns`, and `traefik/traefik`. +- NVIDIA: `NVIDIA/gpu-operator`, `NVIDIA/nvidia-container-toolkit`, + `NVIDIA/libnvidia-container`, `NVIDIA/k8s-device-plugin`, `NVIDIA/DCGM`, + `NVIDIA/dcgm-exporter`, `NVIDIA/go-dcgm`, and `NVIDIA/cuda-samples`. +- LLM applications: `ollama/ollama`, `otwld/ollama-helm`, + `open-webui/open-webui`, `open-webui/helm-charts`, and + `open-webui/pipelines`. +- Observability: `prometheus-community/helm-charts`, + `prometheus-operator/prometheus-operator`, + `prometheus-operator/kube-prometheus`, `grafana/helm-charts`, and the + OpenTelemetry collector, collector-contrib, collector-releases, operator, + Helm charts, Python, Python contrib, and GenAI repositories. +- Supporting tools: `helm/helm`, `cloudflare/cloudflared`, + `apache/tika-helm`, and `apache/echarts`. + +The application chart vendors its exact Helm dependencies under `charts/`, so a +normal local install does not require sibling source clones to build those chart +dependencies. + +## Referenced repositories not available locally + +These are source repositories for direct dependencies or operational tools. Their +absence does not block the validated deployment because released Go modules, +Python packages, npm packages, Helm charts, or container images are consumed +instead. + +| Area | Missing source repository | How it is consumed | +|---|---|---| +| edge-cli | `spf13/cobra`, `spf13/pflag`, `go-yaml/yaml`, `inconshreveable/mousetrap` | Go modules | +| Kubernetes | `kubernetes/kubernetes` | Released k3s/Kubernetes binaries and APIs | +| NVIDIA | `NVIDIA/k8s-dra-driver-gpu` | Reference/documentation; the current platform uses the device plugin and GPU Operator path | +| Metrics | `prometheus/prometheus`, `prometheus/alertmanager`, `prometheus/node_exporter`, `prometheus/blackbox_exporter`, `kubernetes/kube-state-metrics`, `grafana/grafana` | Vendored Helm charts and released images | +| Python service | `fastapi/fastapi`, `encode/uvicorn`, `pydantic/pydantic`, `langchain-ai/langchain`, `encode/httpx`, `prometheus/client_python` | Pinned Python 3.11 packages in `langchain-demo/requirements.txt` | +| Frontend | `ag-grid/ag-grid`, `primefaces/primevue`, `primefaces/primeicons`, `vuejs/core`, `ecomfe/vue-echarts`, `tailwindlabs/tailwindcss`, `vitejs/vite` | Locked npm packages; `apache/echarts` is already local | +| Tika | `apache/tika`, `apache/tika-docker` | Released chart/image; `apache/tika-helm` is local | +| Documentation tooling | `norwoodj/helm-docs`, `helm-unittest/helm-unittest` | Optional development tooling | + +Transitive repositories appearing only inside vendored charts or npm/Go package +metadata are intentionally not treated as required local clones. Cloning every +transitive source would not make the build more reproducible; the relevant lock +files, vendored charts, image tags, and package versions are the reproducibility +boundary. + +## Local duplicates and naming notes + +- `prometheus-community/helm-charts` and + `open-telemetry/opentelemetry-helm-charts` each have duplicate worktrees. One + clean, current clone is sufficient. +- The NVIDIA DRA repository referenced as `NVIDIA/k8s-dra-driver-gpu` is distinct + from the locally available `kubernetes-sigs/dra-driver-nvidia-gpu`. +- `Frontend-Edge-LLM-Observability` is the actual organization repository name; + use that case in documentation and remotes. + +## Audit method + +The inventory combined Git remote enumeration for every `.git` worktree with +repository URL extraction, Go module files, Helm `Chart.yaml`/`Chart.lock`, Python +requirements, npm lock metadata, CI workflows, and runtime manifests from all four +organization projects. Credentials and generated build output were excluded. diff --git a/internal/app/app.go b/internal/app/app.go index b4c4485..2a86e7d 100644 --- a/internal/app/app.go +++ b/internal/app/app.go @@ -7,6 +7,6 @@ const ( InfraRepoName = "k3s-nvidia-edge" ObservabilityRepoName = "llm-observability-stack" - DefaultInfraRepoPath = "/media/waqasm86/External1/Waqas-Projects/Project-Edge-Computing-LLM/k3s-nvidia-edge" - DefaultObsRepoPath = "/media/waqasm86/External1/Waqas-Projects/Project-Edge-Computing-LLM/llm-observability-stack" + DefaultInfraRepoPath = "/media/waqasm86/External1/Waqas-Projects/Project-Linux-Kubernetes-Nvidia/Project-Edge-Computing-LLM/k3s-nvidia-edge" + DefaultObsRepoPath = "/media/waqasm86/External1/Waqas-Projects/Project-Linux-Kubernetes-Nvidia/Project-Edge-Computing-LLM/llm-observability-stack" ) diff --git a/internal/cli/accelerator_test.go b/internal/cli/accelerator_test.go new file mode 100644 index 0000000..90ef9da --- /dev/null +++ b/internal/cli/accelerator_test.go @@ -0,0 +1,12 @@ +package cli + +import "testing" + +func TestProfileForAccelerator(t *testing.T) { + if got := profileForAccelerator("cpu"); got != "cpu-k3s" { + t.Fatalf("CPU profile = %q", got) + } + if got := profileForAccelerator("nvidia"); got != "geforce-940m-k3s" { + t.Fatalf("NVIDIA profile = %q", got) + } +} diff --git a/internal/cli/root.go b/internal/cli/root.go index 7dd0655..79b21b7 100644 --- a/internal/cli/root.go +++ b/internal/cli/root.go @@ -11,6 +11,7 @@ import ( "github.com/Edge-Computing-LLM/edge-cli/internal/execx" "github.com/Edge-Computing-LLM/edge-cli/internal/modules/infra" "github.com/Edge-Computing-LLM/edge-cli/internal/modules/observability" + "github.com/Edge-Computing-LLM/edge-cli/internal/nvidia" "github.com/spf13/cobra" ) @@ -159,6 +160,7 @@ func installObsCmd(g *globals) *cobra.Command { var values repeated var sets repeated var skipInfra bool + var accelerator string c := &cobra.Command{ Use: "observability", Short: "Install llm-observability-stack", @@ -174,7 +176,15 @@ func installObsCmd(g *globals) *cobra.Command { defer cancel() opts := obsOpts(cfg, g, repoPath, values, sets) opts.Yes = yes - opts.Profile = first(profile, opts.Profile) + if profile != "" { + opts.Profile = profile + } else { + mode, err := resolveAccelerator(ctx, accelerator, true, g) + if err != nil { + return err + } + opts.Profile = profileForAccelerator(mode) + } opts.SkipInfraCheck = skipInfra return observability.Install(ctx, opts) }, @@ -185,12 +195,14 @@ func installObsCmd(g *globals) *cobra.Command { c.Flags().Var(&values, "values", "additional Helm values file; may be repeated") c.Flags().Var(&sets, "set", "additional Helm --set override; may be repeated") c.Flags().BoolVar(&skipInfra, "skip-infra-check", false, "skip infra validation before install") + c.Flags().StringVar(&accelerator, "accelerator", "auto", "runtime accelerator: auto, nvidia, or cpu") return c } func installAllCmd(g *globals) *cobra.Command { var yes bool var infraRepoPath, obsRepoPath string + var accelerator string c := &cobra.Command{ Use: "all", Short: "Install infra, validate it, install observability, validate everything", @@ -204,22 +216,33 @@ func installAllCmd(g *globals) *cobra.Command { return err } defer cancel() + mode, err := resolveAccelerator(ctx, accelerator, false, g) + if err != nil { + return err + } + fmt.Printf("Selected accelerator: %s\n", mode) iopts := infraOpts(cfg, g, infraRepoPath) iopts.Yes = yes - if err := infra.Doctor(ctx, iopts); err != nil { - return err + if mode == "cpu" { + iopts.NVIDIAEnabled = false + iopts.SkipToolkitInstall = true + iopts.SkipGPUOperator = true + iopts.SkipCUDAValidation = true + iopts.InstallK3sExec = "server --write-kubeconfig-mode 0644 --disable traefik --disable servicelb --disable metrics-server --node-label workload=edge-ai" } if err := infra.Install(ctx, iopts); err != nil { return err } - if err := infra.Validate(ctx, iopts); err != nil { - return err - } oopts := obsOpts(cfg, g, obsRepoPath, nil, nil) oopts.Yes = yes + oopts.Profile = profileForAccelerator(mode) + oopts.SkipInfraCheck = g.DryRun if err := observability.Install(ctx, oopts); err != nil { return err } + if g.DryRun { + return nil + } if err := observability.Validate(ctx, oopts); err != nil { return err } @@ -233,6 +256,7 @@ func installAllCmd(g *globals) *cobra.Command { c.Flags().BoolVar(&yes, "yes", false, "execute mutating operations") c.Flags().StringVar(&infraRepoPath, "infra-repo-path", "", "path to k3s-nvidia-edge repo") c.Flags().StringVar(&obsRepoPath, "observability-repo-path", "", "path to llm-observability-stack repo") + c.Flags().StringVar(&accelerator, "accelerator", "auto", "runtime accelerator: auto, nvidia, or cpu") return c } @@ -330,7 +354,8 @@ func validateCmd(g *globals) *cobra.Command { } func validateInfraCmd(g *globals) *cobra.Command { - return &cobra.Command{ + var skipCUDA bool + c := &cobra.Command{ Use: "infra", Short: "Validate k3s + NVIDIA GPU readiness", RunE: func(cmd *cobra.Command, args []string) error { @@ -343,9 +368,13 @@ func validateInfraCmd(g *globals) *cobra.Command { return err } defer cancel() - return infra.Validate(ctx, infraOpts(cfg, g, "")) + opts := infraOpts(cfg, g, "") + opts.SkipCUDAValidation = skipCUDA + return infra.Validate(ctx, opts) }, } + c.Flags().BoolVar(&skipCUDA, "skip-cuda", false, "skip the CUDA pod when the GPU is occupied by a workload") + return c } func validateObsCmd(g *globals) *cobra.Command { @@ -500,7 +529,7 @@ func obsOpts(cfg config.Config, g *globals, repoPath string, values, sets []stri opts := observability.DefaultOptions(first(repoPath, cfg.Repos.LLMObservabilityStack), cfg.Repos.K3sNvidiaEdge, cfg.Cluster.DefaultNamespace) opts.Verbose = g.Verbose opts.DryRun = g.DryRun - opts.Timeout = "5m" + opts.Timeout = g.Timeout opts.ValuesFiles = values opts.SetValues = sets return opts @@ -511,7 +540,9 @@ func contextWithTimeout(value string) (context.Context, context.CancelFunc, erro if err != nil { return nil, nil, fmt.Errorf("invalid --timeout: %w", err) } - ctx, cancel := context.WithTimeout(context.Background(), d) + // Give Helm and kubectl enough time to report their own timeout diagnostics + // and perform cleanup before the parent process is cancelled. + ctx, cancel := context.WithTimeout(context.Background(), d+30*time.Second) return ctx, cancel, nil } @@ -523,3 +554,36 @@ func first(values ...string) string { } return "" } + +func resolveAccelerator(ctx context.Context, requested string, cluster bool, g *globals) (string, error) { + if requested == "" { + requested = "auto" + } + switch requested { + case "cpu", "nvidia": + return requested, nil + case "auto": + default: + return "", fmt.Errorf("invalid accelerator %q: must be auto, nvidia, or cpu", requested) + } + r := execx.Runner{Verbose: g.Verbose} + if cluster { + if nvidia.ClusterAvailable(ctx, r) { + return "nvidia", nil + } + fmt.Println("Kubernetes does not advertise nvidia.com/gpu; selecting CPU profile") + return "cpu", nil + } + if nvidia.HostAvailable(ctx, r) { + return "nvidia", nil + } + fmt.Println("No working NVIDIA GPU detected on the host; skipping NVIDIA components") + return "cpu", nil +} + +func profileForAccelerator(accelerator string) string { + if accelerator == "cpu" { + return "cpu-k3s" + } + return "geforce-940m-k3s" +} diff --git a/internal/modules/infra/infra.go b/internal/modules/infra/infra.go index 70324c2..64b5fec 100644 --- a/internal/modules/infra/infra.go +++ b/internal/modules/infra/infra.go @@ -6,6 +6,7 @@ import ( "fmt" "os" "path/filepath" + "time" "github.com/Edge-Computing-LLM/edge-cli/internal/execx" "github.com/Edge-Computing-LLM/edge-cli/internal/kubernetes" @@ -32,6 +33,7 @@ type Options struct { SkipCUDAValidation bool InstallK3sChannel string InstallK3sExec string + NVIDIAEnabled bool } func DefaultOptions(repoPath string) Options { @@ -44,6 +46,7 @@ func DefaultOptions(repoPath string) Options { RequireHostCUDA: false, InstallK3sChannel: "stable", InstallK3sExec: "server --write-kubeconfig-mode 0644 --disable traefik --disable servicelb --disable metrics-server --node-label gpu=nvidia --node-label workload=edge-ai", + NVIDIAEnabled: true, } } @@ -55,8 +58,11 @@ func Doctor(ctx context.Context, opts Options) error { } else { results = append(results, execx.Result{Label: "Linux OS", Output: out}) } - results = append(results, execx.RequiredCommands("sudo", "curl", "apt-get", "systemctl", "kubectl", "helm", "nvidia-smi")...) - results = append(results, nvidia.Checks(ctx, r)...) + results = append(results, execx.RequiredCommands("sudo", "curl", "apt-get", "systemctl", "kubectl", "helm")...) + if opts.NVIDIAEnabled { + results = append(results, execx.RequiredCommands("nvidia-smi")...) + results = append(results, nvidia.Checks(ctx, r)...) + } results = append(results, r.Check(ctx, "k3s service", execx.Command{Name: "systemctl", Args: []string{"is-active", "k3s"}}), r.Check(ctx, "kubectl status", execx.Command{Name: "kubectl", Args: []string{"cluster-info"}}), @@ -64,13 +70,15 @@ func Doctor(ctx context.Context, opts Options) error { r.Check(ctx, "containerd runtime", execx.Command{Name: "systemctl", Args: []string{"is-active", "k3s"}}), ) k := kubernetes.Client{Runner: r} - results = append(results, - k.Nodes(ctx), - k.RuntimeClassNvidia(ctx), - r.Check(ctx, "GPU Operator pods", execx.Command{Name: "kubectl", Args: []string{"get", "pods", "-n", "gpu-operator", "-o", "wide"}}), - r.Check(ctx, "GPU allocatable", execx.Command{Name: "kubectl", Args: []string{"get", "nodes", "-o", "custom-columns=NAME:.metadata.name,GPU:.status.allocatable.nvidia\\.com/gpu"}}), - r.Check(ctx, "CUDA validation support", execx.Command{Name: "kubectl", Args: []string{"get", "runtimeclass", "nvidia"}}), - ) + results = append(results, k.Nodes(ctx)) + if opts.NVIDIAEnabled { + results = append(results, + k.RuntimeClassNvidia(ctx), + r.Check(ctx, "GPU Operator pods", execx.Command{Name: "kubectl", Args: []string{"get", "pods", "-n", "gpu-operator", "-o", "wide"}}), + gpuCapacityResult(ctx, r), + r.Check(ctx, "CUDA validation support", execx.Command{Name: "kubectl", Args: []string{"get", "runtimeclass", "nvidia"}}), + ) + } return execx.PrintResults(results) } @@ -88,6 +96,9 @@ func Status(ctx context.Context, opts Options) error { } func Install(ctx context.Context, opts Options) error { + if !opts.Yes && !opts.DryRun { + return errors.New("install requires --yes; use --dry-run to preview changes") + } if err := ValidateRepo(opts.RepoPath); err != nil { return err } @@ -103,7 +114,7 @@ func Install(ctx context.Context, opts Options) error { return err } } - if !opts.SkipToolkitInstall { + if opts.NVIDIAEnabled && !opts.SkipToolkitInstall { if err := installNvidiaToolkit(ctx, r); err != nil { return err } @@ -119,11 +130,14 @@ func Install(ctx context.Context, opts Options) error { if err := r.Run(ctx, execx.Command{Name: "kubectl", Args: []string{"wait", "--for=condition=Ready", "node", "--all", "--timeout=180s"}, Mutates: true}); err != nil { return err } - if !opts.SkipGPUOperator { + if opts.NVIDIAEnabled && !opts.SkipGPUOperator { if err := installGPUOperator(ctx, r, opts); err != nil { return err } } + if opts.DryRun { + return nil + } return Validate(ctx, opts) } @@ -133,14 +147,18 @@ func Validate(ctx context.Context, opts Options) error { results := []execx.Result{ k.ClusterInfo(ctx), k.Nodes(ctx), - k.RuntimeClassNvidia(ctx), - r.Check(ctx, "GPU Operator pods", execx.Command{Name: "kubectl", Args: []string{"get", "pods", "-n", "gpu-operator", "-o", "wide"}}), - r.Check(ctx, "GPU allocatable", execx.Command{Name: "kubectl", Args: []string{"get", "nodes", "-o", "custom-columns=NAME:.metadata.name,GPU:.status.allocatable.nvidia\\.com/gpu"}}), + } + if opts.NVIDIAEnabled { + results = append(results, + k.RuntimeClassNvidia(ctx), + r.Check(ctx, "GPU Operator pods", execx.Command{Name: "kubectl", Args: []string{"get", "pods", "-n", "gpu-operator", "-o", "wide"}}), + gpuCapacityResult(ctx, r), + ) } if err := execx.PrintResults(results); err != nil { return err } - if opts.SkipCUDAValidation { + if !opts.NVIDIAEnabled || opts.SkipCUDAValidation { return nil } return cudaValidation(ctx, r, opts) @@ -186,6 +204,16 @@ func runner(opts Options) execx.Runner { return execx.Runner{Verbose: opts.Verbose, DryRun: opts.DryRun} } +func gpuCapacityResult(ctx context.Context, r execx.Runner) execx.Result { + out, err := r.Output(ctx, execx.Command{Name: "kubectl", Args: []string{ + "get", "nodes", "-o", "jsonpath={range .items[*]}{.status.allocatable.nvidia\\.com/gpu}{\"\\n\"}{end}", + }}) + if err == nil && !nvidia.HasAllocatableGPU(out) { + err = fmt.Errorf("no node advertises a positive %s allocatable resource", nvidia.GPUResource) + } + return execx.Result{Label: "GPU allocatable", Output: out, Err: err} +} + func hostPreflight(ctx context.Context, opts Options) error { r := runner(opts) results := []execx.Result{} @@ -194,8 +222,11 @@ func hostPreflight(ctx context.Context, opts Options) error { } else { results = append(results, execx.Result{Label: "Linux OS", Output: out}) } - results = append(results, execx.RequiredCommands("sudo", "curl", "apt-get", "systemctl", "kubectl", "helm", "nvidia-smi")...) - results = append(results, r.Check(ctx, "NVIDIA driver", execx.Command{Name: "nvidia-smi"})) + results = append(results, execx.RequiredCommands("sudo", "curl", "apt-get", "systemctl", "kubectl", "helm")...) + if opts.NVIDIAEnabled { + results = append(results, execx.RequiredCommands("nvidia-smi")...) + results = append(results, r.Check(ctx, "NVIDIA driver", execx.Command{Name: "nvidia-smi"})) + } if err := execx.PrintResults(results); err != nil { return err } @@ -327,13 +358,18 @@ func cudaValidation(ctx context.Context, r execx.Runner, opts Options) error { if err := r.Run(ctx, execx.Command{Name: "kubectl", Args: []string{"apply", "-f", tmp}, Mutates: true}); err != nil { return err } + defer func() { + cleanupCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second) + defer cancel() + _ = r.Run(cleanupCtx, execx.Command{Name: "kubectl", Args: []string{"delete", "pod", "cuda-test", "--ignore-not-found"}, Mutates: true}) + }() if err := r.Run(ctx, execx.Command{Name: "kubectl", Args: []string{"wait", "--for=jsonpath={.status.phase}=Succeeded", "pod/cuda-test", "--timeout=180s"}, Mutates: true}); err != nil { return err } if err := r.Run(ctx, execx.Command{Name: "kubectl", Args: []string{"logs", "cuda-test"}}); err != nil { return err } - return r.Run(ctx, execx.Command{Name: "kubectl", Args: []string{"delete", "pod", "cuda-test"}, Mutates: true}) + return nil } func cudaManifest(image string) string { diff --git a/internal/modules/observability/observability.go b/internal/modules/observability/observability.go index fe6a8ca..088afbb 100644 --- a/internal/modules/observability/observability.go +++ b/internal/modules/observability/observability.go @@ -2,10 +2,13 @@ package observability import ( "context" + "encoding/json" "errors" "fmt" "os" "path/filepath" + "strconv" + "strings" "github.com/Edge-Computing-LLM/edge-cli/internal/execx" "github.com/Edge-Computing-LLM/edge-cli/internal/kubernetes" @@ -77,16 +80,23 @@ func Doctor(ctx context.Context, opts Options) error { func Status(ctx context.Context, opts Options) error { r := runner(opts) results := []execx.Result{ - r.Check(ctx, "base GPU layer", execx.Command{Name: "kubectl", Args: []string{"get", "pods", "-n", "gpu-operator", "-o", "wide"}}), - r.Check(ctx, "NVIDIA RuntimeClass", execx.Command{Name: "kubectl", Args: []string{"get", "runtimeclass", "nvidia"}}), r.Check(ctx, "Helm release", execx.Command{Name: "helm", Args: []string{"status", opts.Release, "-n", opts.Namespace}}), r.Check(ctx, "LLM workloads", execx.Command{Name: "kubectl", Args: []string{"get", "pods,deploy,statefulset,svc,pvc", "-n", opts.Namespace, "-o", "wide"}}), r.Check(ctx, "Ollama models", execx.Command{Name: "kubectl", Args: []string{"exec", "-n", opts.Namespace, "deploy/ollama", "--", "ollama", "list"}}), } + if GPUProfile(opts.Profile) { + results = append([]execx.Result{ + r.Check(ctx, "base GPU layer", execx.Command{Name: "kubectl", Args: []string{"get", "pods", "-n", "gpu-operator", "-o", "wide"}}), + r.Check(ctx, "NVIDIA RuntimeClass", execx.Command{Name: "kubectl", Args: []string{"get", "runtimeclass", "nvidia"}}), + }, results...) + } return execx.PrintResults(results) } func Install(ctx context.Context, opts Options) error { + if !opts.Yes && !opts.DryRun { + return errors.New("install requires --yes; use --dry-run to preview changes") + } if err := ValidateRepo(opts.RepoPath); err != nil { return err } @@ -106,9 +116,15 @@ func Install(ctx context.Context, opts Options) error { if err := r.Run(ctx, execx.Command{Name: "helm", Args: []string{"dependency", "build", "."}, Dir: opts.RepoPath, Mutates: true}); err != nil { return err } + if err := recoverPendingInstall(ctx, r, opts); err != nil { + return err + } if err := r.Run(ctx, execx.Command{Name: "helm", Args: helmInstallArgs(opts), Dir: opts.RepoPath, Mutates: true}); err != nil { return err } + if opts.DryRun { + return nil + } k := kubernetes.Client{Runner: r} for _, rollout := range []string{"deploy/ollama", "statefulset/open-webui", "deploy/opentelemetry-collector"} { if err := k.Rollout(ctx, rollout, opts.Namespace, opts.Timeout); err != nil { @@ -135,7 +151,9 @@ func Validate(ctx context.Context, opts Options) error { k.Service(ctx, opts.Namespace, "opentelemetry-collector"), optional(ctx, r, "Prometheus service", execx.Command{Name: "kubectl", Args: []string{"get", "svc", "-n", opts.Namespace, "kube-prometheus-stack-prometheus"}}), optional(ctx, r, "Grafana service", execx.Command{Name: "kubectl", Args: []string{"get", "svc", "-n", opts.Namespace, "llm-observability-stack-grafana"}}), - r.Check(ctx, "Ollama GPU limits", execx.Command{Name: "kubectl", Args: []string{"get", "deploy", "-n", opts.Namespace, "ollama", "-o", "jsonpath={.spec.template.spec.runtimeClassName}{\"\\n\"}{.spec.template.spec.containers[0].resources.limits.nvidia\\.com/gpu}{\"\\n\"}"}}), + } + if GPUProfile(opts.Profile) { + results = append(results, r.Check(ctx, "Ollama GPU limits", execx.Command{Name: "kubectl", Args: []string{"get", "deploy", "-n", opts.Namespace, "ollama", "-o", "jsonpath={.spec.template.spec.runtimeClassName}{\"\\n\"}{.spec.template.spec.containers[0].resources.limits.nvidia\\.com/gpu}{\"\\n\"}"}})) } if err := execx.PrintResults(results); err != nil { return err @@ -212,6 +230,8 @@ func ProfileValuesFile(profile string) string { return "values.enterprise-pilot-k3s.yaml" case "validation-k3s": return "values.validation-k3s.yaml" + case "cpu-k3s": + return "values.cpu-k3s.yaml" case "full-stack-nvidia": return "values.full-stack-nvidia.example.yaml" default: @@ -226,7 +246,7 @@ func GPUProfile(profile string) bool { } func ValidateInstallOptions(opts Options) error { - if opts.SkipInfraCheck && GPUProfile(opts.Profile) { + if opts.SkipInfraCheck && GPUProfile(opts.Profile) && !opts.DryRun { return errors.New("cannot skip infra validation for GPU observability profiles; run edge install infra or edge validate infra first") } return nil @@ -241,6 +261,7 @@ func infraDependencyOptions(opts Options) infra.Options { infraOpts.Verbose = opts.Verbose infraOpts.DryRun = opts.DryRun infraOpts.SkipCUDAValidation = true + infraOpts.NVIDIAEnabled = GPUProfile(opts.Profile) return infraOpts } @@ -269,6 +290,38 @@ func helmInstallArgs(opts Options) []string { return args } +type releaseStatus struct { + Info struct { + Status string `json:"status"` + } `json:"info"` + Version int `json:"version"` +} + +func recoverPendingInstall(ctx context.Context, r execx.Runner, opts Options) error { + out, err := r.Output(ctx, execx.Command{Name: "helm", Args: []string{"status", opts.Release, "-n", opts.Namespace, "-o", "json"}}) + if err != nil { + if strings.Contains(strings.ToLower(out), "release: not found") || strings.Contains(strings.ToLower(out), "release not found") { + return nil + } + return fmt.Errorf("inspect existing Helm release: %s: %w", out, err) + } + var status releaseStatus + if err := json.Unmarshal([]byte(out), &status); err != nil { + return fmt.Errorf("decode Helm release status: %w", err) + } + if status.Info.Status != "pending-install" { + return nil + } + if status.Version < 1 { + return errors.New("Helm reports pending-install without a valid revision") + } + fmt.Printf("Recovering interrupted Helm install at revision %d\n", status.Version) + return r.Run(ctx, execx.Command{Name: "helm", Args: []string{ + "rollback", opts.Release, strconv.Itoa(status.Version), "-n", opts.Namespace, + "--wait", "--timeout", opts.Timeout, + }, Mutates: true}) +} + func contains(value, needle string) bool { for i := 0; i+len(needle) <= len(value); i++ { if value[i:i+len(needle)] == needle { diff --git a/internal/modules/observability/observability_test.go b/internal/modules/observability/observability_test.go index 57ccf7d..b1f47c1 100644 --- a/internal/modules/observability/observability_test.go +++ b/internal/modules/observability/observability_test.go @@ -1,12 +1,16 @@ package observability -import "testing" +import ( + "encoding/json" + "testing" +) func TestProfileValuesFile(t *testing.T) { cases := map[string]string{ "": "values.geforce-940m-k3s.yaml", "geforce-940m-k3s": "values.geforce-940m-k3s.yaml", "local-k3s": "values.local-k3s.yaml", + "cpu-k3s": "values.cpu-k3s.yaml", "custom-values.yml": "custom-values.yml", } for input, want := range cases { @@ -33,6 +37,15 @@ func TestCPUInstallCanSkipInfraCheck(t *testing.T) { } } +func TestGPUDryRunCanSkipInfraCheck(t *testing.T) { + opts := DefaultOptions("/tmp/obs", "/tmp/infra", "llm-observability") + opts.SkipInfraCheck = true + opts.DryRun = true + if err := ValidateInstallOptions(opts); err != nil { + t.Fatalf("GPU dry run should not require a live infrastructure layer: %v", err) + } +} + func TestGPUHelmInstallForcesBaseLayerChartsDisabled(t *testing.T) { opts := DefaultOptions("/tmp/obs", "/tmp/infra", "llm-observability") opts.SetValues = []string{"nvidia-device-plugin.enabled=true"} @@ -59,6 +72,25 @@ func TestObservabilityDependencyCheckDoesNotConsumeGPU(t *testing.T) { } } +func TestCPUObservabilityDependencyUsesBasicK3sValidation(t *testing.T) { + opts := DefaultOptions("/tmp/obs", "/tmp/infra", "llm-observability") + opts.Profile = "cpu-k3s" + infraOpts := infraDependencyOptions(opts) + if infraOpts.NVIDIAEnabled { + t.Fatal("CPU profile should not require NVIDIA infrastructure validation") + } +} + +func TestReleaseStatusDecodesPendingInstallRevision(t *testing.T) { + var status releaseStatus + if err := json.Unmarshal([]byte(`{"info":{"status":"pending-install"},"version":3}`), &status); err != nil { + t.Fatal(err) + } + if status.Info.Status != "pending-install" || status.Version != 3 { + t.Fatalf("unexpected release status: %+v", status) + } +} + func lastSetValue(args []string, key string) string { last := "" for i := 0; i < len(args)-1; i++ { diff --git a/internal/nvidia/nvidia.go b/internal/nvidia/nvidia.go index 98db377..39c7e28 100644 --- a/internal/nvidia/nvidia.go +++ b/internal/nvidia/nvidia.go @@ -2,10 +2,36 @@ package nvidia import ( "context" + "strconv" + "strings" "github.com/Edge-Computing-LLM/edge-cli/internal/execx" ) +const GPUResource = "nvidia.com/gpu" + +func HostAvailable(ctx context.Context, r execx.Runner) bool { + _, err := r.Output(ctx, execx.Command{Name: "nvidia-smi", Args: []string{"--query-gpu=name", "--format=csv,noheader"}}) + return err == nil +} + +func ClusterAvailable(ctx context.Context, r execx.Runner) bool { + out, err := r.Output(ctx, execx.Command{Name: "kubectl", Args: []string{ + "get", "nodes", "-o", "jsonpath={range .items[*]}{.status.allocatable.nvidia\\.com/gpu}{\"\\n\"}{end}", + }}) + return err == nil && HasAllocatableGPU(out) +} + +func HasAllocatableGPU(output string) bool { + for _, field := range strings.Fields(output) { + count, err := strconv.ParseInt(field, 10, 64) + if err == nil && count > 0 { + return true + } + } + return false +} + func Checks(ctx context.Context, r execx.Runner) []execx.Result { return []execx.Result{ r.Check(ctx, "nvidia-smi", execx.Command{Name: "nvidia-smi"}), diff --git a/internal/nvidia/nvidia_test.go b/internal/nvidia/nvidia_test.go new file mode 100644 index 0000000..4e0368b --- /dev/null +++ b/internal/nvidia/nvidia_test.go @@ -0,0 +1,24 @@ +package nvidia + +import "testing" + +func TestHasAllocatableGPU(t *testing.T) { + tests := []struct { + name string + output string + want bool + }{ + {name: "gpu", output: "1", want: true}, + {name: "multiple nodes", output: "\n0\n2\n", want: true}, + {name: "zero", output: "0", want: false}, + {name: "missing", output: "", want: false}, + {name: "empty", output: "", want: false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := HasAllocatableGPU(tt.output); got != tt.want { + t.Fatalf("HasAllocatableGPU(%q) = %t, want %t", tt.output, got, tt.want) + } + }) + } +} From 8f062acb3f8ebc98a1bb0387ee5472faade371dc Mon Sep 17 00:00:00 2001 From: waqasm86 Date: Sat, 18 Jul 2026 00:21:32 +0500 Subject: [PATCH 3/6] Validate Qwen for the GeForce profile --- README.md | 4 ++++ internal/cli/root.go | 2 ++ internal/modules/observability/observability.go | 12 ++++++++++-- internal/modules/observability/observability_test.go | 9 +++++++++ 4 files changed, 25 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 4489727..72372a0 100644 --- a/README.md +++ b/README.md @@ -94,6 +94,10 @@ edge status working GPU. Without NVIDIA hardware it skips the toolkit and GPU Operator, installs or validates basic k3s, and selects `values.cpu-k3s.yaml`. +On the validated GeForce 940M profile, the application smoke test targets +`qwen-1-8b-chat-q4-k-m-local`; CPU and general profiles retain their configured +Gemma model defaults. + Explicit modes are useful in automation: ```bash diff --git a/internal/cli/root.go b/internal/cli/root.go index 79b21b7..9b1f49e 100644 --- a/internal/cli/root.go +++ b/internal/cli/root.go @@ -185,6 +185,7 @@ func installObsCmd(g *globals) *cobra.Command { } opts.Profile = profileForAccelerator(mode) } + opts.Model = observability.ModelForProfile(opts.Profile) opts.SkipInfraCheck = skipInfra return observability.Install(ctx, opts) }, @@ -236,6 +237,7 @@ func installAllCmd(g *globals) *cobra.Command { oopts := obsOpts(cfg, g, obsRepoPath, nil, nil) oopts.Yes = yes oopts.Profile = profileForAccelerator(mode) + oopts.Model = observability.ModelForProfile(oopts.Profile) oopts.SkipInfraCheck = g.DryRun if err := observability.Install(ctx, oopts); err != nil { return err diff --git a/internal/modules/observability/observability.go b/internal/modules/observability/observability.go index 088afbb..8c6d41f 100644 --- a/internal/modules/observability/observability.go +++ b/internal/modules/observability/observability.go @@ -40,18 +40,26 @@ var forcedBaseLayerDisables = []string{ } func DefaultOptions(repoPath, infraRepoPath, namespace string) Options { + profile := "geforce-940m-k3s" return Options{ RepoPath: repoPath, InfraRepoPath: infraRepoPath, Namespace: namespace, Release: "llm-observability-stack", - Profile: "geforce-940m-k3s", + Profile: profile, Timeout: "5m", - Model: "gemma3-1b-it-gguf-local", + Model: ModelForProfile(profile), OllamaSmoke: true, } } +func ModelForProfile(profile string) string { + if ProfileValuesFile(profile) == "values.geforce-940m-k3s.yaml" { + return "qwen-1-8b-chat-q4-k-m-local" + } + return "gemma3-1b-it-gguf-local" +} + func Doctor(ctx context.Context, opts Options) error { if err := ValidateRepo(opts.RepoPath); err != nil { return err diff --git a/internal/modules/observability/observability_test.go b/internal/modules/observability/observability_test.go index b1f47c1..2e6de60 100644 --- a/internal/modules/observability/observability_test.go +++ b/internal/modules/observability/observability_test.go @@ -20,6 +20,15 @@ func TestProfileValuesFile(t *testing.T) { } } +func TestModelForProfile(t *testing.T) { + if got := ModelForProfile("geforce-940m-k3s"); got != "qwen-1-8b-chat-q4-k-m-local" { + t.Fatalf("GeForce model = %q", got) + } + if got := ModelForProfile("cpu-k3s"); got != "gemma3-1b-it-gguf-local" { + t.Fatalf("CPU model = %q", got) + } +} + func TestGPUInstallCannotSkipInfraCheck(t *testing.T) { opts := DefaultOptions("/tmp/obs", "/tmp/infra", "llm-observability") opts.SkipInfraCheck = true From a20394e6525df77ab1da7625855030899e0bce08 Mon Sep 17 00:00:00 2001 From: waqasm86 Date: Sat, 18 Jul 2026 00:44:30 +0500 Subject: [PATCH 4/6] Document the Qwen evidence companion --- README.md | 7 +++++++ docs/ARCHITECTURE.md | 13 +++++++++++-- docs/LOCAL-DEPENDENCY-INVENTORY.md | 1 + 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 72372a0..b0ae921 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,12 @@ It coordinates two current layers: - `llm-observability-stack`: LLMOps workloads, Helm charts, Open WebUI, Ollama, OpenTelemetry, Prometheus, Grafana, and related services. +The organization also publishes +[`qwen-gguf-observability`](https://github.com/Edge-Computing-LLM/qwen-gguf-observability), +a read-only evidence companion for the deployed Qwen runtime. It consumes the +status produced by these layers; `edge-cli` does not install it as another +cluster layer. + The CLI supports automatic NVIDIA/CPU selection. It does not currently install AMD, Intel, or Apple Silicon accelerator runtimes. @@ -216,3 +222,4 @@ commands such as `kubectl`, `helm`, `apt-get`, `systemctl`, `k3s`, and - [Troubleshooting](docs/TROUBLESHOOTING.md) - [Contributing](CONTRIBUTING.md) - [Security](SECURITY.md) +- [Qwen GGUF runtime evidence companion](https://github.com/Edge-Computing-LLM/qwen-gguf-observability) diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 4e7f3e7..45ed650 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -13,8 +13,12 @@ The platform is split into ordered layers: Toolkit, GPU Operator, RuntimeClass, and CUDA validation. - Layer 2, `llm-observability-stack`: LLMOps layer for Helm workloads such as Ollama, Open WebUI, OpenTelemetry Collector, Prometheus, Grafana, and related tools. -- Future Layer 3 repositories, such as a data/storage stack, should declare - their dependency on the previous layer before they can be installed. +- Evidence companion, `qwen-gguf-observability`: read-only Qwen runtime contract + checks and sanitized point-in-time evidence. It owns no cluster resources and + is intentionally outside the ordered install/uninstall layer graph. +- Future Layer 3 repositories that deploy resources, such as a data/storage + stack, should declare their dependency on the previous layer before they can + be installed. ## Module Model @@ -58,6 +62,11 @@ Full infra validation owns CUDA pod execution. Observability dependency checks verify the ready base layer without launching a CUDA pod, so validation remains usable after Ollama has reserved the only GPU on a low-VRAM laptop. +After Layer 2 is healthy, `qwen-gguf-observability` may independently read the +Kubernetes, Helm, Ollama, and `nvidia-smi` status surfaces. This avoids copying +deployment logic into an evidence repository or coupling evidence capture to an +`edge install` operation. + ## Execution Model The CLI is 100% Go. It does not create Bash scripts. Host and cluster operations diff --git a/docs/LOCAL-DEPENDENCY-INVENTORY.md b/docs/LOCAL-DEPENDENCY-INVENTORY.md index c708506..5325777 100644 --- a/docs/LOCAL-DEPENDENCY-INVENTORY.md +++ b/docs/LOCAL-DEPENDENCY-INVENTORY.md @@ -13,6 +13,7 @@ lockfiles, generated package metadata, and documentation. | Control | `Edge-Computing-LLM/edge-cli` | Detect the accelerator, orchestrate installation, and expose status and validation commands. | | Infrastructure | `Edge-Computing-LLM/k3s-nvidia-edge` | Install or validate k3s and, conditionally, NVIDIA Container Toolkit, GPU Operator, RuntimeClass, device plugin, and DCGM. | | Application | `Edge-Computing-LLM/llm-observability-stack` | Deploy CPU- or GPU-profiled Ollama, Open WebUI, OpenTelemetry, Prometheus, and Grafana workloads. | +| Evidence | `Edge-Computing-LLM/qwen-gguf-observability` | Validate the live Qwen runtime contract and capture sanitized evidence without owning cluster resources. | | Dashboard | `Edge-Computing-LLM/Frontend-Edge-LLM-Observability` | Present LLM, Kubernetes, and accelerator telemetry. | The NVIDIA infrastructure layer is conditional. CPU hosts skip its GPU-specific From 110605a70641faa542c7d2c2c953bb0f9d57a6f3 Mon Sep 17 00:00:00 2001 From: waqasm86 Date: Sat, 18 Jul 2026 01:10:51 +0500 Subject: [PATCH 5/6] Define organization language boundaries --- README.md | 6 ++++++ docs/LANGUAGE-BOUNDARIES.md | 28 ++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+) create mode 100644 docs/LANGUAGE-BOUNDARIES.md diff --git a/README.md b/README.md index b0ae921..7c60965 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,11 @@ a read-only evidence companion for the deployed Qwen runtime. It consumes the status produced by these layers; `edge-cli` does not install it as another cluster layer. +[`Frontend-Edge-LLM-Observability`](https://github.com/Edge-Computing-LLM/Frontend-Edge-LLM-Observability) +is the TypeScript/Vue presentation companion. It is also outside the ordered +cluster lifecycle and consumes controlled metrics endpoints rather than direct +host or Kubernetes credentials. + The CLI supports automatic NVIDIA/CPU selection. It does not currently install AMD, Intel, or Apple Silicon accelerator runtimes. @@ -213,6 +218,7 @@ commands such as `kubectl`, `helm`, `apt-get`, `systemctl`, `k3s`, and ## Documentation - [Architecture](docs/ARCHITECTURE.md) +- [Programming language and script boundaries](docs/LANGUAGE-BOUNDARIES.md) - [Accelerator selection](docs/ACCELERATOR-MODES.md) - [Local dependency and repository inventory](docs/LOCAL-DEPENDENCY-INVENTORY.md) - [Live validation - 2026-07-17](docs/LIVE-VALIDATION-2026-07-17.md) diff --git a/docs/LANGUAGE-BOUNDARIES.md b/docs/LANGUAGE-BOUNDARIES.md new file mode 100644 index 0000000..8ab6c42 --- /dev/null +++ b/docs/LANGUAGE-BOUNDARIES.md @@ -0,0 +1,28 @@ +# Programming language and script boundaries + +The organization uses languages by operational responsibility, not by forcing +every repository into one implementation language. + +| Language or format | Use it for | Do not use it for | +|---|---|---| +| Go | Durable CLIs, install/uninstall workflows, validation gates, typed configuration, command execution, and cross-repository orchestration | Browser UI or one-off data analysis | +| Python 3.11 | Structured evidence, benchmarks, API services, notebooks, Kubernetes reporting, and test tooling | Host package installation or a second deployment control plane | +| Bash | Short transparent wrappers around Helm, kubectl, container image import, and workshop command sequences | Growing business logic, complex parsing, persistent state, or duplicated Go workflows | +| TypeScript/Vue | Browser dashboards, typed frontend models, chart interaction, and client-side Prometheus parsing | Direct privileged Kubernetes access or host orchestration | +| Helm/YAML | Declarative Kubernetes resources and deployment profiles | Imperative host installation logic | + +## Repository mapping + +- `edge-cli`: Go only for product behavior; Make and YAML remain build/CI glue. +- `k3s-nvidia-edge`: Go for infrastructure workflows and Helm/YAML for the GPU + Operator profile. +- `llm-observability-stack`: Helm/YAML for resources, Go for its legacy/helper + CLI, Python 3.11 for application/benchmark/diagnostic code, and Bash only for + thin local operator wrappers. +- `qwen-gguf-observability`: dependency-free Python 3.11 for structured, + read-only evidence collection. +- `Frontend-Edge-LLM-Observability`: Vue 3 and TypeScript for the browser. + +New deployable repositories require a clear ownership boundary. A new +repository should not be created merely to move an existing script to another +language. From da8f710ee3f871c49743ee82dbf85f99eda85ec2 Mon Sep 17 00:00:00 2001 From: waqasm86 Date: Sat, 18 Jul 2026 01:11:58 +0500 Subject: [PATCH 6/6] Add Go validation workflow --- .github/workflows/validate.yaml | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 .github/workflows/validate.yaml diff --git a/.github/workflows/validate.yaml b/.github/workflows/validate.yaml new file mode 100644 index 0000000..97be8d1 --- /dev/null +++ b/.github/workflows/validate.yaml @@ -0,0 +1,24 @@ +name: validate + +on: + push: + pull_request: + +permissions: + contents: read + +jobs: + go: + runs-on: ubuntu-24.04 + steps: + - uses: actions/checkout@v7 + - uses: actions/setup-go@v7 + with: + go-version-file: go.mod + cache: true + - run: go mod verify + - name: Check formatting + run: test -z "$(gofmt -l .)" + - run: go test ./... + - run: go vet ./... + - run: go build ./...