Added ops console

[goransle]

No description provided.

[Copilot]

Pull request overview

Adds a development-only operations console to the router, including a same-origin browser UI at /_ops/, a hardened backend (sessions, CSRF, origin/fetch-metadata checks, strict/bounded JSON), and a two-listener model (public HTTP + optional mTLS HTTPS listener). This also extends internal services (downloader/builder) with authenticated /v1/ops/* endpoints and introduces broad unit/integration/browser test coverage plus CI wiring.

Changes:

• Introduces ops-console runtime components: config parsing + token verifier, sessions/rate limiting, strict HTTP helpers, internal ops routers, cache manager, and telemetry snapshotting.
• Integrates ops console into router/server (separate public + ops servers), adds correlation IDs + request telemetry, and blocks public exposure of /ops static paths.
• Adds end-to-end validation: Hurl integration suites, Docker-based integration runner, and Playwright + axe accessibility tests; updates CI workflow and docs accordingly.

Reviewed changes

Copilot reviewed 64 out of 68 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/test.js	Includes new ops-console-related test suites in the unit test runner.
test/server.js	Adds server tests for two-listener behavior and mTLS gating for /_ops/*.
test/router.js	Verifies correlation IDs + telemetry snapshot behavior and blocks ops assets on public static paths.
test/ops/stack.js	Adds a docker-compose harness app for ops-console integration scenarios.
test/ops/fixture-service.js	Adds internal fixture services to simulate snapshot/cache behaviors and failures.
test/ops-telemetry.js	Unit tests for ops telemetry limits, sanitization, and merging semantics.
test/ops-internal.js	Unit tests for authenticated internal ops router behavior and bounded JSON transport.
test/ops-config.js	Unit tests for ops console configuration validation, verifier derivation, and loopback/mTLS rules.
test/ops-cache.js	Unit tests for ops cache manager correctness, safety checks, and bounded snapshots.
test/JobQueue.test.js	Migrates JobQueue tests to mocha and adds stronger concurrency/failure coverage.
test/hurl/ops-snapshot.hurl	Integration checks for snapshot freshness/timeout/oversize behavior.
test/hurl/ops-security.hurl	Integration checks ensuring /_ops/* is fully disabled when console is off.
test/hurl/ops-cache.hurl	Integration checks for cache mutation flows, audit safety, and bounds.
test/hurl/ops-auth.hurl	Integration checks for auth/cookies/CSRF/origin/fetch-metadata/rate limits.
test/fixtures/mtls/untrusted-client.key	Adds mTLS test fixture key for untrusted client scenarios.
test/fixtures/mtls/untrusted-client.crt	Adds mTLS test fixture cert for untrusted client scenarios.
test/fixtures/mtls/untrusted-ca.crt	Adds mTLS test fixture CA for negative trust-chain testing.
test/fixtures/mtls/server.key	Adds mTLS test server key fixture.
test/fixtures/mtls/server.crt	Adds mTLS test server certificate fixture.
test/fixtures/mtls/client.key	Adds trusted mTLS client key fixture.
test/fixtures/mtls/client.crt	Adds trusted mTLS client certificate fixture.
test/fixtures/mtls/ca.crt	Adds trusted CA fixture used to validate mTLS client certs.
test/downloader-service.js	Expands downloader tests for auth, bounded ops endpoints, concurrency sharing, and sanitized failures.
test/download.js	Updates download tests to treat HTTP bodies as raw bytes (Buffer) and adds binary-path coverage.
test/browser/ops.spec.js	Adds Playwright + axe end-to-end accessibility and UX tests for the console UI.
static/ops/login.js	Implements browser login flow for the ops console.
static/ops/login.html	Adds ops console login page markup.
static/ops/index.html	Adds ops console UI shell (overview/activity/cache/session panels).
static/ops/console.css	Adds styling for the ops console UI, including reduced-motion handling.
src/utils.ts	Switches webpack invocation to execFile (avoids shell string execution).
src/tsconfig.json	Extends TS libs to include newer Promise APIs.
src/JobQueue.ts	Enhances JobQueue to return job results, support failure propagation, and expose queue metrics.
scripts/test-ops-integration.js	Adds docker-compose-based ops-console integration runner (incl. log scrubbing checks).
README.md	Documents local ops-console enablement, env vars, token verifier derivation, and test commands.
playwright.ops.config.js	Adds dedicated Playwright config for ops-console browser tests.
package.json	Adds Playwright/Axe/Hurl deps and new ops integration/browser test scripts.
downloader-server.js	Adds internal ops router + correlation/observability plumbing and safer public error mapping.
docs/operations-console.md	Expands the formal spec: two-listener model, mTLS contract, and deployment guidance.
Dockerfile.router	Exposes ops mTLS port (8443) in router container image metadata.
compose.yaml	Adds ops-console env wiring and bind-address support for safer local exposure defaults.
compose.ops-test.yaml	Adds dedicated integration test compose stack for ops-console scenarios.
builder-server.js	Adds internal ops router + correlation/observability plumbing and safer public error mapping.
app/utils.js.map	Updates compiled sourcemap to reflect safer execFile usage.
app/utils.js	Updates compiled JS to reflect safer execFile usage.
app/service-client.js	Adds correlation IDs, bounded request/response bodies, and optional timeout-through-body handling.
app/server.js	Adds ops console router to Express app and introduces public+ops server creation (HTTP + optional mTLS HTTPS).
app/router.js	Adds correlation ID generation, dependency/trace telemetry, disables public cleanup when console enabled, blocks /ops static paths.
app/ops/sessions.js	Implements ops sessions, CSRF tokens, and rate limiting primitives.
app/ops/internal-router.js	Adds authenticated internal /v1/ops/* endpoints with strict validation and bounds.
app/ops/http.js	Adds strict JSON parsing, security headers, request-context enforcement, and observed-route helpers.
app/ops/config.js	Adds ops-console config parsing, token verifier logic, and loopback vs mTLS mode rules.
app/ops/cache.js	Adds cache inspection/snapshotting and bounded/safe cache maintenance operations.
app/JobQueue.js.map	Updates compiled sourcemap for new JobQueue behavior.
app/JobQueue.js	Updates compiled JobQueue behavior (result propagation + metrics).
app/JobQueue.d.ts	Updates generated types to include queue metrics and new job result shape.
app/download.js	Treats upstream bodies as raw buffers and parses JSON explicitly as UTF-8 when needed.
.github/workflows/test.yml	Updates CI runner and adds ops integration + Playwright browser test steps.
.env.example	Documents new env variables and safe defaults for local binding + ops-console config.

Comments suppressed due to low confidence (1)

builder-server.js:68

• The builder cleanup interval logs raw errors via console.error, which will include stack traces. Elsewhere in this PR (e.g. downloader cleanup) errors are logged with a sanitized message; keeping this as console.error risks leaking internal paths and can break the ops integration log-scrub checks that fail on stack traces.

  const app = createApp()
  const interval = Number(process.env.BUILDER_CLEAN_INTERVAL || config.builderCleanInterval || 2 * 60 * 1000)
  const timer = setInterval(() => app.locals.service.cleanup().catch(console.error), interval)
  timer.unref()
  return app.listen(Number(process.env.BUILDER_PORT || config.builderPort || 8082))