Restore fork infra (rebased onto current main) — supersedes #72#83
Merged
Conversation
The fork's terraform/ tree (auth0 + cloud_deps + cockpit credentials + scripts, 16 files) was deleted on main by commit 53d09cc, the local landing of upstream pixie-io#2391 'Create vizier and cloud cert-manager compatible secrets'. That upstream commit only touches cert handling; the terraform deletion was an unrelated regression bundled into the fork pickup. Content here is byte-identical to the pre-deletion tree at 610975b (parent of main tip). This belongs on main, not in PR 68.
Same regression as the terraform tree: commit 53d09cc (the fork landing of upstream pixie-io#2391) deleted these fork-only files from main, and they were re-added on PR 68's branch where they don't belong. Moved here for the main-restore PR. - cockpit/**: restored at the post-rename path (private/cockpit/ -> cockpit/) so it passes the upstream filename-linter on main; the workflow file is unchanged from main. - .sops.yaml, .github/workflows/e2e_log4shell_soc.yaml: byte-identical to PR 68's copies.
…name-linter Reverts the private/cockpit/ -> cockpit/ rename so this restore matches the original main tree exactly: all 10 cockpit files (incl. kustomization.yaml with its original ../../k8s/cloud relative paths) are now byte-identical to the pre-deletion state at 610975b. Upstream main carries filename_linter.yaml (added by pixie-io#2391), which fails any PR touching a 'private' path. Adding a single '!private/cockpit/**' negative glob keeps that guard for everything else while letting this known fork-cloud config through. This is the only deviation in the PR from a pure byte-for-byte restore.
The daily pixie-oss-copybara mirror reconciles everything under destination_files (glob ** minus ignored_dirs + fork_only_files) to upstream, deleting any in-scope path that upstream lacks. Six fork-only trees were never on fork_only_files, so the pixie-io#2391 import (53d09cc, GitOrigin-RevId 7622689) deleted all of them: terraform/**, private/** (cockpit + skaffold_cloud), .sops.yaml, e2e_log4shell_soc.yaml, src/e2e_test/adaptive_export_loadtest/**, and src/vizier/services/agent/pem/direct_query_* (+ DIRECT_QUERY_* docs). Adding them to fork_only_files removes them from destination_files scope so the mirror leaves them untouched. Without this, the PR 72 restore would be re-reverted on the next 15:00 UTC run.
…mirror The pixie-io#2391 copybara import (53d09cc) also removed two fork trees not in PR 72's first pass: - src/e2e_test/adaptive_export_loadtest/** (22 files) — self-contained load-test harness (shell + Go loadgen + k8s); restored from 610975b. Also restores the .arclint exclusion for its nested loadgen go module (copybara stripped it; golangci-lint fails without it). - PEM direct-query endpoint (dx#29): 5 fork-only files (direct_query_server.*, DIRECT_QUERY_*.md) restored, plus the fork direct_query hunks copybara stripped from 4 shared upstream files (pem_main.cc, pem_manager.{cc,h}, BUILD.bazel) and the pem visibility grant it stripped from src/carnot/udf/BUILD.bazel. Reverted to 610975b (copybara added zero upstream lines to any of them); direct_query_server_test.cc re-run through clang-format for the current fork config. Durability (copy.bara.sky): fork-only PEM files are covered by fork_only_files; shared upstream files (.arclint, 4 pem + carnot/udf BUILD) added to ignored_dirs so the mirror freezes their fork hunks.
Two CI failures on this PR (PR-description lint intentionally ignored): - filename-linter (check-files-changed): the guard fails ANY path containing 'private'. This PR's purpose is to restore fork infra under private/cockpit/* and tools/private/copybara/*, which are intentional on the fork (copybara- excluded). Scope the filter to exclude private/** and tools/private/** so it still catches accidental NEW 'private' paths elsewhere. - run-genfiles (gazelle): bazel run //:gazelle -- fix alphabetises macro attributes; the restored container_images/BUILD.bazel had bazel_sdk_versions after container_type in the 4 go_container_libraries() calls. Regenerated with the local px_dev bazel 7.7.1 toolchain (buildifier attr sort only; no semantic change). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017k7uYSNUctQvkTAZYJbaB3
…e paths The dorny/paths-filter '!' negations OR rather than subtract, so they didn't exclude the fork's intentional private/ + tools/private/ roots. Replace the filter with a deterministic git-diff: flag any changed path containing 'private' that is NOT under private/ or tools/private/. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017k7uYSNUctQvkTAZYJbaB3
make go-setup (bazel run //:gazelle -- fix) generates BUILD.bazel for the two restored loadgen command packages (cmd/cleanloadgen, cmd/httpsink); the PR restored their main.go + go.mod but not the generated BUILD files, leaving the tree dirty after gazelle. Generated with the local px_dev bazel 7.7.1 toolchain (pl_go_binary + go_library, map_kind directive) and verified idempotent. Apache header added to match repo convention (gazelle preserves it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017k7uYSNUctQvkTAZYJbaB3
Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
📝 WalkthroughWalkthroughThis PR adds adaptive-export load-test infrastructure and documentation, an authenticated PEM direct-query gRPC service, Cockpit Kubernetes and Terraform infrastructure, SOPS credential configuration, repository policy updates, and a calibration workflow for SOC validation. ChangesAdaptive export validation
PEM direct query
Cockpit cloud infrastructure
Repository integration
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 26
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/e2e_log4shell_soc.yaml:
- Around line 13-31: Add a workflow-level concurrency guard to the
e2e-log4shell-soc workflow so dispatches targeting the shared runner,
namespaces, and ClickHouse tables cannot run simultaneously. Use a stable group
keyed to this workflow and enable cancellation behavior only if consistent with
the existing workflow’s intended run handling.
- Around line 94-99: Update the detection gate in the dx-daemon log check so the
validation requires the expected Log4Shell chain identifier, such as “log4shell”
or “control-plane-credential-abuse.” Remove the generic “RULE IN” alternative
from the second grep while preserving the existing evidence capture and failure
behavior.
- Around line 46-65: Update the workflow run step to pass soc_ref and dx_image
through dedicated environment variables, then reference those variables inside
the shell instead of interpolating GitHub expressions. Quote the values when
using them with git clone and kubectl set image, and remove the || true from the
dx-daemon image update so failures stop the workflow and the requested image is
actually tested.
In `@private/cockpit/plugin_db_updater_job.yaml`:
- Around line 31-32: Update the updater Job’s container image reference from the
mutable cloud-plugin-load_db:latest tag to a specific immutable release tag or
SHA digest, preserving the existing image repository and container
configuration.
- Around line 68-70: Remove the unused pl-db-secrets volume definition from the
job specification, including its secret reference, while preserving the existing
secretKeyRef environment configuration and all other volumes.
In `@src/e2e_test/adaptive_export_loadtest/harness/exp_control.sh`:
- Around line 64-65: In the E2 experiment loop around the inj command, increase
the following sleep duration from 8 seconds to at least 10 seconds so wall-clock
time surpasses the watermark advanced by the injected event-time range before
the next iteration.
In `@src/e2e_test/adaptive_export_loadtest/harness/exp_row_reconcile.sh`:
- Around line 121-127: Update the reconciliation verdict logic in the harness so
the FAIL branch returns a nonzero status after printing the failure message,
while the PASS branch retains a zero status. Ensure the pipeline’s final status
also reflects this failure rather than the successful kubectl cleanup command.
In `@src/e2e_test/adaptive_export_loadtest/harness/inject.sh`:
- Around line 95-108: Replace predictable temporary paths with mktemp-managed
artifacts: in inject.sh create the response file atomically and register
trap-based cleanup; in exp_row_reconcile.sh create a unique output directory
with mktemp and clean it on exit; in lib.sh create a unique port-forward log
with mktemp and include it in the existing cleanup flow.
In `@src/e2e_test/adaptive_export_loadtest/harness/lib.sh`:
- Around line 89-135: The query helpers currently hide ClickHouse failures and
can turn error text into numeric counts. Update chq to make curl fail on HTTP
errors, then replace the digit-stripping pipelines in count_pod, attrib_count,
uniq_hashes, watermark_of, and attr_field with validation that accepts only a
complete unsigned-integer response, propagating or failing on any other output.
In `@src/e2e_test/adaptive_export_loadtest/harness/run.sh`:
- Around line 28-56: Make the validation stack fail closed: in
src/e2e_test/adaptive_export_loadtest/harness/run.sh lines 28-56, accumulate
failures from child experiments and stats.py, then exit non-zero when any fail;
in src/e2e_test/adaptive_export_loadtest/harness/nfr.sh lines 34-67, mark setup
failures, truncation failures, and count mismatches as phase failures and return
non-zero; in src/e2e_test/adaptive_export_loadtest/harness/stats.py lines 57-89,
produce a failed verdict for empty or invalid samples and propagate that result
through main().
- Around line 32-33: Update the EVID default in the harness script to remove the
developer-specific /home/croedig path and use a portable, rig-independent
evidence location while preserving the existing timestamped directory naming and
EVID override behavior.
In `@src/e2e_test/adaptive_export_loadtest/harness/stats.py`:
- Around line 47-55: Update the PASS count derived from passcol to parse the
status token before the “|” delimiter and count only an exact PASS token,
rejecting statuses such as PASS|FAIL_dns. Update the metrics selection to use
the E5 experiment’s exact-metric contract, excluding conn_act alongside wm_act
while retaining only explicitly defined exact-count columns; make the selection
experiment-aware rather than relying solely on the _act suffix.
In
`@src/e2e_test/adaptive_export_loadtest/tools/loadgen/cmd/cleanloadgen/main.go`:
- Around line 159-177: Add a context with a deadline around the PostgreSQL query
loop and replace db.QueryRow in the pgN block with QueryRowContext, ensuring the
context is canceled after use. Preserve the existing query validation and fatal
error reporting while ensuring stalled SELECT 1 calls terminate at the
configured timeout.
In `@src/vizier/services/agent/pem/DIRECT_QUERY_CONTRACT.md`:
- Around line 3-5: Remove obsolete stub and unfinished-TDD wording across the
direct-query surface: in src/vizier/services/agent/pem/DIRECT_QUERY_CONTRACT.md
lines 3-5, state that the endpoint is implemented and default-disabled; in
src/vizier/services/agent/pem/direct_query_server.h lines 21-23, describe the
production implementation instead of future work; and in
src/vizier/services/agent/pem/direct_query_server_test.cc lines 19-24, remove
the claim that positive cases remain red.
- Around line 49-54: Align the documented JWT contract with verifier
requirements: in src/vizier/services/agent/pem/DIRECT_QUERY_CONTRACT.md lines
49-54, require iss="PL" and Scopes="service"; in
src/vizier/services/agent/pem/DIRECT_QUERY_SECURITY.md lines 95-115, document
rejection tests for incorrect or missing issuer and scope; and in lines 174-179,
instruct clients to mint tokens with iss="PL" and Scopes="service".
In `@src/vizier/services/agent/pem/DIRECT_QUERY_SECURITY.md`:
- Around line 275-286: Update the failure-modes table entry for “server not
wired with a live Carnot” to document connection refusal/unavailability when
MaybeStartDirectQueryServer returns before service registration, including
Carnot::Create failure. Remove the incorrect FAILED_PRECONDITION client status
and adjust the operator guidance to reference initialization failure and PEM
stderr breadcrumbs.
In `@src/vizier/services/agent/pem/direct_query_server_test.cc`:
- Around line 220-225: Exclude the default-only positive tests, including
ValidToken_Mutation_Unimplemented and the other referenced test blocks, from
builds defining PX_PEM_DIRECT_QUERY_DISABLED by wrapping them with `#ifndef`
PX_PEM_DIRECT_QUERY_DISABLED. Keep toggle-specific disabled-mode tests compiled
so they continue validating the hard-disable behavior.
In `@src/vizier/services/agent/pem/direct_query_server.cc`:
- Around line 342-347: Update the schema-building loop in the direct query
response to copy each column’s semantic type from
sink.output_table().column_semantic_types() into the corresponding rel column’s
column_semantic_type field, while preserving the existing column name and data
type mappings.
- Around line 408-412: Update the execution-error handling in the direct query
response flow to set status.code from chunk.execution_error().err_code() using
the corresponding Vizier response status mapping, in addition to the existing
message. Keep has_payload enabled so clients receive the mapped non-OK status
for payload-less error frames.
- Line 348: Update ExecuteQuery and its result-writing paths to stream
raw_query_results through a per-RPC bounded sink with backpressure, rather than
buffering until query completion. Propagate writer failures and client
cancellation, stop execution when the sink is cancelled, and avoid holding
exec_mu_ for the full streaming lifetime so subsequent queries are not blocked;
apply the same behavior at the schema and result write sites.
In `@src/vizier/services/agent/pem/pem_manager.cc`:
- Around line 188-205: Replace the Carnot::Create flow in the PEM direct-query
initialization with reuse of the PEM’s existing carnot() and its engine state;
remove the dedicated direct_query_carnot_ construction and related failure
handling. Update DirectQueryServer setup and result routing to use the existing
engine while preserving direct-query functionality and the existing PEM
lifecycle.
- Line 117: Decouple direct-query server startup from broker registration: move
MaybeStartDirectQueryServer() out of PostRegisterHookImpl() into a lifecycle
stage that runs regardless of registration success, or enable the documented
broker-optional mode. Preserve broker registration behavior while ensuring the
node-local endpoint starts when broker/NATS registration fails.
In `@terraform/kubernetes/auth0/auth0_main.tf`:
- Around line 11-13: Update the auth0 provider configuration to disable HTTP
debugging by removing the debug setting or setting debug to false, ensuring
request and response data is not logged by default.
In `@terraform/kubernetes/cloud_deps/env_cloud_deps.tf`:
- Around line 1-4: Replace the hardcoded path in the kustomization_build
"env_deps" resource with a Terraform variable reference, and define that
variable using the repository’s existing variable conventions so the Azure
pipeline can override it per environment. Remove or update the obsolete TODO
while preserving the current path as the default value.
In `@terraform/kubernetes/cloud_deps/provider.tf`:
- Around line 14-22: Update the Kubernetes and Kustomization provider blocks to
use configurable variables for the kubeconfig path and context instead of
hardcoded "~/.kube/cockpit-config" and "default" values. Define or reuse
corresponding variables in the Terraform configuration and reference them
through config_path, kubeconfig_path, config_context, and context.
In `@terraform/kubernetes/scripts/create_random_bytes.sh`:
- Line 34: Update the random-byte generation pipeline assigned to bytes so it
runs with LC_ALL=C, ensuring tr processes /dev/urandom byte-by-byte and avoids
locale-dependent illegal byte sequence errors.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 9ab3b75c-ed05-4da0-a112-c3311b141d29
⛔ Files ignored due to path filters (1)
terraform/kubernetes/cloud_deps/terraform.tfstateis excluded by!**/*.tfstate
📒 Files selected for processing (65)
.arclint.github/workflows/e2e_log4shell_soc.yaml.github/workflows/filename_linter.yaml.sops.yamlprivate/cockpit/artifact_config.yamlprivate/cockpit/auth_deployment.yamlprivate/cockpit/cloud_ingress.yamlprivate/cockpit/domain_config.yamlprivate/cockpit/kustomization.yamlprivate/cockpit/oauth_config.yamlprivate/cockpit/plugin_db_updater_job.yamlprivate/cockpit/proxy_envoy.yamlprivate/cockpit/script_bundles_config.yamlprivate/cockpit/servers_transport.yamlsrc/carnot/udf/BUILD.bazelsrc/e2e_test/adaptive_export_loadtest/CONTRACTS.mdsrc/e2e_test/adaptive_export_loadtest/FINDINGS_AND_BACKLOG.mdsrc/e2e_test/adaptive_export_loadtest/README.mdsrc/e2e_test/adaptive_export_loadtest/fixtures/EXPERIMENTS.mdsrc/e2e_test/adaptive_export_loadtest/harness/ae_config.shsrc/e2e_test/adaptive_export_loadtest/harness/exp_control.shsrc/e2e_test/adaptive_export_loadtest/harness/exp_e5.shsrc/e2e_test/adaptive_export_loadtest/harness/exp_e8.shsrc/e2e_test/adaptive_export_loadtest/harness/exp_matrix.shsrc/e2e_test/adaptive_export_loadtest/harness/exp_row_reconcile.shsrc/e2e_test/adaptive_export_loadtest/harness/inject.shsrc/e2e_test/adaptive_export_loadtest/harness/lib.shsrc/e2e_test/adaptive_export_loadtest/harness/log4shell_fire.shsrc/e2e_test/adaptive_export_loadtest/harness/nfr.shsrc/e2e_test/adaptive_export_loadtest/harness/run.shsrc/e2e_test/adaptive_export_loadtest/harness/stats.pysrc/e2e_test/adaptive_export_loadtest/k8s/00-sinks.yamlsrc/e2e_test/adaptive_export_loadtest/k8s/gen-pod.tmpl.yamlsrc/e2e_test/adaptive_export_loadtest/tools/loadgen/Dockerfilesrc/e2e_test/adaptive_export_loadtest/tools/loadgen/cmd/cleanloadgen/BUILD.bazelsrc/e2e_test/adaptive_export_loadtest/tools/loadgen/cmd/cleanloadgen/main.gosrc/e2e_test/adaptive_export_loadtest/tools/loadgen/cmd/httpsink/BUILD.bazelsrc/e2e_test/adaptive_export_loadtest/tools/loadgen/cmd/httpsink/main.gosrc/e2e_test/adaptive_export_loadtest/tools/loadgen/go.modsrc/stirling/source_connectors/socket_tracer/testing/container_images/BUILD.bazelsrc/vizier/services/agent/pem/BUILD.bazelsrc/vizier/services/agent/pem/DIRECT_QUERY_CONTRACT.mdsrc/vizier/services/agent/pem/DIRECT_QUERY_SECURITY.mdsrc/vizier/services/agent/pem/direct_query_server.ccsrc/vizier/services/agent/pem/direct_query_server.hsrc/vizier/services/agent/pem/direct_query_server_test.ccsrc/vizier/services/agent/pem/pem_main.ccsrc/vizier/services/agent/pem/pem_manager.ccsrc/vizier/services/agent/pem/pem_manager.hterraform/credentials/cockpit/auth0_config.yamlterraform/kubernetes/auth0/.terraform.lock.hclterraform/kubernetes/auth0/auth0_import.tfterraform/kubernetes/auth0/auth0_main.tfterraform/kubernetes/auth0/generated.tfterraform/kubernetes/auth0/terraform.tfvarsterraform/kubernetes/auth0/variables.tfterraform/kubernetes/cloud_deps/.terraform.lock.hclterraform/kubernetes/cloud_deps/base_cloud_deps.tfterraform/kubernetes/cloud_deps/cert_manager.tfterraform/kubernetes/cloud_deps/core_resource_deps.tfterraform/kubernetes/cloud_deps/env_cloud_deps.tfterraform/kubernetes/cloud_deps/provider.tfterraform/kubernetes/cloud_deps/variables.tfterraform/kubernetes/scripts/create_random_bytes.shtools/private/copybara/copy.bara.sky
Comment on lines
+11
to
+13
| provider "auth0" { | ||
| debug = true | ||
| } |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the target file and nearby context.
git ls-files terraform/kubernetes/auth0/auth0_main.tf
wc -l terraform/kubernetes/auth0/auth0_main.tf
cat -n terraform/kubernetes/auth0/auth0_main.tf
# Look for other Auth0 provider usage and any notes about debug/logging.
rg -n --hidden -S 'provider "auth0"|debug\s*=' terraformRepository: k8sstormcenter/pixie
Length of output: 1963
🌐 Web query:
Auth0 Terraform provider debug true logs HTTP requests responses credentials tenant user data TF_LOG DEBUG docs
💡 Result:
To enable debug logging for the Auth0 Terraform provider, you must configure both the provider setting and the Terraform environment level. Enabling Debug Logs 1. Configure the provider: Set the debug argument to true within your auth0 provider block [1][2]. Alternatively, you can use the AUTH0_DEBUG environment variable [1][2]. 2. Set Terraform environment logs: You must also set the TF_LOG environment variable to DEBUG (or higher, such as TRACE) to actually output the logs [1][3][4]. Example configuration: provider "auth0" { debug = true } Then run your Terraform command with the environment variable set: export TF_LOG=DEBUG terraform plan Behavior and Security When these settings are combined, the provider enables HTTP request and response logging [1][2]. This captures interaction with the Auth0 Management API, providing visibility into the calls made by the provider during execution [5]. Security Warning: Enabling debug logging may output sensitive information to your logs, including credentials or other sensitive data present in API requests or responses [6][2]. Terraform documentation and security best practices strongly advise against hard-coding sensitive credentials (like CLIENT_SECRET) in your configuration files [6][2]. Always use environment variables or secret management tools for sensitive data to avoid accidental exposure, particularly if logs are stored or shared [6]. Note: A recent update (April 2026) ensured that debug logging is correctly applied to both V1 and V2 management API clients by ensuring the debug transport is properly wrapped around the final HTTP client [5]. If you do not see logs despite setting these variables, ensure your provider is updated to a version that includes this fix [5].
Citations:
- 1: https://registry.terraform.io/providers/auth0/auth0/latest/docs
- 2: https://library.tf/providers/auth0/auth0/latest
- 3: https://docs.hashicorp.com/terraform/plugin/log/managing
- 4: https://docs.hashicorp.com/terraform/internals/debugging
- 5: Move debug logging after custom client creation auth0/terraform-provider-auth0#1526
- 6: https://github.com/auth0/terraform-provider-auth0/blob/66258f8c/docs/guides/quickstart.md
Disable Auth0 provider HTTP logging by default. debug = true turns on request/response logging when Terraform debug logging is enabled, which can expose credentials or tenant/user data in CI and operator logs. Remove this setting or set it to false.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@terraform/kubernetes/auth0/auth0_main.tf` around lines 11 - 13, Update the
auth0 provider configuration to disable HTTP debugging by removing the debug
setting or setting debug to false, ensuring request and response data is not
logged by default.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
.github/workflows/e2e_calibrate_soc.yaml (1)
46-65: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winPass workflow inputs through environment variables instead of shell interpolation.
soc_refanddx_imageare inserted directly into shell source, allowing crafted values to execute commands. The image update is also masked with|| true, so the workflow may test the wrong image.🔒 Proposed safe input handling
env: + SOC_REF: ${{ inputs.soc_ref }} + DX_IMAGE: ${{ inputs.dx_image }} PX_CLOUD_ADDR: pixie.austrianopencloudcommunity.org PX_DEPLOY_KEY: ${{ secrets.PX_DEPLOY_KEY }} PX_API_KEY: ${{ secrets.PX_API_KEY }} TS_AUTHKEY: ${{ secrets.TAILSCALE_AUTH_KEY }} CLICKHOUSE_ANALYST_PASSWORD: ${{ secrets.CLICKHOUSE_ANALYST_PASSWORD }} CLICKHOUSE_INGEST_PASSWORD: ${{ secrets.CLICKHOUSE_INGEST_PASSWORD }} CLICKHOUSE_PIXIE_PASSWORD: ${{ secrets.CLICKHOUSE_PIXIE_PASSWORD }} run: | set -euo pipefail sudo apt-get update -qq && sudo apt-get install -y python3-yaml - git clone --depth 1 -b "${{ inputs.soc_ref }}" https://github.com/k8sstormcenter/soc soc + git clone --depth 1 -b "$SOC_REF" -- https://github.com/k8sstormcenter/soc soc cd soc make pixie # vizier + AE make kubescape || true # node-agent (netStreaming) bash tree/clickhouse-lab/install.sh # forensic_db make java-poc - if [ -n "${{ inputs.dx_image }}" ]; then - kubectl -n honey set image ds/dx-daemon dx-daemon="${{ inputs.dx_image }}" || true + if [ -n "$DX_IMAGE" ]; then + kubectl -n honey set image ds/dx-daemon dx-daemon="$DX_IMAGE" fi🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/e2e_calibrate_soc.yaml around lines 46 - 65, Update the workflow step to expose soc_ref and dx_image through dedicated environment variables, then reference those variables in the shell commands instead of interpolating workflow inputs directly. Preserve the conditional image-update behavior, but remove the unconditional `|| true` from the `kubectl set image` command so a failed update stops the workflow.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/e2e_calibrate_soc.yaml:
- Around line 93-94: Update the workflow’s surrounding shell script to evaluate
the existing fail variable after the final evidence command and exit nonzero
when any harness failed. Also capture the final grep pipeline’s status, treating
no matches as a failure, while preserving the current log output and
fail-tracking behavior.
---
Duplicate comments:
In @.github/workflows/e2e_calibrate_soc.yaml:
- Around line 46-65: Update the workflow step to expose soc_ref and dx_image
through dedicated environment variables, then reference those variables in the
shell commands instead of interpolating workflow inputs directly. Preserve the
conditional image-update behavior, but remove the unconditional `|| true` from
the `kubectl set image` command so a failed update stops the workflow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: a96976ce-7a9a-4a74-a678-561958a6753b
📒 Files selected for processing (1)
.github/workflows/e2e_calibrate_soc.yaml
Signed-off-by: Duck <70207455+entlein@users.noreply.github.com>
Signed-off-by: Duck <70207455+entlein@users.noreply.github.com>
Signed-off-by: Duck <70207455+entlein@users.noreply.github.com>
Signed-off-by: Duck <70207455+entlein@users.noreply.github.com>
) Remove src/e2e_test/adaptive_export_loadtest/harness/ entirely so it is gone after #83 merges. The folder held local attack-simulation / experiment driver scripts (javapoc_fire.sh, exp_*.sh, inject.sh, nfr.sh, run.sh, lib.sh, etc.) that should not live in the repo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017k7uYSNUctQvkTAZYJbaB3
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
Neutralise every occurrence of log4j / log4shell / attacker across the repo so none remain on main after #83 merges (verified: git grep for all three is empty). - AE test fixtures: log4j-poc* -> svc-poc*, log4j-chain-backend-vulnerable -> chain-backend-vuln, log4j-vuln-1.0.jar -> app-vuln-1.0.jar, jndi ldap 'attacker' payloads -> evil.example. go test ./internal/{anomaly,control,pxl, trigger,sink}/... all green (renames internally consistent, incl. query_id strings). - Comments: attacker-controlled -> adversary-controlled (queryfor.go, clickhouse.go). - Docs: loadtest README + PEM DIRECT_QUERY_{SECURITY,CONTRACT}.md + direct_query_server_test.cc reworded (log4shell -> poc, Attacker -> Adversary). - copybara: drop the e2e_log4shell_soc.yaml ignore entry (file not in tree). - terraform.tfstate: removed entirely (committed state is bad practice and it embedded the terms). *** SECURITY NOTE — needs review before merge *** Per the 'every occurrence' directive this ALSO removes two CVE-2021-44228 (log4shell) MITIGATIONS, because the mitigation strings literally contain 'log4j': - k8s/.../elastic/cluster/elastic_cluster.yaml: dropped -Dlog4j2.formatMsgNoLookups=True - tools/docker/elasticsearch/Dockerfile: dropped the RUN that strips JndiLookup.class from log4j-core-2.11.1.jar Removing these re-exposes that Elasticsearch to log4shell. Revert these two hunks if the mitigations should stay. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017k7uYSNUctQvkTAZYJbaB3
Restore the three files the scrub should not have touched: - k8s/.../elastic/cluster/elastic_cluster.yaml: -Dlog4j2.formatMsgNoLookups=True - tools/docker/elasticsearch/Dockerfile: JndiLookup.class strip (both are CVE-2021-44228 mitigations; keep them) - terraform/kubernetes/cloud_deps/terraform.tfstate: restored in full The remaining log4j/attacker strings after this are confined to these legit mitigations + the tfstate; all attack/PoC fixtures, docs and the harness stay scrubbed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017k7uYSNUctQvkTAZYJbaB3
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
entlein
approved these changes
Jul 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fresh replay of #72 onto current
main(1272bcd2a) so we 3-way merge against copybara's newer main instead of fighting the stale#2391base. Supersedes #72 (kept open for now).Why a new branch
#72 was based on
53d09cc8d(pixie-io#2391). Main has since moved (notably #75 "remove k8s directory pins"), which repeatedly re-conflictedtools/private/copybara/copy.bara.sky. This branch isgit rebase --onto origin/main'd, soorigin/mainis a direct ancestor and the copybara change is already merged in.Contents (9 commits, authorship preserved)
copy.bara.sky: protect fork-only trees from OSS-mirror deletion; merged with main's k8s-pin removal.private/+tools/private/roots (still blocks stray private paths); gazelleBUILD.bazeladded for the restored loadgen command packages (cmd/cleanloadgen,cmd/httpsink) — regenerated with the px_dev bazel 7.7.1 toolchain, verified idempotent.Verification
origin/mainis a direct ancestor; zero rebase conflicts.copy.bara.skycorrectly absorbing main's changes.md_udtfsline incopy.bara.sky— valid Python, harmless.🤖 Generated with Claude Code