Skip to content

feat(observability): profiles end-to-end (gateway + UI panel)#5382

Draft
Ma77Ball wants to merge 195 commits into
apache:mainfrom
Ma77Ball:obs/pr8/profiles
Draft

feat(observability): profiles end-to-end (gateway + UI panel)#5382
Ma77Ball wants to merge 195 commits into
apache:mainfrom
Ma77Ball:obs/pr8/profiles

Conversation

@Ma77Ball

@Ma77Ball Ma77Ball commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Continuous-profiling retrieval through the gateway plus a flame-graph UI panel.
Backend:

  • ParcaClient: gRPC-Web client to the Parca server, with a profiles query builder and parseProfiles.
  • ProfilesResource exposing scoped profile queries (optional workflow and execution id), registered in TexeraWebApplication.
    Frontend:
  • Profiles panel with a flame-chart component, scoped by workflow and execution id, with explicit empty states for "no samples yet" and for environments where profiling is disabled.

Any related issues, documentation, or discussions?

Closes: #5374
Part of #4070. Stacked on #5381.

How was this PR tested?

  • Backend specs for ParcaClient, the profiles query builder, and the parser; sbt scalafmtCheckAll passes.
  • Frontend profiles-panel and flame-chart specs; prettier-eslint and eslint pass.
  • Compile and the full test suites run in this PR's CI.

Was this PR authored or co-authored using generative AI tooling?

Co-authored with Claude Opus 4.8 in compliance with ASF

Ma77Ball and others added 8 commits June 5, 2026 04:49
…, SDK bootstrap (default-off)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s panel

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nt scope, health, routing)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ca/eBPF profiling

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tracing primitives

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…I flame-graph panel

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… trace tree panel

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…UI metrics panel (ECharts)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added engine dependencies Pull requests that update a dependency file frontend Changes related to the frontend GUI docs Changes related to documentations infra common labels Jun 5, 2026
@codecov-commenter

codecov-commenter commented Jun 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 59.08892% with 934 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.22%. Comparing base (7a38b6c) to head (5e8640f).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
...observability/gateway/ObservabilityResources.scala 0.00% 314 Missing ⚠️
...texera/web/observability/gateway/ParcaClient.scala 40.10% 98 Missing and 20 partials ⚠️
...observability/logs-panel/logs-panel.component.html 56.71% 58 Missing ⚠️
...ala/org/apache/texera/observability/OtelInit.scala 63.30% 43 Missing and 8 partials ⚠️
...apache/texera/web/observability/gateway/dtos.scala 76.28% 40 Missing and 6 partials ⚠️
...texera/web/observability/gateway/AuditLogger.scala 0.00% 41 Missing ⚠️
...xera/web/observability/gateway/ScopeResolver.scala 19.44% 27 Missing and 2 partials ⚠️
...ra/web/observability/gateway/ResponseParsers.scala 77.96% 10 Missing and 16 partials ⚠️
...rvability/traces-panel/traces-panel.component.html 42.10% 22 Missing ⚠️
...era/web/observability/gateway/GatewayContext.scala 0.00% 21 Missing ⚠️
... and 30 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #5382      +/-   ##
============================================
+ Coverage     54.89%   55.22%   +0.32%     
- Complexity     2962     2996      +34     
============================================
  Files          1117     1154      +37     
  Lines         43133    45602    +2469     
  Branches       4648     5098     +450     
============================================
+ Hits          23680    25184    +1504     
- Misses        18064    18898     +834     
- Partials       1389     1520     +131     
Flag Coverage Δ *Carryforward flag
access-control-service 70.14% <100.00%> (+0.14%) ⬆️
agent-service 34.36% <ø> (ø) Carriedforward from 6c77d4a
amber 56.37% <48.95%> (-0.70%) ⬇️
computing-unit-managing-service 0.00% <0.00%> (ø)
config-service 50.76% <0.00%> (-0.80%) ⬇️
file-service 58.88% <0.00%> (-0.15%) ⬇️
frontend 49.94% <80.24%> (+1.33%) ⬆️
notebook-migration-service 78.57% <ø> (ø)
pyamber 91.18% <ø> (+0.98%) ⬆️ Carriedforward from 6c77d4a
python 90.76% <ø> (ø) Carriedforward from 6c77d4a
workflow-compiling-service 54.74% <0.00%> (-0.41%) ⬇️

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Ma77Ball added 3 commits June 23, 2026 15:21
  catch JSON/quoted creds, token=/api_key=/secret=,
  and JWTs (real leak gap).
  2. ParcaClient — bounds-checked the protobuf parser
  (corrupt frame could OOM/crash) + reused one
  HttpClient.
  3. Frontend — deduped 4 copy-pasted humanizeError
  functions into one shared util.
  4. Profiles deep link — validate comm before
  building the Parca URL.
  5. Stale comments — removed outdated coming soon notes;
@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @bobbai00, @aicam, @Yicong-Huang
    You can notify them by mentioning @bobbai00, @aicam, @Yicong-Huang in a comment.

Ma77Ball added 26 commits June 26, 2026 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common dependencies Pull requests that update a dependency file docs Changes related to documentations engine frontend Changes related to the frontend GUI infra platform Non-amber Scala service paths

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Observability] Continuous profiling: gateway endpoint and flame-graph panel

2 participants