Skip to content

[Feature] : API ENDPOINTS PR 7 : Test Artifacts and Cleanups#1141

Open
pulk17 wants to merge 9 commits into
CCExtractor:masterfrom
pulk17:api-pr7-artifacts
Open

[Feature] : API ENDPOINTS PR 7 : Test Artifacts and Cleanups#1141
pulk17 wants to merge 9 commits into
CCExtractor:masterfrom
pulk17:api-pr7-artifacts

Conversation

@pulk17

@pulk17 pulk17 commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

[FEATURE]

In raising this pull request, I confirm the following (please check boxes):

  • I have read and understood the contributors guide.
  • I have checked that another pull request for this purpose does not exist.
  • I have considered, and confirmed that this submission will be valuable to others.
  • I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
  • I give this submission freely, and claim no ownership to its content.

My familiarity with the project is as follows (check one):

  • I have never used the project.
  • I have used the project briefly.
  • I have used the project extensively, but have not contributed previously.
  • I am an active contributor to the project.

Test Artifacts and API Cleanup (PR 7)

Summary

Follow-up to the REST API stack (#1130#1135). This PR completes the
test-artifact feature end to end: the CI test VMs now upload the run binary,
any core dump, and the combined stdout/stderr log, and the platform stores them
where GET /runs/{id}/artifacts (from PR 3) already serves them. It also folds
in the small leftover review nits from the API stack and adds a way to verify
the runner scripts.

Stacking: builds on PR 6 (#1135). Until #1135 merges, the diff here will
include its files — please review #1135 first.

How artifacts flow

  1. VM → platform (producers). ci-linux/ci/runCI and ci-windows/ci/runCI.bat
    capture the build binary, a core dump (Linux), and the run's combined
    stdout/stderr, and upload them over the existing reportURL channel with a
    new type=artifactupload form — the same mechanism already used for
    logupload. Uploads are best-effort and never abort a run. Windows skips the
    core dump for v1 (the API tolerates a missing one).
  2. Platform ingestion. progress_reporter gains an artifactupload handler
    that stores each file under SAMPLE_REPOSITORY/test_artifacts/<run_id>/ using
    the exact name the API resolves. The target name is derived server-side from
    the artifact type + platform — never from the uploaded filename — so a crafted
    filename can't escape the artifact directory.
  3. API serving. No change needed — GET /runs/{id}/artifacts already reads
    test_artifacts/<run_id>/... and returns signed URLs. (As with the existing
    log/result uploads, this assumes SAMPLE_REPOSITORY is the GCS-backed mount
    on the platform host.)

startup-script.sh also gets three small VM-provisioning fixes (apt -y, a
TempFiles dir for the tester, and a short wait for the gcsfuse mounts).

Verifying the runner scripts

New tests/ci-vm/ checks, runnable locally and in CI, with no real VM/GCS:

  • Linux (runCI): behavioural smoke test — stubs curl/sudo/shutdown
    and the tester, runs the script in a sandbox, and asserts it posts
    preparation → testing → completed, uploads the build log, and uploads the
    binary and combined_stdout artifacts.
  • Windows (runCI.bat): static check — a faithful behavioural mock isn't
    practical on Windows (postStatus's retry loop needs a real curl, an
    interactive timeout, and set /p file reads), so this asserts the script
    parses and contains the artifact-upload wiring; the real behaviour runs on the
    Windows CI VM.
  • A ci-vm-scripts GitHub Actions matrix runs shellcheck + the Linux smoke
    test on ubuntu, and PSScriptAnalyzer + the static check on windows, whenever
    anything under install/ci-vm/ changes.

While writing the Windows harness I found and fixed a pre-existing Windows
bug in runCI.bat: the preparation status used call :postStatus ... >> %logFile%,
an outer redirect that the Linux runner and every other postStatus call don't
have. On Windows that outer redirect holds log.html open, so postStatus's
internal >> %logFile% to the same file fails with a sharing violation. Dropping
the redundant redirect (now consistent with the other calls) resolves it.

Leftover API fixes (from review of the stack)

  • list_tokens is admin-only (contributor/tester could never hold the required
    tokens:manage scope, so the wider role list was misleading).
  • Removed the unreachable cross-user-revoke 403 branch in revoke_specific_token.
  • Dropped the dead repository-optional fallback in create_run (the field is
    required by the schema).
  • Trimmed the per-sample ?status filter to the statuses the engine can emit.
  • Infrastructure-error reporting no longer counts user-initiated (via API)
    cancellations as infra failures.
  • The build-log ?level filter is now case-insensitive.

Testing

  • New platform-ingestion unit tests and an infra-error test; the affected route
    suites (auth, runs, samples) and the error-service suite pass.
  • The Linux runner smoke test passes; the Windows static check passes; the
    Windows runner's end-to-end behaviour is validated on the windows-latest
    CI job.
  • isort / pydocstyle / pycodestyle / mypy clean on the changed modules.

@pulk17 pulk17 force-pushed the api-pr7-artifacts branch from 542cd43 to 3236d01 Compare June 29, 2026 18:57
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant