Skip to content

Protect tracklet evaluation inputs from output collisions - #271

Draft
FlorianPfaff wants to merge 2 commits into
mainfrom
agent/protect-tracklet-evaluation-paths
Draft

Protect tracklet evaluation inputs from output collisions#271
FlorianPfaff wants to merge 2 commits into
mainfrom
agent/protect-tracklet-evaluation-paths

Conversation

@FlorianPfaff

Copy link
Copy Markdown
Member

Summary

  • resolve the effective metrics, Markdown report, and match CSV paths before evaluation starts
  • reject output paths that alias the truth labels, prediction tracks, or another output artifact
  • account for normalized relative paths, existing symlinks, and hard links where the filesystem can resolve them
  • add regressions proving rejected configurations preserve both truth input and existing output bytes

Root cause

beltmap-evaluate-tracklets forwarded user-selected output paths directly to generate_tracklet_evaluation_report(). The generator writes metrics JSON, then match CSV, then the Markdown report without checking whether those paths are distinct.

Consequently, an output could overwrite the truth or prediction input after it had been loaded, or a later output could silently replace an earlier artifact while the command still returned success. Equivalent path spellings such as reports/../truth.json were also accepted.

Impact

Destructive or ambiguous path layouts now fail through the CLI parser before any evaluation read/write workflow begins. Normal default and distinct explicit paths are unchanged.

Validation

  • branch is 2 commits ahead of and 0 commits behind main
  • regression covers a metrics path aliasing the truth input and verifies the truth bytes remain unchanged
  • regression covers metrics/report output aliasing and verifies an existing output remains unchanged
  • repository-wide tests and lint are delegated to GitHub Actions

@github-actions

Copy link
Copy Markdown
Contributor

MegaLinter analysis: Success

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ ACTION actionlint 10 0 0 0.78s
✅ COPYPASTE jscpd yes no no 3.25s
✅ MARKDOWN markdownlint 24 0 0 0 1.96s
✅ PYTHON ruff 207 1 0 0 0.39s
✅ REPOSITORY git_diff yes no no 0.04s
✅ YAML prettier 12 0 0 0 1.27s
✅ YAML v8r 12 0 0 4.55s
✅ YAML yamllint 12 0 0 0.89s

Notices

📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)

See detailed reports in MegaLinter artifacts

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant