All issues

Bare-name tool alias silently re-registers on a 3rd+ collision, only the 2nd logs a warning

#515

· AmirF194 opened

on Aug 17, 2026

An empty completion crashes `plan_execute` with a `TypeError` that blames the regex

#511

· arthapraha opened

on Aug 14, 2026

None of the 2,667 FailureSensorIQ questions reference an entity in the loaded database

#509

· arthapraha opened

on Aug 13, 2026

Two scenarios are answerable with no tools mounted — a contamination control, and what it caught

#508

· arthapraha opened

on Aug 13, 2026

14 of 55 scenario questions can be answered from the shipped data — the audit, with dispositions

#507

· arthapraha opened

on Aug 13, 2026

Are the public artefacts meant to be scoreable end-to-end? Documented ground truth for MAIN and the shipped data overlap by one asset

#506

· arthapraha opened

on Aug 13, 2026

`static_json` scores differently-named assets as perfect matches when their names share a digit

#504

· arthapraha opened

on Aug 13, 2026

Evidence from 936 runs: an answer-format contract makes `plan_execute` output gradeable

#503

· arthapraha opened

on Aug 13, 2026

Industrial Automation Challenge Track 1 grader format clarification

#502

· Harshul-24 opened

on Aug 13, 2026

plan-execute runner's metered token usage never reaches persisted trajectories, so evaluation reports 0 tokens / no cost

#499

· AmirF194 opened

on Aug 8, 2026

Fix static JSON evaluator parsing for noisy final numeric answers

#494

· ChathurangiShyalika opened

on Jul 29, 2026

Static JSON evaluator fails noisy final MCQ answers

#475

· ChathurangiShyalika opened

on Jul 22, 2026