Is your feature request related to a problem?
Prompt iteration currently works on the old evaluation flow: LLM-as-judge correctness and cosine similarity results were used to generate prompt improvements. V2 evals now produce three metrics — adherence to ground truth, adherence to prompt, and adherence to knowledge base — each with a score and judge reasoning. The existing endpoint neither consumes the new results shape nor reasons over the per-metric scores.
Describe the solution you'd like
- Add a v2 prompt iteration endpoint that consumes the three-metric eval results instead of correctness + cosine similarity
- Use both the score and the judge reasoning of each of the three metrics when generating prompt improvement suggestions
- Update the recommendation prompt accordingly
- Keep the API extensible for future recommendation types (knowledge base gaps, model changes) — prompt recommendations first
Is your feature request related to a problem?
Prompt iteration currently works on the old evaluation flow: LLM-as-judge correctness and cosine similarity results were used to generate prompt improvements. V2 evals now produce three metrics — adherence to ground truth, adherence to prompt, and adherence to knowledge base — each with a score and judge reasoning. The existing endpoint neither consumes the new results shape nor reasons over the per-metric scores.
Describe the solution you'd like