Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- ---
- name: judge-evaluation
- description: "Judge an evaluate-turn report and keep only findings that are real and worth fixing. Use when asked to judge an evaluation, filter a review report, or decide which flagged issues actually matter. Not for re-reviewing the diff from scratch or implementing fixes."
- ---
- # judge-evaluation
- Input is the evaluate-turn report, not the chat that produced it. The diff is evidence. Open it only to confirm or kill a finding. Do not start a new review.
- ## Keep a finding only if all of these hold
- 1. The line exists and the report describes what that line does.
- 2. The wrong behavior is reachable on the claim path or by a current caller.
- 3. Fixing it changes a result, an error, persisted data, or a safety property. Deleting noise counts only if the noise hides one of those.
- 4. A senior reviewer would send the patch back for this, not just mention it.
- If any check fails, drop it. Do not downgrade it into a nit to keep it alive.
- ## Drop
- - Restating the claim, or a finding with no path:line.
- - "Could be more generic", "add a comment", "consider extracting".
- - Style the surrounding untouched code already uses.
- - Hypothetical load, concurrency, or abuse the claim does not require.
- - A pre-existing bug the diff does not make reachable.
- - Duplicate of a stronger finding. Keep the one with the failing input.
- - A blocker whose path is unexported and has no caller. That is a defect, or nothing.
- ## Severity check
- - Blocker: wrong result on the stated path, unwired claim, data loss, auth or injection hole, test that cannot fail on broken code.
- - Defect: wrong edge or error path a caller can hit.
- - Slop: dead code or a swallowed error in the diff that conceals one of the above. Cap at the ones you kept.
- The report's severity is a claim. Override it.
- ## Report
- Verdict: accept | amend | discard
- Kept: <n> of <m>
- Keep
- - [severity] path:line — failing input or caller. Why it is worth a fix.
- Drop
- - [original severity] path:line — which check failed, in one line.
- Overrides
- - path:line — from <old> to <new>, and why. Omit if none.
- `discard` if nothing is kept. `amend` if anything was dropped or re-ranked. `accept` only if every original finding survived unchanged. Do not add findings the evaluation did not make.
Advertisement
Add Comment
Please, Sign In to add comment