RajB47

judge evaluation skill

Oct 1st, 2026
233
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 2.27 KB | Source Code | 0 0
  1. ---
  2. name: judge-evaluation
  3. description: "Judge an evaluate-turn report and keep only findings that are real and worth fixing. Use when asked to judge an evaluation, filter a review report, or decide which flagged issues actually matter. Not for re-reviewing the diff from scratch or implementing fixes."
  4. ---
  5.  
  6. # judge-evaluation
  7.  
  8. Input is the evaluate-turn report, not the chat that produced it. The diff is evidence. Open it only to confirm or kill a finding. Do not start a new review.
  9.  
  10. ## Keep a finding only if all of these hold
  11.  
  12. 1. The line exists and the report describes what that line does.
  13. 2. The wrong behavior is reachable on the claim path or by a current caller.
  14. 3. Fixing it changes a result, an error, persisted data, or a safety property. Deleting noise counts only if the noise hides one of those.
  15. 4. A senior reviewer would send the patch back for this, not just mention it.
  16.  
  17. If any check fails, drop it. Do not downgrade it into a nit to keep it alive.
  18.  
  19. ## Drop
  20.  
  21. - Restating the claim, or a finding with no path:line.
  22. - "Could be more generic", "add a comment", "consider extracting".
  23. - Style the surrounding untouched code already uses.
  24. - Hypothetical load, concurrency, or abuse the claim does not require.
  25. - A pre-existing bug the diff does not make reachable.
  26. - Duplicate of a stronger finding. Keep the one with the failing input.
  27. - A blocker whose path is unexported and has no caller. That is a defect, or nothing.
  28.  
  29. ## Severity check
  30.  
  31. - Blocker: wrong result on the stated path, unwired claim, data loss, auth or injection hole, test that cannot fail on broken code.
  32. - Defect: wrong edge or error path a caller can hit.
  33. - Slop: dead code or a swallowed error in the diff that conceals one of the above. Cap at the ones you kept.
  34.  
  35. The report's severity is a claim. Override it.
  36.  
  37. ## Report
  38.  
  39. Verdict: accept | amend | discard
  40. Kept: <n> of <m>
  41.  
  42. Keep
  43. - [severity] path:line — failing input or caller. Why it is worth a fix.
  44.  
  45. Drop
  46. - [original severity] path:line — which check failed, in one line.
  47.  
  48. Overrides
  49. - path:line — from <old> to <new>, and why. Omit if none.
  50.  
  51. `discard` if nothing is kept. `amend` if anything was dropped or re-ranked. `accept` only if every original finding survived unchanged. Do not add findings the evaluation did not make.
Tags: blog prompts
Advertisement
Add Comment
Please, Sign In to add comment