Banger paper introducing Jev-as-a-Judge. The overall finding is that you want to use a cheap judge for most of your evals and send only the uncertain calls to a frontier model. This paper measures how well that works with JEV, TypeSafe AI's decision-only judge. On 510 held-out

DAIR.AI @dair_ai ·
Banger paper introducing Jev-as-a-Judge. The overall finding is that you want to use a cheap judge…

更多来自 @dair_ai

  1. #1

    Insightful paper from Microsoft and colleagues. If you have ever had an agent run fail 80 steps ago with no way to find where, this one is for you. (bookmark it) Agent failures sh…

    DAIR.AI @dair_ai ·
    Insightful paper from Microsoft and colleagues. If you have ever had an agent r…
  2. #2

    Brilliant new paper from the Qwen team. It provides insights into where agent training environments actually come from. Terminal agent trajectories have accumulated at scale while…

    DAIR.AI @dair_ai ·
    Brilliant new paper from the Qwen team. It provides insights into where agent t…
  3. #3

    Great tips on working with reasoning models. Normally you would append what the model figured out after the document and ask again. It turns out that where you put the reasoning t…

    DAIR.AI @dair_ai ·
    Great tips on working with reasoning models. Normally you would append what the…
  4. #4

    Brilliant paper on long-horizon agents. They cut 78.9% of an agent's LLM calls while raising its success rate. Here is how: It turns out that ReAct issues one primitive action per…

    DAIR.AI @dair_ai ·
    Brilliant paper on long-horizon agents. They cut 78.9% of an agent's LLM calls…
  5. #5

    Banger paper from BAAI. If you are building research agents, this one is worth your time. (bookmark it) They find that adding skills scores 134.3% higher on MLE-bench, 34.4% highe…

    DAIR.AI @dair_ai ·
    Banger paper from BAAI. If you are building research agents, this one is worth…
  6. #6

    Good measurement work on whether retrieved agent skills actually help. They report that agent skills that lift your aggregate score can be hurting every task they touch. The usual…

    DAIR.AI @dair_ai ·
    Good measurement work on whether retrieved agent skills actually help. They rep…

查看 @dair_ai 的全部帖子