Users and enterprises are handing AI models and agents more autonomy， so the guardrails that screen their inputs and outputs matter more than ever. However， the benchmarks for evaluating those guardrails haven't kept pace with model intelligence

In partnership with @nvidia， we independently benchmarked guardrail and moderation models across three open datasets， measuring detection quality， latency， and the tradeoff between catching unsafe content and over-refusing safe content. No model wins outright， and there is still no common standard for judging them. We see this as an early step in a measurement problem that will continue to grow more important as models take on more real-world work.