随着用户和企业赋予 AI 模型与智能体更高自主权,其输入输出护栏的重要性持续上升。Artificial Analysis 与 NVIDIA 合作,在三个开放数据集上独立基准测试了护栏与审核模型,评估检测质量、延迟以及在捕获不安全内容与过度拒绝安全内容之间的权衡。结果显示无模型全面领先,且业内仍缺乏统一评判标准。该研究被视为这一日益重要的评估问题的早期探索。
Users and enterprises are handing AI models and agents more autonomy, so the guardrails that screen their inputs and outputs matter more than ever. However, the benchmarks for evaluating those guardrails haven't kept pace with model intelligence
In partnership with @nvidia, we independently benchmarked guardrail and moderation models across three open datasets, measuring detection quality, latency, and the tradeoff between catching unsafe content and over-refusing safe content. No model wins outright, and there is still no common standard for judging them. We see this as an early step in a measurement problem that will continue to grow more important as models take on more real-world work.