Ethan Mollick 指出,GPT-5.5 Pro Extended 和 Claude 5 Fable Max 在 Beninatto‑Trombetti 翻译测试中失败。该测试要求将“Solo 3 parole: non sei solo”译为英语,同时将 meta‑linguistic 声明从“3 parole”更新为“4 words”(正确译文:“Just 4 words: you are not alone”)。但前沿模型拒绝修改措辞,即使提示扮演翻译角色仍回避变更。Valerio Capraro 认为,Claude 5 Fable 作为最新 LLM 仍无法通过此简单测试,说明 LLM 擅重组已知知识但缺乏真正理解,AGI 仍遥远。
This is an interesting test, and the frontier models (GPT-5.5 Pro Extended, Claude 5 Fable Max) do fail. They refuse to turn the "three words" into "four" if that fits better
Prompting the AI to act like a translator surfaces the problem, but it still avoids changing the wording