Rohan Paul@rohanpaul_ai · 1小时前73

A Nature Medicine study found general-purpose LLMs are now outperforming dedicated medical AI products on physician-reviewed clinical tasks. The authors compared OpenEvidence and UpToDate Expert AI with GPT-5.2, Gemini 3.1 Pro, and Claude Opus 4.6 on medical exam questions, clinician-style answers, and real questions doctors asked during care. In 100 de-identified physician questions from live clinical use, blinded clinicians again preferred the frontier models, especially on completeness and clarity,

译《自然·医学》一项研究发现，通用大语言模型在经医生评审的临床任务上已超越专用医疗 AI 产品。研究对比了 OpenEvidence、UpToDate Expert AI 与 GPT-5.2、Gemini 3.1 Pro、Claude Opus 4.6 在医学考试题、医生风格回答及实时临床提问上的表现。在来自真实临床场景的 100 个脱敏医生问题中，盲审医生更偏好前沿模型，尤其在其回答的完整性和清晰度方面。

Rohan Paul@rohanpaul_ai · 2小时前53

Beautiful paper from Google DeepMind. Explains the pathways from AGI to ASI, and why that jump could happen through several routes. The authors frame the AGI-to-ASI transition around 4 technical pathways: - continued scaling of compute, model size, data, and test-time inference; - algorithmic paradigm shifts beyond today’s transformer-based foundation-model stack; - recursive self-improvement, where AI accelerates AI R&D and improves future systems; and - multi-agent collective intelligence, where large populations of specialized agents coordinate into a superhuman group agent. Scaling may work for a while, but it could hit limits in data, compute, energy, or weaker returns from making systems larger. Recursive improvement is the most uncertain path, because AI could speed up AI research, but that loop may also slow if hard research problems need real-world testing, scarce hardware, or new ideas. Multi-agent collectives may be the most underappreciated path, because a society of competent digital workers could outperform a brilliant individual model through specialization, speed, and coordination. The big point is that ASI may not arrive as 1 sudden event, but as a chain of faster changes as AI helps create better AI and stronger scientific tools. ---- Link – arxiv. org/abs/2606.12683 Title: "From AGI to ASI"

译Google DeepMind新论文提出从通用人工智能到超级智能的四条路径：持续扩展（计算、模型规模、数据、测试时推理）、算法范式革新（超越Transformer架构）、递归自我改进（AI加速自身研发）、多智能体集体智能（众多专业AI智能体协作出超人类智能）。扩展可能遇到数据、算力、能源瓶颈；递归改进最不确定；多智能体路径最易被低估，通过专业化与协调能超越单个强模型。ASI可能不是单次跃迁，而是AI辅助创造更好AI的加速链。

Microsoft Research@MSFTResearch · 3小时前15

Project Ire examined a timely malware sample and determined its intent through reverse engineering—identifying LOTUSLITE characteristics even as most major EDR tools did not detect it. https://msft.it/6011viy4N

译Project Ire 分析了一个及时的恶意软件样本，并通过逆向工程确定其意图——识别出 LOTUSLITE 特征，即使大多数主流 EDR 工具未检测到它。https://msft.it/6011viy4N

AK@_akhaliq · 5小时前46

SpenseGPT Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference

译SpenseGPT 实用的一次性剪枝，实现LLM推理的稀疏和密集GEMM

Rohan Paul@rohanpaul_ai · 6小时前43

Most AI agents do not forget because they lack memory; they fail because they remember badly. AGENTCL asks a simple question: does an AI agent really learn from experience, or merely carry clutter forward? Today's agents can spend enormous effort solving one task, then enter the next one almost as if nothing happened. AGENTCL says AI agents need better tests for whether their memory actually helps them learn across tasks. The paper’s main idea is to build task streams where earlier tasks clearly contain pieces that later tasks can reuse, such as a small coding function, evidence for a research question, or a useful workflow. It compares these careful “compositional” streams with normal “naive” streams, where tasks come from the same area but do not have a guaranteed reuse link. Agent memory is easy to overrate when the benchmark is messy. If tasks are not carefully connected, a memory system may look good for the wrong reason, or bad for a reason the test cannot explain. AGENTCL tries to fix that by making the task relationships clear, then measuring whether memory helps on later tasks, stays useful, and transfers to unseen tasks. The key finding is that today’s memory methods can reuse past work when the connection is obvious, but they still struggle to avoid confusion when the next task is different. ---- Link – arxiv. org/abs/2606.02461 Title: "AGENTCL: Toward Rigorous Evaluation of Continual Learning in Language Agents"

译AGENTCL 提出评估 AI 智能体是否真正从经验学习，而非单纯累积信息。通过构建组合任务流（前序任务包含可被后续任务复用的代码片段、研究证据或工作流），与无固定复用线索的随意任务流对比。关键发现：当前记忆方法在任务连接明显时可复用过去经验，但当任务差异较大时仍难以避免混淆。论文旨在为智能体持续学习提供更清晰的测评标准。

Epoch AI@EpochAIResearch · 6小时前64

FrontierMath: Tiers 1–4 (v2) is live. We concluded an audit that addressed errors in 42% of problems. Rankings are similar but scores are higher across the board. The current leaders are GPT-5.5 (xhigh) with 85% on Tiers 1–3 and Google’s AI co-mathematician with 76% on Tier 4.

译FrontierMath: Tiers 1–4 (v2) 现已上线。 我们完成了一项审计，修正了 42% 的问题中的错误。排名相似，但整体得分更高。目前的领先者是 GPT-5.5 (xhigh)，在 Tiers 1–3 上达到 85%，以及 Google 的 AI co-mathematician，在 Tier 4 上达到 76%。

Jeff Dean@JeffDean · 7小时前48

Quite interesting thread on capabilities of real biological neurons (spoiler: they're way more capable than classical artificial neurons in a perceptron) . Nice work @IdoAizenbud and collaborators!

译据 Jeff Dean 转发，Ido Aizenbud 与合作者的新研究发现，单个皮层神经元能够对猫狗进行分类、识别口语单词并解决 10 位奇偶校验——这些任务此前被认为需要整个网络才能完成。

Ethan Mollick@emollick · 9小时前72

There has been a push to use OpenEvidence AI for doctors. But this paper suggests general models are much better: “Frontier LLMs outperformed clinical AI tools in all three evaluations. Clinical AI tools performed comparably to auto-enabled Google Search AI Overview on the RCQ.”

译一项发表在Nature Medicine的研究显示，通用前沿大语言模型（Google、OpenAI、Anthropic）在医学信息评估中全面优于专门的临床AI工具（OpenEvidence和UpToDate）。12名美国临床医生进行随机盲测，Frontier LLMs在三项评估中均胜出。临床AI工具的表现与自动启用的Google Search AI Overview在RCQ测试中相当。

Alibaba Cloud@alibaba_cloud · 19小时前66

🚀 Taming Agent Chaos? Paper reveals NLAH: Replace rigid code harnesses with executable natural language. ✅ Performance matches code, tokens drop 95% (60k→2.9k) ✅ Modular design enables precise value attribution ✅ Identifies "negative assets" like multi-candidate search Shift from glue code to scientific strategy. 💡https://int.alibabacloud.com/m/1000414388/ #AgentHarness #NLAH #LLMEngineering

译🚀 驯服智能体混乱？ 论文揭示NLAH：用可执行自然语言替代僵硬的代码框架。 ✅ 性能媲美代码，模型token降低95%（60k→2.9k） ✅ 模块化设计实现精确的价值归因 ✅ 识别“负面资产”，如多候选搜索 从胶水代码转向科学策略。 💡https://int.alibabacloud.com/m/1000414388/ #AgentHarness #NLAH #LLMEngineering

Alibaba Cloud@alibaba_cloud · 19小时前66

🚀 Taming Agent Chaos? Paper reveals NLAH: Replace rigid code harnesses with executable natural language. ✅ Performance matches code, tokens drop 95% (60k→2.9k) ✅ Modular design enables precise value attribution ✅ Identifies "negative assets" like multi-candidate search Shift from glue code to scientific strategy. 💡https://int.alibabacloud.com/m/1000414388/ #AgentHarness #NLAH #LLMEngineering

译🚀 驯服智能体混乱？ 论文揭示NLAH：用可执行自然语言替代刚性代码框架。 ✅ 性能与代码持平，token减少95%（60k→2.9k） ✅ 模块化设计实现精准价值归因 ✅ 识别“负资产”如多候选搜索 从胶水代码转向科学策略。 💡https://int.alibabacloud.com/m/1000414388/ #AgentHarness #NLAH #LLMEngineering

AK@_akhaliq · 20小时前67

Agents' Last Exam

译智能体的最后考试

AK@_akhaliq · 20小时前62

CHORUS Decentralized Multi-Embodiment Collaboration with One VLA Policy

译CHORUS 去中心化多本体协作，基于单一VLA策略。

Rohan Paul@rohanpaul_ai · 1天前62

This paper shows an AI improving itself better when it rewrites its setup and updates its model. The problem is that most AI progress still depends on people changing prompts, tools, code, training data, and model weights by hand. The paper’s idea is SIA, a loop where one AI watches how a task agent performs, then either changes the agent’s outer setup or trains the model itself. The outer setup means things like prompts, tools, retry rules, and output parsing, while weight updates mean changing the model’s learned behavior through task feedback. The loop works like this: the task agent tries many answers or programs, the verifier scores them, and those scores become training feedback. Then the system updates a small add-on set of weights called LoRA weights, which changes the model’s behavior without retraining the whole model. So the base model stays mostly the same, but the LoRA adapter learns, “outputs like this got high reward, outputs like that failed.” The authors tested this on 3 very different tasks: Chinese legal charge classification, GPU kernel speed tuning, and single-cell RNA denoising. The combined version beat setup-only improvement on all 3 tasks, reaching 70.1% on LawBench, faster GPU code than the prior best, and 0.289 on denoising. The main lesson is that better scaffolding helps the agent act better, but weight updates help it learn task patterns that prompts and tools alone did not find. ---- Link – arxiv. org/abs/2605.27276 Title: "SIA: Self Improving AI with Harness & Weight Updates"

译该论文提出SIA框架，让AI自动循环改进：一个观察者AI监控任务代理的表现，然后修改其外部设置（提示词、工具、重试规则、输出解析）或通过LoRA权重更新训练模型本身，模型主体不变，仅适配器从任务反馈中学习。在三个任务上测试：中文法律罪名分类（LawBench达70.1%）、GPU内核速度调优（生成代码优于此前最佳）、单细胞RNA降噪（得分0.289）。综合版本在所有任务上超越仅修改设置的方案，表明权重更新能帮助模型学到提示和工具无法发现的模式。

Epoch AI@EpochAIResearch · 1天前66

The record for computing capacity in a single data center has doubled every 7 months. Colossus 1, Anthropic-Amazon New Carlisle, and Meta Prometheus have each claimed the top spot in turn.

译单个数据中心的计算能力记录每 7 个月翻倍一次。 Colossus 1、Anthropic-Amazon New Carlisle 和 Meta Prometheus 依次登顶。

Artificial Analysis@ArtificialAnlys · 1天前61

Users and enterprises are handing AI models and agents more autonomy, so the guardrails that screen their inputs and outputs matter more than ever. However, the benchmarks for evaluating those guardrails haven’t kept pace with model intelligence In partnership with @nvidia, we independently benchmarked guardrail and moderation models across three open datasets, measuring detection quality, latency, and the tradeoff between catching unsafe content and over-refusing safe content. No model wins outright, and there is still no common standard for judging them. We see this as an early step in a measurement problem that will continue to grow more important as models take on more real-world work.

译随着用户和企业赋予 AI 模型与智能体更高自主权，其输入输出护栏的重要性持续上升。Artificial Analysis 与 NVIDIA 合作，在三个开放数据集上独立基准测试了护栏与审核模型，评估检测质量、延迟以及在捕获不安全内容与过度拒绝安全内容之间的权衡。结果显示无模型全面领先，且业内仍缺乏统一评判标准。该研究被视为这一日益重要的评估问题的早期探索。

AK@_akhaliq · 1天前60

Toward Generalist Autonomous Research via Hypothesis-Tree Refinement

译通过假设树精炼迈向通用自主研究

AK@_akhaliq · 1天前58

TRL-Bench Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders

译TRL-Bench 标准化跨范式表格编码器的表示级评估

AK@_akhaliq · 1天前61

Redesign Mixture-of-Experts Routers with Manifold Power Iteration

译用流形幂迭代重新设计混合专家路由器

Rohan Paul@rohanpaul_ai · 1天前55

The paper argues that sparse autoencoders may not be bad steering tools after all, and much of the earlier failure may have come from choosing and naming the wrong features. The problem is that earlier work made sparse autoencoders look weak because their features were labelled in a way that may not match what those features actually cause inside the model. A sparse autoencoder is a small helper model that breaks an LLM’s hidden activity into many possible “features,” such as a topic, style, or concept. So a sparse autoencoder finds directions inside a model, but an unnamed direction is not yet a usable control knob. The authors replace vague or inherited labels with a supervised pipeline that asks whether one feature’s activity reliably tracks a real label in data. As to the mechanism, if a feature fires on “alcohol,” and forcing that feature upward makes the model talk about alcohol, the label is no longer just descriptive; it has causal weight. The paper also finds that very high sparsity may not be necessary, meaning the feature does not need to be extremely rare to be useful for steering. Also to note here, both prompting and feature steering are ways to push an LLM toward a desired behavior. Prompting remains stronger because the model was trained to obey prompts, while feature steering is more like pressing directly on the machinery and hoping the rest stays intact. Prompting says “write about alcohol” in the input; feature steering instead turns up the model’s internal “alcohol-related” feature and sees whether the output changes in that direction. ---- Link – arxiv. org/abs/2605.31183 Title: "Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines"

译论文认为稀疏自编码器作为LLM控制工具并非此前认为的那么差，失败源于特征标注方式与模型内部实际因果不匹配。作者提出用监督管道替代模糊标签，验证特征活动是否真实追踪数据标签，使特征具有因果权重。例如，强制“酒精”特征增强可使模型输出转向酒精话题。论文还发现极高稀疏度并非必要。与提示工程相比，提示更强（模型经训练服从提示），而特征控制更像直接拨动机器。

Rohan Paul@rohanpaul_ai · 1天前63

LLM judges can change their safety verdict when the same answer is translated or rewritten. The problem is that many AI teams now use LLMs to judge whether another model’s answer is safe, but safety is not always a simple yes or no question. Those judges can be shaky exactly where careful judgment matters most. The paper proposes a stress test where the same basic answer is shown to judges after translation or rewriting, then the researchers check whether the judges still give the same safety verdict. They are better when harm is obvious, as in violent or extremist content, because the cues are loud and familiar. They become much weaker when safety depends on context, judgment, and regulation, as in financial advice, creditworthiness, or culturally sensitive responses. They also disagreed with each other a lot, and high raw agreement sometimes hid weak real reliability because many judges kept choosing the same label by default. ---- Link – arxiv. org/abs/2605.31381 Title: "LLM Judges Inconsistently Disagree Across Safety Criteria and Harm Categories"

译一项新研究指出，用大语言模型评判其他模型回答是否安全的“LLM安全法官”存在严重不稳定：将相同回答翻译或改写后，法官可能给出不同安全判定。在暴力、极端内容等明显危害场景下表现较好，但在需结合上下文判断的金融建议、信用评估、文化敏感回复等场景中可靠性显著下降。不同法官之间也常出现分歧，高原始一致性有时会掩盖低真实可靠性——因为许多法官默认选择同一标签。论文标题为“LLM Judges Inconsistently Disagree Across Safety Criteria and Harm Categories”。

Rohan Paul@rohanpaul_ai · 1天前67

Today’s frontier agents are far less ready for real-world automation than their benchmark scores suggest. This paper proposes a Agents’ Last Exam, a benchmark that asks AI agents to finish real expert work, and today’s agents mostly fail. Even strong agents of today are nowhere near reliable on the hardest real workflows, which means benchmark success has not yet become broad workplace capability. So this paper shifts the question from “can AI answer hard questions?” to “can AI complete real work that people get paid to do?” Most of today's AI benchmarks show impressive scores, but they do not prove that agents can finish useful work in real jobs. Agents’ Last Exam tries to fix this by testing agents on long tasks from 55 digital work areas, including engineering, finance, medicine, law, media, and science. The tasks come from experts’ real completed projects, and the agent must use normal computer tools like files, browsers, command lines, and desktop software to produce a finished result. The authors tested many current agent systems and models, then scored their finished work with automatic checks or strict rubrics instead of loose human opinions. The main result is that today’s best systems still struggle badly, with an average full pass rate of only 2.6% on the hardest tier. ---- Link – arxiv. org/abs/2606.05405 Title: "Agents' Last Exam"

译一篇新论文提出“Agents’ Last Exam”基准，测试 AI 智能体完成真实专家工作的能力。任务来自工程、金融、医学、法律、媒体、科学等 55 个数字工作领域的实际项目，要求智能体使用文件、浏览器、命令行、桌面软件等常规工具产出可交付成果。评测采用自动检查或严格评分标准。结果显示，当前最强智能体在最难任务层级的平均完全通过率仅 2.6%，远低于其基准测试分数所暗示的水平。论文指出，基准成功尚未转化为广泛的职场能力。

AK@_akhaliq · 2天前53

SCAIL-2 Unifying Controlled Character Animation with End-to-end In-Context Conditioning

译SCAIL-2 统一可控角色动画与端到端上下文条件化

Google DeepMind@GoogleDeepMind · 2天前64

In Sierra Leone, a surging student population is outpacing available teachers. Our latest research explores how AI can act as a partner to support educators in these environments – amplifying their reach without replacing their essential expertise and skills. 🧵

译在塞拉利昂，激增的学生人数正超过可用教师资源。 我们最新的研究探索了AI如何在这些环境中作为合作伙伴支持教育工作者——扩大他们的影响力，同时不取代其核心的专业知识与技能。🧵

elvis@omarsar0 · 3天前60

// Self-Harness: Harnesses That Improve Themselves // (bookmark this one) Most of the agent scaffolds we rely on today are built once and remain frozen or mostly unchanged. The harness, like the skills, needs to evolve with new models. What if the scaffold rewrites itself? This new work treats the harness, the prompts, tools, and control flow around the model as a learnable artifact that improves from its own runs rather than staying a fixed wrapper you hand-maintain. The scaffolding becomes the part that compounds, run after run. If you run long-horizon agents, a self-modifying harness turns scaffold upkeep from manual work into something the system earns on its own. Paper: https://arxiv.org/abs/2606.09498 Learn to build effective AI agents in our academy: https://academy.dair.ai/

译当前多数智能体脚手架（scaffold）构建后保持静态。新研究Self-Harness将harness（提示词、工具、控制流）作为可学习的工件，通过自身运行迭代改进，而非手动维护的固定包装器。运行长周期智能体时，自我修改的harness将维护工作转化为系统自动获得的能力。论文：arxiv.org/abs/2606.09498。

Satya Nadella@satyanadella · 3天前62

Today in @naturemethods, we shared research on how AI can help us better understand cell behavior, offering new insights into why cancer medicines do not work the same for everyone. By learning more about cell state — how individual cancer cells respond to their surroundings — we have the potential to match therapies more precisely to each patient and improve outcomes. https://news.microsoft.com/signal/articles/why-dont-cancer-medicines-work-the-same-for-everyone-ex-vivo/

译今天在《自然方法》上，我们分享了关于AI如何帮助我们更好地理解细胞行为的研究，为癌症药物为何对每个人的效果不同提供了新的见解。 通过学习更多关于细胞状态——单个癌细胞如何响应周围环境——我们有可能更精确地为每位患者匹配疗法并改善结果。https://news.microsoft.com/signal/articles/why-dont-cancer-medicines-work-the-same-for-everyone-ex-vivo/

AK@_akhaliq · 3天前56

SWE-Explore Benchmarking How Coding Agents Explore Repositories

译SWE-Explore 评估编码智能体如何探索仓库

AK@_akhaliq · 3天前57

On the Geometry of On-Policy Distillation

译关于在策略蒸馏的几何

AK@_akhaliq · 3天前66

Latent Spatial Memory for Video World Models

译视频世界模型的潜在空间记忆

Microsoft Research@MSFTResearch · 3天前63

New research in Nature Methods from Project Ex Vivo shows AI models learn more from diverse cell states than from scaled datasets alone, a finding that could reshape how therapies are matched to patients. https://msft.it/6013vgE8l

译在《Nature Methods》上发表的最新研究来自Project Ex Vivo，表明AI模型从多样化的细胞状态中学到的知识，比仅从规模化数据集中学到的更多，这一发现可能重塑疗法与患者的匹配方式。https://msft.it/6013vgE8l

AK@_akhaliq · 3天前51

SpatialWorld Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks

译SpatialWorld 评测多模态智能体在真实世界任务中的交互式空间推理能力

Tencent Hy@TencentHunyuan · 3天前74

🚀Introducing UniRL, an RL infra for unified multimodal models. Together with two new RL algorithms: DRPO and Flow-DPPO. One RL loop across diffusion/flow matching models, LLMs/VLMs, and unified multimodal models👇 Code: http://github.com/Tencent-Hunyuan/UniRL (yes — U(you)-ni-(need) RL 😉)

译🚀推出UniRL，一个用于统一多模态模型的RL基础设施。附带两种新RL算法：DRPO和Flow-DPPO。 一个覆盖扩散/流匹配模型、LLM/VLM以及统一多模态模型的RL循环👇 代码：http://github.com/Tencent-Hunyuan/UniRL （是的——U(you)-ni-(need) RL 😉）

Rohan Paul@rohanpaul_ai · 3天前64

Interesting, this paper shows that Transformers may not need separate key and value projections to work well. This paper's design cut the KV cache by 50% in language modeling with only 3.1% higher perplexity, meaning inference memory fell sharply while prediction quality stayed close. A normal attention layer makes Query to ask what each token needs, Key to label what each token offers, and Value to carry the information sent back. Here, the surprising result is that Key and Value can often share the same learned map, because the model can use one representation both as an address and as the content being retrieved. The best variant, Q-K=V, kept Query separate, so attention still had direction: one token can ask a different token for information instead of every relation becoming mirror-like. When stacked with GQA and MQA, the same idea reached 87.5% and 96.9% cache cuts, because it reduces projection storage while those methods reduce stored heads. The weak variant is Q=K-V, because tying Query and Key makes attention too symmetric for causal language, and it gives no KV-cache savings. ---- Link – arxiv. org/abs/2606.04032v2 Title: "Do Transformers Need Three Projections? Systematic Study of QKV Variants"

译一篇论文系统研究了Transformer注意力中QKV投影的必要性，发现Key和Value可共享同一投影（Q-K=V变体），仅增加3.1%的困惑度，便将KV cache削减50%，大幅降低推理内存。最佳变体保留Query独立，使注意力保持方向性。与GQA和MQA结合时，可分别实现87.5%和96.9%的cache缩减。弱变体Q=K-V因导致因果注意力过于对称且无cache节省而无效。

meng shao@shao__meng · 3天前72

Cognition 推出「FrontierCode」：把 Coding 评估标准，从可用，提升到高质量、可合并！ 评估结果 Top2：Claude Opus 4.8、GPT-5.5 https://cognition.ai/blog/frontier-code FrontierCode 评估内容 规模与结构： · 150 个任务，来自 36 个 flagship 开源仓库 · 20+ 维护者参与，每任务投入 40+ 小时 · 三层嵌套难度：Extended（150）→ Main（100 最难）→ Diamond（50 最难） 两个核心指标： · Pass rate：通过全部 blocker 标准（维护者眼中的 hard stop） · Score：rubric 加权得分；任一 blocker 失败则 score = 0 评测体系：不止 unit test FrontierCode 沿六个维度评估 mergeability： · 行为正确性 — 是否解决问题 · 回归安全 — 是否破坏现有功能 · 机械整洁 — build / lint / style 是否通过 · 测试质量 — agent 写的测试是否真测到行为 · Scope 纪律 — 是否只改该改的 · 代码质量 — 风格、设计模式、可读性、仓库惯例 三种较新的 grading 方法： · Reverse-classical：把 agent 写的测试跑在未修复的base commit 上，必须 fail —— 证明测试有意义 · Scope：文件边界、diff 大小、语义局部性（如是否只改某个函数内） · Adaptive classical grading（mutagent）：用 LLM 微调测试或应用代码，对齐 agent 的实现细节，在保持确定性的同时允许多种合法解法 Criteria 分 blocker（不通过就不能 merge）和 non-blocker（影响 score，但不一票否决）。 评估结果：前沿模型仍远未饱和 · Diamond 子集：Claude Opus 4.8：13.4% score；GPT-5.5：6.3%；Gemini 3.1 Pro：4.7% · Main 子集：Opus 4.8：34.3% · Extended 子集：Opus 4.8：51.8% 几个值得注意的点： · Diamond 几乎未被“刷满” —— 最强模型也只有 13.4%，说明高难度子集仍有大量 headroom · 闭源 vs 开源差距大：最佳开源 Kimi K2.6 在 Diamond 仅 3.8% · 成本 vs 能力：GPT-5.5 分数低于 Opus，但 token 用量约为其 1/4，性价比更优

译Cognition 发布 FrontierCode，含 150 个任务（来自 36 个开源仓库，每任务 40+ 小时），按难度分 Extended/Main/Diamond 三层。沿行为正确性、回归安全等六维度衡量 mergeability，指标为 Pass rate 与 Score。Diamond 子集最高分：Claude Opus 4.8 达 13.4%，GPT-5.5 为 6.3%，Gemini 3.1 Pro 4.7%；Main 子集 Opus 4.8 为 34.3%。开源最佳 Kimi K2.6 仅 3.8%。GPT-5.5 token 用量约为 Opus 四分之一，性价比更优。

Rohan Paul@rohanpaul_ai · 4天前65

AI agent can get better at long tasks without retraining the agent itself, by using a separate small model to clean and organize its context. Moves context management outside the agent, so a separate helper can clean up the task history while the main agent stays unchanged. The paper proposes AdaCoM, which is a separate LLM that edits the agent’s working context before the agent takes its next step. AdaCoM places a separate, trained manager between the task history and the frozen agent, so the agent does not need to learn a new memory habit or expose its weights. Before each step, this manager can rewrite, merge, prune, or preserve parts of the running context, then the original agent acts on the cleaned version. That sounds like summarization, but the distinction matters. A summary assumes the right answer is compression, while AdaCoM learns that different agents need different kinds of context to stay competent, because stronger agents can use more raw history while weaker agents need shorter and cleaner notes. They tested AdaCoM on web search and deep research tasks across several agents, and it improved average web search performance by 39%. ---- Link – arxiv. org/abs/2605.30785 Title: "Learning Agent-Compatible Context Management for Long-Horizon Tasks"

译论文提出 AdaCoM，一个独立的 LLM，在智能体每步操作前编辑其工作上下文。它可重写、合并、剪枝或保留任务历史，使主智能体保持冻结，无需重新训练或暴露权重。与简单摘要不同，AdaCoM 学习不同智能体需要不同类型上下文——强智能体保留更多原始历史，弱智能体需更短更清晰的笔记。在 web search 和 deep research 任务上测试，平均提升 39%。

elvis@omarsar0 · 4天前62

New paper on how AI agents are reshaping knowledge work. This is a nice economic read on where agents actually change knowledge work to meet that gap directly. (bookmark it) It studies agent adoption across three dimensions: autonomy, efficiency, and the scope of tasks workers hand off. The friction people keep hitting with agents is rarely model quality. It is that almost nobody has been taught how to work this way. Paper: https://arxiv.org/abs/2606.07489 Learn to build effective AI agents in our academy: https://academy.dair.ai/

译一篇新论文从自主性、效率和工人移交任务的范围三个维度，分析AI智能体如何重塑知识工作。研究指出，当前人们使用智能体的主要障碍并非模型质量，而是几乎没有人接受过如何以这种方式工作的培训。

Rohan Paul@rohanpaul_ai · 4天前63

This paper proposes a new test to see whether AI agents truly get better as they gain experience and finds they mostly still confuse memory with learning. Shows that simple full-context learning beats the more specialized memory systems, with Claude Sonnet 4.6 using plain context getting the best overall score. That distinction matters because the next wave of AI is not supposed to answer isolated prompts. It is supposed to live inside codebases, databases, markets, sensors, clinics, and workflows where yesterday’s mistake should make tomorrow’s action sharper. The authors build CL-BENCH, a benchmark where an agent works through connected tasks in 6 domains, including coding, databases, forecasting, radio signals, poker, and disease studies. Each task hides a pattern the agent can learn over time, like a database layout, a codebase structure, or an opponent’s strategy, so better performance should come from experience rather than pretraining. They test frontier LLM systems with simple full-context memory, scratchpad notes, retrieval memory, playbook-style memory, and coding-agent setups. The key finding is that current memory-heavy AI agents are not reliably better learners than just keeping the full conversation in context. That means long-running AI agents still need better ways to remember useful lessons, forget stale ones, and adapt when the environment changes. ---- Link – arxiv. org/abs/2606.05661 Title: "Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments"

译新论文构建 CL-BENCH 基准，评估 AI 智能体在编程、数据库、预测、无线电信号、扑克、疾病研究 6 个领域中的持续学习能力。每个任务隐藏可随时间习得的模式，考察智能体能否超越预训练知识。测试前沿 LLM 系统采用全上下文记忆、草稿笔记、检索记忆、剧本式记忆及编码智能体设置，结果发现当前记忆密集型 AI 智能体并未可靠优于简单保留完整对话上下文。Claude Sonnet 4.6 使用普通上下文取得最佳总体分数。论文指出智能体仍需更好方法记住有用经验、遗忘过时信息并适应环境变化。

Perplexity@perplexity_ai · 4天前76

We published new research with Harvard on the shift from chat interfaces to autonomous agents like Computer. Over 3 months, findings show workers using Computer finish tasks in 87% less time at 94% lower cost than Search alone, with higher satisfaction. https://research.perplexity.ai/articles/how-ai-agents-reshape-knowledge-work

译我们与哈佛大学发表新研究，关于从聊天界面转向像Computer这样的自主智能体的转变。 超过3个月的研究结果表明，使用Computer的工人在完成任务上比仅使用搜索快87%，成本低94%，且满意度更高。 https://research.perplexity.ai/articles/how-ai-agents-reshape-knowledge-work

Tencent Hy@TencentHunyuan · 4天前69

Can AI truly edit audio, not just generate it? 🎧 Tencent Hy, in collaboration with SJTU, SII, NTU, TJU, ZODA, PKU, FDU, and other collaborators, introduces MMAE. MMAE--A Massive Multitask Audio Editing Benchmark, is the first comprehensive evaluation benchmark for speech and audio "Banana🍌" Instead of simply requiring the AI to "generate" audio, it demands that the AI understand an existing audio clip and precisely modify it according to natural language instructions—altering what needs to be changed while leaving the rest untouched. Current models show an Exact Match Rate (EMR) below 5%, revealing a major gap in reliable audio editing. MMAE includes: ✅ 2,000 high-fidelity samples from real-world scenarios ✅ 17,741 fine-grained rubric evaluation items ✅ 7 modality settings across sound, music, speech and their mixtures ✅ 6 task complexity from basic modifications to multi-hop reasoning and multi-round editing ✅ 8 operation types across local and global granularities How to use: arXiv: http://arxiv.org/abs/2606.07229 GitHub: https://github.com/ddlBoJack/MMAE HuggingFace: https://huggingface.co/datasets/BoJack/MMAE Demo: https://youtu.be/6At5nTWhlXI

译腾讯混元联合上海交大、南洋理工等机构推出MMAE（Massive Multitask Audio Editing Benchmark），这是首个全面评估AI语音/音频编辑能力的基准。MMAE要求模型理解现有音频并按自然语言指令精确修改，而非简单生成。当前模型在该基准上的精确匹配率（EMR）低于5%，暴露了可靠音频编辑的短板。MMAE包含2000个真实场景高保真样本、17741条细粒度评估项，覆盖声音/音乐/语音及混合共7种模态、6种任务复杂度（基础修改到多跳推理及多轮编辑）、8种操作类型（局部到全局）。论文、代码、数据集和演示已公开。

Rohan Paul@rohanpaul_ai · 4天前60

Great Stanford + MIT + Harvard + Anthropic paper. Gives a clear training-based reason for why larger models learn abilities smaller models miss. Says bigger AI models learn rare skills because they forget them less during training, their extra space protects weak learning signals. The authors say the issue is not just whether a small model could represent the task, but whether training lets it keep that task while many common tasks keep pushing on the same limited parts. Their core idea is that common tasks take up the model’s neurons first, so rare tasks get overwritten before they appear often enough to build into stable knowledge. In a crowded data mixture, common patterns get first claim on the model’s internal machinery. Small models may briefly pick up a rare signal, but the next wave of common-task updates overwrites it before the signal appears again. They tested this first with controlled toy tasks where they could change how rare and complex each task was, then with OLMo language models from 4M to 4B parameters. The main result is that bigger models learned low-frequency tasks much better, kept more task features inside their representations, and showed less gradient interference, which means common-task updates disturbed rare-task learning less. Larger models can remember weak rare signals long enough to turn them into real learned skills. ---- Link – arxiv. org/abs/2605.29548 Title: "Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention"

译该论文指出，更大模型能学到罕见技能，是因为训练中遗忘更少，其额外容量保护了弱学习信号。核心机制：常见任务先抢占神经元，罕见任务在出现频率足够形成稳定知识前就被覆盖。小模型可能短暂捕捉到罕见信号，但随即被下一波常见任务更新覆盖。实验使用OLMo语言模型（4M–4B参数）验证：大模型在低频任务上表现更优，保留更多任务特征，且常见任务更新对罕见任务的梯度干扰更小。作者强调，问题不仅在于小模型能否表征任务，更在于训练中罕见任务能否在众多常见任务反复冲击下持续存在。

Rohan Paul@rohanpaul_ai · 4天前56

Strong AI agents still struggle with long research work because they often fail to keep testing and improving. New Stanford, MIT, NVIDIA, Google and other top labs paper shows shows that today’s strongest research agents win less by brilliance than by refusing to stop testing. The paper proposes AutoLab, a benchmark with 36 tasks where each agent starts from working but weak code and must make it better within a fixed time limit. The tasks cover system speedups, puzzles, model development, and CUDA kernel work, so the test is not just about writing code once but about managing a long work session. The authors tested 17 strong models and found that the best results did not mainly come from the first idea being good, but from the model staying active, testing often, and using feedback well. The best first idea was not the strongest predictor of success; persistence was. Claude Opus 4.6 led the benchmark not because it always guessed the right move immediately, but because it kept benchmarking and folding empirical feedback into the next attempt. Several other frontier models failed in a more revealing way: they either quit early with time left on the clock, or thought so long that they ran out of time before submitting anything useful. ---- Link – arxiv. org/abs/2606.05080 Title: "AutoLab: Can Frontier Models Solve Long-Horizon Auto Research and Engineering Tasks?"

译斯坦福、MIT、英伟达、谷歌等顶级实验室联合提出新基准 AutoLab，包含 36 个任务。每个任务中，智能体从可工作的弱代码起步，需在固定时间内迭代优化。任务涵盖系统加速、谜题、模型开发和 CUDA 内核。17 个前沿模型测试结果显示，成功的关键不是初版方案有多好，而是能否持续测试、频繁实验并利用实证反馈。Claude Opus 4.6 领跑基准，靠的是坚持迭代而非初始判断力，而其他前沿模型要么提前放弃，要么思考过久导致超时。