推荐11
与同批认证绕过漏洞叠加,可把相邻网络访问升级为网关上的代码执行。
Philips Hue Bridge Pro SQL Injection Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-891
ZDI-CAN-33089
CVE ID
CVE-2026-118632
CVSS Score
8.0
AV:A/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H
Affected Vendors
Philips
Affected Products
Hue Bridge Pro
Vulnerability Details
This vulnerability allows network-adjacent attackers to execute arbitrary code on affected installations of Philips Hue Bridge Pro. Although authentication is required to exploit this vulnerability, the existing authentication mechanism can be bypassed.
The specific flaw exists within the HTTP service, which listens on TCP port 443 by default. The issue results from the lack of proper validation of a user-supplied string before using it to construct SQL queries. An attacker can leverage this vulnerability to execute code in the context of root.
Additional Details
Fixed in Bridge Pro firmware version 6.1.2071524030 (or later)
https://www.philips-hue.com/en-ca/support/release-notes/bridge-pro
Disclosure Timeline
2026-08-25
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
Connor Ford @ByteInsight of TrendAI Zero Day Initiative
Back to Advisories
sqlirceiotcve
@zdi-advisories
原文 ↗
SSRF 可借设备身份探测内网,是 IoT 网关横向移动的常见跳板。
(Pwn2Own) Home Assistant Green aiohttp_client Server-Side Request Forgery Vulnerability
November 6th, 2026
ZDI-26-897
ZDI-CAN-28361
CVE ID
CVE-2026-91131
CVSS Score
5.4
AV:A/AC:L/PR:N/UI:N/S:U/C:L/I:L/A:N
Affected Vendors
Home Assistant
Affected Products
Green
Vulnerability Details
This vulnerability allows network-adjacent attackers to initiate arbitrary server-side requests on affected installations of Home Assistant Green. Authentication is not required to exploit this vulnerability.
The specific flaw exists within the aiohttp_client module. The issue results from following HTTP redirects. An attacker can leverage this in conjunction with other vulnerabilities to execute arbitrary code in the context of root.
Additional Details
Home Assistant has issued an update to correct this vulnerability. More details can be found at:
https://github.com/home-assistant/core/security/advisories/GHSA-6456-4qg9-cqpj
Disclosure Timeline
2025-11-05
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
Team Neodyme (@Neodyme)
Back to Advisories
ssrfiotcve
@zdi-advisories
原文 ↗
设计类软件常跨团队传阅模型文件,解析型内存破坏类漏洞利用成本低。
Trimble SketchUp SKP File Parsing Use-After-Free Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-893
ZDI-CAN-30421
CVE ID
CVE-2026-121741
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
Trimble
Affected Products
SketchUp
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of Trimble SketchUp. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of SKP files. The issue results from the lack of validating the existence of an object prior to performing operations on the object. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
Fixed in SketchUp 2027.0
Disclosure Timeline
2026-05-21
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
r1ngz3ro
Back to Advisories
uafrce文件解析cve
@zdi-advisories
原文 ↗
这是链路中的一环,需先有容器内执行权限,单独利用价值有限。
Philips Hue Bridge Pro Argument Injection Restricted File Write Vulnerability
November 6th, 2026
ZDI-26-890
ZDI-CAN-33090
CVE ID
CVE-2026-118631
CVSS Score
3.8
AV:L/AC:L/PR:L/UI:N/S:C/C:N/I:L/A:N
Affected Vendors
Philips
Affected Products
Hue Bridge Pro
Vulnerability Details
This vulnerability allows local attackers to modify files on affected installations of Philips Hue Bridge Pro. An attacker must first obtain the ability to execute low-privileged code within a container on the target system, in order to exploit this vulnerability.
The specific flaw exists within the factory data service, which listens on TCP port 5001 by default. The issue results from the lack of proper validation of a user-supplied string before using it to execute a system call. An attacker can leverage this vulnerability to modify files in the context of the service account.
Additional Details
Fixed in Bridge Pro firmware version 6.1.2071524030 (or later)
https://www.philips-hue.com/en-ca/support/release-notes/bridge-pro
Disclosure Timeline
2026-08-27
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
Connor Ford @ByteInsight of TrendAI Zero Day Initiative
Back to Advisories
参数注入iot容器cve
@zdi-advisories
原文 ↗
科研数据流转常用文件格式,打开他人发送的工程文件即触发,属于典型钓鱼载荷面。
OriginLab Origin Viewer OPJ File Parsing Heap-based Buffer Overflow Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-885
ZDI-CAN-31587
CVE ID
CVE-2026-114568
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
OriginLab
Affected Products
Origin Viewer
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of OriginLab Origin Viewer. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of OPJ files. The issue results from the lack of proper validation of the length of user-supplied data prior to copying it to a heap-based buffer. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
OriginLab has issued an update to correct this vulnerability. More details can be found at:
https://docs.originlab.com/cve/CVE-2026-114568
Disclosure Timeline
2026-06-24
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
chwrld (@chwrld24)
Back to Advisories
rcebuffer_overflowcve文件解析
@zdi-advisories
原文 ↗
与堆溢出同类但成因不同,栈破坏更易构造稳定利用,值得连同本批公告一起评估。
OriginLab Origin Viewer OGWU File Parsing Stack-based Buffer Overflow Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-880
ZDI-CAN-30737
CVE ID
CVE-2026-114563
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
OriginLab
Affected Products
Origin Viewer
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of OriginLab Origin Viewer. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of OGWU files. The issue results from the lack of proper validation of the length of user-supplied data prior to copying it to a stack-based buffer. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
OriginLab has issued an update to correct this vulnerability. More details can be found at:
https://docs.originlab.com/cve/CVE-2026-114563
Disclosure Timeline
2026-05-21
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
rgod
Back to Advisories
rcebuffer_overflowcve文件解析
@zdi-advisories
原文 ↗
CVSS 7.8,仅需打开恶意 OPJ 文件,同产品近期连出三条解析漏洞。
OriginLab Origin Viewer OPJ File Parsing Out-Of-Bounds Write Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-877
ZDI-CAN-30432
CVE ID
CVE-2026-114560
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
OriginLab
Affected Products
Origin Viewer
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of OriginLab Origin Viewer. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of OPJ files. The issue results from the lack of proper validation of user-supplied data, which can result in a write past the end of an allocated data structure. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
OriginLab has issued an update to correct this vulnerability. More details can be found at:
https://docs.originlab.com/cve/CVE-2026-114560
Disclosure Timeline
2026-05-21
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
rgod
Back to Advisories
cverce文件解析
@zdi-advisories
原文 ↗
数百条失败轨迹归纳出六条 Agent 工程约束,可直接借鉴。
SemiAnalysis 的 GDPx 评测把主流大模型放进云端真实工位连跑 8 小时解决 206 个业务难题,最强模型平均仅得约 12%。数百次失败轨迹揭示了文档权威度地图、样本外回滚门禁、确定性测量脚本、外部排障证据账本、次旗舰总账路由与物理白名单加挂起出口六条工程经验。
ai_agentllm报告
@yage-share
原文 ↗
记忆组件设计直接决定 Agent 的上下文污染与越权边界。
Søren Kierkegaard wrote that life can only be understood
backward
, but it must be lived
forward
. We’ve encountered a similar bind when designing memory for AI agents.
As humans, we encode memory constantly as we go about our lives.
The hippocampus
plays a central role in this, capturing experiences in the moment so that memories form quickly and are immediately available. But only
some experiences
are consolidated into long-term knowledge.
Much of that consolidation appears to happen when our brains are offline. As we sleep, and during quiet
rest
, the hippocampus replays our recent experiences. This is thought to gradually distill
those experiences and integrate them
into the neocortex—the part of our brains that stores most of what we think of as knowledge.
Agent memory can use a similar division of labor. Agents capture much of their work in logs or transcripts. Researchers at Letta and UC Berkeley proposed
sleep-time compute
as a way for models to use idle time between interactions to reorganize their existing context. The same idea can help agents organize their memory by reflecting on logs to fix mistakes or produce new notes and representations. This process is often
called
dreaming
(Figure 1).
We’ve found that dreaming has been a useful tool for helping us give Claude effective memory. In this post, we share how we’ve come to think about agent memory and walk through several evaluations we’ve used to explore agent memory, including the use of dreaming, and what each component contributes to memory organization.
Claude can use dreaming to organize memory. This can correct mistakes, create notes, build new memory representations, or shape what’s preemptively loaded into future sessions.
Memory notes
By “agent memory,” we mean knowledge that Claude gains from experience—including interactions with users and tasks it’s been assigned—and that it retains between sessions, saved as context it can read later. It excludes other ways models can learn or manage context, such as changes to their
weights
(the internal parameters set during training),
in-context learning
(knowledge gained within a single conversation), and
compaction
(summaries written by Claude that replace its older turns so it can continue working in a new context window).
In the human brain, chemical signals released before, during, or after an experience can
mark a memory
as important. Marked memories tend to be replayed more often than unmarked ones during offline periods. This process, called
replay
, helps stabilize and integrate them into long-term memory.
Similarly, Claude can tag salient events in a session log as they happen, so they’re easier to find later, or it can
write
them to a memory store, such as a folder of memory notes (Figure 2). But how can Claude tell what’s salient in the moment, before it knows what’s coming next?
Claude can write notes to memory.
Memory can be a prediction about what the future will need
Memory is, occasionally, a prediction. By writing down a rule, an agent predicts that the pattern it’s describing will be relevant again. For instance, writing down a verified fact is a prediction that an agent in a future session will want to rely on that fact without double-checking it. Over time, Claude has become much better at making these kinds of predictions.
Consider a task from
Continual Learning Bench (CL-Bench) 1.0
, a benchmark that measures an agent’s ability to use memory in order to tackle long-horizon tasks. One particular task asks Claude to answer a series of questions by looking up information in a database. After Claude gives an answer, it’s told the correct solution and can write a memory to help answer future questions.
Memory can support a learning loop here. If Claude gets a question wrong, it can record the failure, come up with theories about what went wrong, test those theories, and turn what it finds into a rule about how to answer in the future.
We ran the CL-Bench 1.0 task using
Claude Managed Agents
. Claude started each question in a fresh session, with one exception: it could retrieve files from a
memory directory
that persisted from one session to the next.
Different models achieved different levels of success at creating their own learning loops. Claude Sonnet 4.6 got as far as generating theories about why an answer was wrong. Its memory directory contained a scattering of files, each listing several untested guesses.
CL-Bench database task, comparing memory with a stateless baseline across models.
Claude Opus 4.7 consolidated its notes into a single reference document and began marking which facts it had checked and which were still unconfirmed. Claude Sonnet 5 marked each answer as confirmed after the grader had checked it, and wrote a correction when it had answered wrong.
Claude Opus 5 was the first of these models to close the loop. Most of its entries included a rule (and also how it knew the rule was true), answers to earlier questions, and clear warnings about traps it had discovered along the way (Figure 3).
Like Opus 5, Claude Opus 5.5 and Claude Sonnet 5.5 both closed the loop, using memory to capture validated rules about how to best answer future questions. With memory, both scored about the same as Opus 5, though their baseline performance without memory was slightly higher.
Claude has learned to judge salience
We also evaluated Claude’s ability to play
Pokémon Red
, a game that
challenges Claude to navigate
complex long-horizon tasks, as a test of Claude’s judgment about
what
to save to memory. At the start of each run, we gave each model a folder of memory files that it could read from and write to while it played (Figure 4).
We compared the memory folders from one run each of Claude Sonnet 3.7 and Claude Fable 5, when each was about 10,000 steps into the game. A key difference we noticed was what each model thought was worth remembering. Sonnet 3.7 assumed everything might matter and wrote to memory indiscriminately. Its memory folder contained an archive of encounters, often with a separate file for each battle it fought, each item it found, and each non-player character it met. Ultimately, Sonnet 3.7 wrote 413 memory files across 225 folders and earned one badge (badges mark a player’s progress through the game).
Fable 5, by comparison, wrote nine files in two folders, plus a single top-level file called
global.md
, and earned four badges.
Claude’s memory directories for Sonnet 3.7 versus Fable 5 while playing Pokémon.
Sonnet 3.7’s approach was a problem mainly because the game state of Pokémon changes constantly—Sonnet 3.7’s detailed record was continuously going out of date. As a result, as Claude progressed, it had to spend more time and resources combing through its memories rather than playing the game. Around step 10,000, for example, Sonnet 3.7 started its fourth folder of “master” memory files. Over the next 17 steps, it created six files, deleted nine others, and took no actions in the game itself.
Fable 5’s approach used memory for things the game didn’t explicitly tell it. Some of the kinds of information Sonnet 3.7 wrote to memory—such as how much money the player has or where the player is on the map—can be read off the screen every turn, so storing it is redundant. By contrast, memory files about strategy, plans, and tested rules exist only if Claude works them out, and they disappear unless Claude writes them down.
Fable 5 organized its memory accordingly. Its
global.md
file acted as a dashboard, holding the current goal, a prewritten plan for tackling gyms (the challenges where badges are won), and the model’s own rules for using memory. A
locations
folder held plans for navigating particular routes, and a
gameplay
folder held strategies based on what it had learned about how the game works. For example, after seven Poké Balls in a row failed to catch a Magikarp (one of the easiest Pokémon to catch), Claude wrote, “If 3+ balls fail in a row, VARY the timing (insert extra ‘wait’ presses between menu steps) - the catch landed right after doing this.”
With a much smaller memory folder, Fable 5 progressed much further in the game than Sonnet 3.7. Opus 5.5 took a similar approach: it kept a small folder of memory notes that tracked its current goal and next steps, rewrote them as it played, and beat the game.
Steering Claude to write memory
We’ve found that it helps to tell Claude what memory is for, but not
how
to use it. On CL-Bench, we simply told Claude that a memory folder was available. That general guidance was enough for Opus 4.7 and Opus 5 to use memory effectively, and it gave both a noticeable boost in performance (Figure 3).
We also tested giving the models specific rules about what they should write to memory, with one rule per test. We compared how much each model gained from memory with each rule against how much it gained without it. When we set a rule to “record only failures,” both Opus 5 and Opus 4.7 lost much of the performance boost that memory gave them (Figure 5, right). For stronger models, giving them freedom to judge salience was more effective than setting fixed rules in advance, and those rules could end up weakening performance.
Left: Sonnet 4.6 on the CL-Bench database task with and without prompting to guide memory usage. Right: how much each model’s gain from memory changed when it was told to record only failures.
We also noticed that steering helped older models the most. For instance, Sonnet 4.6 often didn’t read files from its memory directory and failed to test its theories about why an answer was wrong.
In this case, prompting Claude to read its memory files before answering—and then to explicitly test its theories—nearly doubled its benchmark score (Figure 5, left). Sonnet 5 does this on its own. Without being prompted to, it read its memory before answering 9 out of 10 times. By comparison, Sonnet 4.6 did this less than half of the time.
With frontier Claude models, we typically give simple guidance to create memory notes when a person asks them to or when something is clearly worth remembering, and we leave that judgment up to Claude.
Dreaming can correct memory
Since Claude writes to memory based on what it judges to be salient in-session, a note that’s wrong from the start or that drifts as the environment changes can be read, and trusted, by Claude in a later session.
Dreaming gives Claude a way to test its notes against the original transcripts (Figure 6). We saw a positive example of this when Claude played Pokémon using Managed Agents with a mounted directory for its memories.
Dreaming can correct mistakes in memory.
In one part of the game, the player has to get through Mt. Moon, a cave system with its entrance on the west side of the map and its exit on the east. Near the entrance is a trap: a one-way ledge that drops the player onto the route they came from, with no clear way to climb back up. Claude was using the PokéCenter, a building at the mountain’s entrance, as a landmark, so whether it walked off the ledge depended on which side of the mountain it thought the PokéCenter was on.
At one point, Claude wrote down (incorrectly) that the PokéCenter was on the east side. When it walked out of the PokéCenter, it announced, “Mt Moon is behind me,” and updated its notes: “Mt Moon DONE.” But Mt. Moon was not behind it. A few actions later, Claude went over the ledge and landed back on the route it had come from (Figure 7). Its navigation tool didn’t show the drop, so Claude concluded, “I overshot far west,” and that it had been on the correct side all along. It hadn’t.
A corrected memory note changed the outcome in Pokémon. With the uncorrected note, Claude says, “Mt Moon is behind me,” heads for the next city, and ends up back on the route it came from. With the note corrected by dreaming, it says, “Now I need to find the Mt Moon cave entrance to re-enter,” and goes back into the cave.
We then tested whether dreaming could help Claude avoid this trap. We ran dreaming on the memory folder that contained the wrong location for the PokéCenter: Claude checked the note against the session transcript, caught its error, and rewrote it. The transcript gave Claude a ground truth to validate its memory against. With the corrected memory note, Claude went straight back into Mt. Moon and eventually reached Cerulean City.
In the run with the uncorrected note, Claude dropped back onto that earlier route three times. After the third failure, Claude seemed to give up, writing, “I’m going to stop rather than keep looping.” Our setup’s automatic reply instructed it to continue. So it did, all the way back to Pewter City, the town before the mountain, where it hoped to heal and restock. The run ended without Claude ever reaching Cerulean City.
Dreaming corrects the memory, and Claude goes back into the cave and reaches the next city. The run with the uncorrected note (“Raw memory”) never gets past the mountain, and neither does the run with no memory.
Dreaming can build new memory representations
With dreaming, Claude can build representations to help navigate memory (Figure 9). One type of memory representation it uses is called a
wiki
, a collection of pages written and maintained by Claude that gathers what Claude knows about one subject and links it to other notes. These links effectively provide Claude with a map it can use to sift through its memory and efficiently locate memories when they’re needed.
Dreaming can build memory representations.
Representations like wikis can help Claude traverse its ever-growing knowledge base. We explored the benefit of wikis as a memory representation using a corpus of internal documents—about 350,000 files, roughly 420M tokens—from Stainless, a software company we acquired in May. We created an evaluation set of 25 questions about this corpus and gave Claude a search tool to explore it. We also instructed Claude to build a wiki based on these files using dreaming. Over 128 sessions, we prompted Opus 5 agents to explore the files, write or correct linked wiki pages, merge duplicates, and check claims against the sources.
Each session built on the progress that agents made in prior sessions, using a shared log to coordinate work. As we scaled dreaming from two sessions to 128, performance on the 25 questions grew from 13% to 48% of graded criteria (Figure 10).
Claude builds its wikis incrementally. After creating an initial wiki, it uses subsequent rounds of dreaming to add to, edit, or prune it. In the early rounds of dreaming, Claude’s knowledge is like that of a new employee, and its ability to answer questions based on the wiki alone is limited. The more it dreams, the richer its knowledge map grows, like that of a seasoned employee. In our tests, we found that the wiki helped Claude the most when we also gave it access to a search tool. When it used both search and the wiki, Claude’s aggregate score rose from 54% to 61% as that wiki grew, compared with 53% when it had access to the search tool alone. Claude’s best score was 48% when it used only the wiki to answer questions. Using search and the wiki also saved about 20,000 tokens per question.
Center: scores on 25 questions about a corpus of documents from a company as Claude built wikis over sequential sessions. Left and right: an example wiki after two and 128 sessions; each dot is a page, and lines join linked pages.
We wanted to test which kinds of tasks benefited from Claude having a wiki. To do this, we used the
Harvey LAB
benchmark along with a collection of documents about a merger. The benchmark gives Claude a set of documents and asks Claude to generate a legal brief based on their content. The benchmark then grades the brief against a list of criteria. While a tad less engaging, perhaps, than a Pokémon run, these evaluations give us diverse collections of documents to build wikis from.
We instructed Claude to build a wiki from the documents. We then asked Claude to answer questions (about the merger) or generate legal briefs (the LAB benchmark) both from the wiki and from a direct file search. One test asked 24 questions about the merger (Figure 11, left), more than half of which needed Claude to find information across several documents, and many of which also needed Claude to use judgment or do math. On those items (red and orange lines), Claude scored 87% when it consulted the wiki, compared with 73% when it used search alone.
Each panel lists what was graded (left) and the source documents needed to meet each one (right). Left: 24 questions about a merger, which can be answered by looking at mostly public filings. More than half of the criteria required combining information from several documents. Right: a LAB task where the wiki showed no clear gain. Most criteria could be met from a single document.
In other cases, the wiki was less useful. On one LAB task (Figure 11, right), brief criteria often could be retrieved with a text search from a single document, and the wiki showed no clear gain in performance relative to search. On another LAB task, the wiki grew larger than the documents themselves (Figure 12). Like the mapmakers in Borges’s one-paragraph short story “On Exactitude in Science,” who drafted a map so detailed it was as large as the empire it represented, Claude had made an extremely detailed wiki that effectively abstracted nothing.
The wiki grew to be larger than the raw files.
Dreaming can promote notes up the memory hierarchy
Dreaming can also float notes to the top of the memory hierarchy, and the agent harness can preemptively load them into Claude’s context at the start of every session (Figure 13). This means Claude doesn’t need to search for them, or even to know there’s something to search for.
Promoting memories into a new session (a form of
context injection
) must be done carefully. Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on
Claude Code
warns that longer instruction files reduce adherence. So the top layer of the memory hierarchy has to stay relatively small, and much of dreaming’s work here is keeping it that way—by moving some details down the hierarchy and leaving links behind.
Claude Code’s auto memory
keeps its injected layer small in the same way, with Claude maintaining it during the session. Claude doesn’t load all of its auto memory into every session’s context window. It loads only the first 200 lines or 25 KB of
MEMORY.md
. This header serves as an index, which shows Claude what it knows and where to find it.
Dreaming can write to the top of the memory hierarchy. Injected context is loaded into every session, and the layers below it are read or searched when Claude needs them.
Dreaming can focus memory on a specific goal
Dreaming can also shape how memory grows. An assistant like
Claude Duet
, for example, often benefits from having memories that are organized around the person it’s helping, and dreaming can be tuned to build memory in service of a particular goal.
The instructions that guide how memory grows differ by task and tell Claude what information actually matters. For example, one task in the LAB benchmark simulates a company acquisition and provides a set of contracts. The brief is graded on whether it correctly describes the contract terms.
We started with basic instructions for generating the wiki that didn’t tell Claude
how
to use the wiki. This task relied heavily on contract terms. While the wiki was being built, several agents wrote to a shared table of contract terms at the same time (Figure 14, left).
How many graded contract facts made it into the wiki, before and after instructions changed. Left: several agents writing to one shared page; 41 of 57 facts survived. Right: one agent per contract, each writing its own page and quoting terms word for word; 57 of 57 facts survived.
The result was a wiki that was missing key facts. When we graded a brief generated from the wiki, Claude had included only 41 of the 57 facts (72%). When we looked at what had gone wrong, we found that the shared table summarized the contract terms poorly and was missing some of them altogether, because agents had overwritten one another’s work.
The contract terms were the most important part of the wiki, but the instructions didn’t say that. To find better instructions, we ran a loop in which Claude reflected on its failures, proposed a change to the instructions, and rebuilt the wiki. This loop allowed Claude to discover how important it was to follow the contract terms, and it rewrote the instructions for building the wiki accordingly.
Each contract got its own page in the wiki, written by a single agent, which quoted the terms word for word rather than summarizing them into a shared table. When we graded the brief generated from this improved wiki, 100% of the criteria could be met, up from 72%.
Dreaming can fill in gaps a session leaves behind
When Claude writes memory notes, it sees only the context in
that
particular session. Dreaming can search across many sessions to find common patterns or correct common mistakes.
When building the wikis for the LAB benchmark (as an example, see Figure 14), sometimes Claude would link to pages that hadn’t yet been created. A second dreaming pass closed many of these gaps. On one task (Figure 15), dreaming found nine links to a missing page in the wiki and repointed them to an existing page for the same document. It also added three new pages, including an index, to improve the wiki.
The wiki before and after a second dreaming pass.
Thinking backward, looking forward
There are many open questions about how to design memory systems for agents. One is when Claude should create memory. When Claude works on a task, it can write to a memory store (e.g., a folder, file, or database), capturing salience at the source and making important details immediately available for future turns or sessions. But logging notes in the moment inevitably draws Claude’s attention away from its main task. In contrast, dreaming can push memory creation to the background, but any outputs from dreaming may not be immediately accessible to Claude.
Another question is how to keep memory reliably up to date. Claude can make incorrect predictions or write rules to memory that are wrong from the start. Memory can also drift and become irrelevant as the environment changes. Dreaming is one way to fix these mistakes (Figure 8), but it requires compute. Directing Claude to continuously check
all
its memories might not be practical. It may be important instead to focus dreaming on information that’s likely to change or to carefully tune the cadence of dreaming (e.g., based on how often memories are accessed).
It’s also reasonable to consider transcript search as the foundation of Claude’s memory. Combining representations (e.g., wikis) with search (Figure 10) can sometimes be effective. But Claude needs to use compute to maintain memory representations and make sure they don’t accumulate errors or hallucinations.
Even though Claude’s judgment about salience has improved, no model can really know what will happen in future turns. Dreaming gives Claude an opportunity to look
backward
and use compute to organize its memory, correct mistakes, and maintain representations that allow future sessions to access memory more readily. Memory systems will evolve as Claude gets more capable, and we still have a lot to learn about designing Claude’s memory systems.
See our cookbooks for
getting started with dreaming
and
knowledge wikis
.
Authors & acknowledgements
Acknowledgements
Written by Lance Martin with editorial guidance from Caitlin Love, Tom Chivers, Rebecca Hiscott, Jake Eaton, and Blake Wilson. We’re grateful to Barry Zhang, Theodora Chu, Daliang Li, Mahesh Murag, Jake Noble, Ravi Trivedi, Tina Vachovsky, Brad Abrams, Henry de Valence, Jeremy Fox, Matt Bell, Allen Li, Chris Gorgolewski, Eugene Yan, Artur Renault, Wei Zhang, and Kevin Chen for helpful discussions and contributions, and to David Hershey for Claude Plays Pokémon. Figures by Monika Tuchowska.
ai_agentllm架构
@anthropic-research
原文 ↗
对全库锁和动态打包瓶颈的架构取舍做出了拆解,对自建代码托管平台有参考价值。
相隔不到三周,Anthropic 花 5 个月抛弃 Git 重造了 Scion,而 OpenAI 只用 10 周重写服务端 Pando,连日常 Git 命令都没改就把推送延迟砍掉八成、扛住每天近 1 PB 流量。拆解全库锁与动态打包两副旧脚镣,看团队知识基因如何决定架构取舍。
云基础设施工程实践git
@yage-share
原文 ↗
多窗口调度与群聊失控的实践教训,对自研编码 Agent 的架构选型有直接借鉴。
DeepSeek 开源的 3 万行常驻编程代理原型抓准了多窗口调度和群聊失控两个真痛点,却用上一代软件工程补丁叠补丁的状态机去替模型短板兜底。换上指令遵循强的旗舰模型,只需保留薄薄一层消息分流,再直接抄走四条保护人类注意力的轻量规矩。
ai_agentllm工程实践
@yage-share
原文 ↗
更多15
未认证远程即可打停备份服务,但仅造成可用性影响。
Quest NetVault Backup nvpmgr Integer Underflow Denial-of-Service Vulnerability
November 6th, 2026
ZDI-26-889
ZDI-CAN-28744
CVE ID
CVE-2026-13270
CVSS Score
7.5
AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H
Affected Vendors
Quest
Affected Products
NetVault Backup
Vulnerability Details
This vulnerability allows remote attackers to create a denial-of-service condition on affected installations of Quest NetVault Backup. Authentication is not required to exploit this vulnerability.
The specific flaw exists within the nvpmgr service, which listens on UDP port 20031 by default. The issue results from the lack of proper validation of user-supplied data, which can result in an integer underflow before allocating a buffer. An attacker can leverage this vulnerability to create a denial-of-service condition on the system.
Additional Details
Fixed in NetVault version 14.1.3
https://support.quest.com/technical-documents/netvault/14.1.3/release-notes/2#TOPIC-2383658
Disclosure Timeline
2026-04-23
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
06fe5fd2bc53027c4a3b7e395af0b850e7b8a044
Back to Advisories
dos备份cve
@zdi-advisories
原文 ↗
与同厂商另一条 DoS 漏洞根因类似,增量信息有限。
Quest NetVault Backup StnzCreateKey Numeric Truncation Error Denial-of-Service Vulnerability
November 6th, 2026
ZDI-26-888
ZDI-CAN-28743
CVE ID
CVE-2026-13269
CVSS Score
7.5
AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H
Affected Vendors
Quest
Affected Products
NetVault Backup
Vulnerability Details
This vulnerability allows remote attackers to create a denial-of-service condition on affected installations of Quest NetVault Backup. Authentication is not required to exploit this vulnerability.
The specific flaw exists within the StnzCreateKey function. The issue results from the lack of proper validation of user-supplied data, which can result in a numeric truncation error before allocating a buffer. An attacker can leverage this vulnerability to create a denial-of-service condition on the system.
Additional Details
Fixed in NetVault version 14.1.3
https://support.quest.com/technical-documents/netvault/14.1.3/release-notes/2#TOPIC-2383658
Disclosure Timeline
2026-04-23
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
06fe5fd2bc53027c4a3b7e395af0b850e7b8a044
Back to Advisories
dos备份cve
@zdi-advisories
原文 ↗
OriginLab Origin Viewer OPJU File Parsing Out-Of-Bounds Write Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-886
ZDI-CAN-30738
CVE ID
CVE-2026-114558
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
OriginLab
Affected Products
Origin Viewer
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of OriginLab Origin Viewer. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of OPJU files. The issue results from the lack of proper validation of user-supplied data, which can result in a write past the end of an allocated data structure. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
OriginLab has issued an update to correct this vulnerability. More details can be found at:
https://docs.originlab.com/cve/CVE-2026-114558
Disclosure Timeline
2026-05-21
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
rgod
Back to Advisories
rce越界写cve
@zdi-advisories
原文 ↗
OriginLab Origin Viewer OGWU File Parsing Out-Of-Bounds Write Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-884
ZDI-CAN-30771
CVE ID
CVE-2026-114567
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
OriginLab
Affected Products
Origin Viewer
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of OriginLab Origin Viewer. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of OGWU files. The issue results from the lack of proper validation of user-supplied data, which can result in a write past the end of an allocated data structure. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
OriginLab has issued an update to correct this vulnerability. More details can be found at:
https://docs.originlab.com/cve/CVE-2026-114567
Disclosure Timeline
2026-06-02
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
rgod
Back to Advisories
rce越界写cve
@zdi-advisories
原文 ↗
OriginLab Origin Viewer OGM File Parsing Memory Corruption Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-883
ZDI-CAN-30770
CVE ID
CVE-2026-114566
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
OriginLab
Affected Products
Origin Viewer
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of OriginLab Origin Viewer. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of OGM files. The issue results from the lack of proper validation of user-supplied data, which can result in a memory corruption condition. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
OriginLab has issued an update to correct this vulnerability. More details can be found at:
https://docs.originlab.com/cve/CVE-2026-114566
Disclosure Timeline
2026-06-02
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
rgod
Back to Advisories
rce内存破坏cve
@zdi-advisories
原文 ↗
OriginLab Origin Viewer OGG File Parsing Out-Of-Bounds Write Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-882
ZDI-CAN-30769
CVE ID
CVE-2026-114565
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
OriginLab
Affected Products
Origin Viewer
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of OriginLab Origin Viewer. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of OGG files. The issue results from the lack of proper validation of user-supplied data, which can result in a write past the end of an allocated data structure. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
OriginLab has issued an update to correct this vulnerability. More details can be found at:
https://docs.originlab.com/cve/CVE-2026-114565
Disclosure Timeline
2026-05-21
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
rgod
Back to Advisories
rce越界写cve
@zdi-advisories
原文 ↗
OriginLab Origin Viewer OGG File Parsing Out-Of-Bounds Write Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-881
ZDI-CAN-30768
CVE ID
CVE-2026-114564
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
OriginLab
Affected Products
Origin Viewer
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of OriginLab Origin Viewer. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of OGG files. The issue results from the lack of proper validation of user-supplied data, which can result in a write past the end of an allocated data structure. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
OriginLab has issued an update to correct this vulnerability. More details can be found at:
https://docs.originlab.com/cve/CVE-2026-114564
Disclosure Timeline
2026-05-21
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
rgod
Back to Advisories
rce越界写cve
@zdi-advisories
原文 ↗
OriginLab Origin Viewer OGWU File Parsing Out-Of-Bounds Write Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-879
ZDI-CAN-30736
CVE ID
CVE-2026-114562
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
OriginLab
Affected Products
Origin Viewer
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of OriginLab Origin Viewer. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of OGWU files. The issue results from the lack of proper validation of user-supplied data, which can result in a write before the start of an allocated data structure. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
OriginLab has issued an update to correct this vulnerability. More details can be found at:
https://docs.originlab.com/cve/CVE-2026-114562
Disclosure Timeline
2026-05-21
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
rgod
Back to Advisories
rce越界写cve
@zdi-advisories
原文 ↗
OriginLab Origin Viewer OPJU File Parsing Out-Of-Bounds Write Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-878
ZDI-CAN-30433
CVE ID
CVE-2026-114561
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
OriginLab
Affected Products
Origin Viewer
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of OriginLab Origin Viewer. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of OPJU files. The issue results from the lack of proper validation of user-supplied data, which can result in a write past the end of an allocated data structure. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
OriginLab has issued an update to correct this vulnerability. More details can be found at:
https://docs.originlab.com/cve/CVE-2026-114561
Disclosure Timeline
2026-05-21
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
rgod
Back to Advisories
rce越界写cve
@zdi-advisories
原文 ↗
OriginLab Origin Viewer OGG File Parsing Out-Of-Bounds Write Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-876
ZDI-CAN-30431
CVE ID
CVE-2026-114559
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
OriginLab
Affected Products
Origin Viewer
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of OriginLab Origin Viewer. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of OGG files. The issue results from the lack of proper validation of user-supplied data, which can result in a write past the end of an allocated data structure. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
OriginLab has issued an update to correct this vulnerability. More details can be found at:
https://docs.originlab.com/cve/CVE-2026-114559
Disclosure Timeline
2026-05-21
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
rgod
Back to Advisories
cverce文件解析
@zdi-advisories
原文 ↗
OriginLab Origin Viewer OGWU File Parsing Heap-based Buffer Overflow Remote Code Execution Vulnerability
November 6th, 2026
ZDI-26-875
ZDI-CAN-30430
CVE ID
CVE-2026-114569
CVSS Score
7.8
AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H
Affected Vendors
OriginLab
Affected Products
Origin Viewer
Vulnerability Details
This vulnerability allows remote attackers to execute arbitrary code on affected installations of OriginLab Origin Viewer. User interaction is required to exploit this vulnerability in that the target must visit a malicious page or open a malicious file.
The specific flaw exists within the parsing of OGWU files. The issue results from the lack of proper validation of the length of user-supplied data prior to copying it to a heap-based buffer. An attacker can leverage this vulnerability to execute code in the context of the current process.
Additional Details
OriginLab has issued an update to correct this vulnerability. More details can be found at:
https://docs.originlab.com/cve/CVE-2026-114569
Disclosure Timeline
2026-05-21
- Vulnerability reported to vendor
2026-11-06
- Coordinated public release of advisory
2026-11-06
- Advisory Updated
Credit
rgod
Back to Advisories
cverce文件解析
@zdi-advisories
原文 ↗
前言
本文主要介绍个人 Agent 框架 cc_agent
的设计与实现,包括用户管理、会话管理、模型配置、Skill、MCP
工具调用以及完整的对话流程。
姑且将 AI 系列后续内容分为以下 4 个部分:
学术篇:
LLM
基础知识、模型训练、各类算法的原理与实现等。
开发篇:
Agent 框架、各类 AI 应用的开发等。
运维篇:
Agent 应用的部署、推理参数测试等。
杂项:
AI
渗透、视频生成、音频生成、超分辨率等其他内容。
前天已经完成了 Agent 框架的全部基础组件,不过目前还在开发类似 Voyager
的自学习 Skill
功能。考虑到博客已经有一段时间没有更新了,就先整理一下目前的开发成果。
这次主要介绍自己开发的 Agent 框架。
首先需要明确一点:Agent 并不是简单地接入一个
LLM,再封装一层输入输出就能实现的。LLM 只是 Agent
的一部分,可以把它理解为 Agent
的「大脑」,负责理解、思考和决策,而其他组件则负责提供必要的功能和支持。
以人为例,人由大脑、眼睛、耳朵、四肢等多个部分组成,各个部分相互配合才能构成一个完整的人。Agent
也是如此,需要多个组件协同工作,才能完成完整的任务流程。
因此,这篇文章主要介绍如何构建 Agent
的各个组件,以及如何将它们组织起来,最终组成一个完整的
Agent。具体实现会在正文中逐一介绍。
开发 Agent,本质上也是在开发 Agent 框架。
那么,什么是 Agent 框架?简单来说,就是一套用于组织 Agent
组件、管理组件之间交互,以及实现任务执行流程的方法和规则。
既然市面上已经有这么多成熟的框架,为什么还要自己开发?
主要有三个原因:
学习目的:阅读源码固然能够了解框架的实现原理,但很多东西只有自己动手实现一遍,才能真正理解其中的设计思路和组件之间的关系。
满足自己的需求:现有框架不一定完全符合自己的使用习惯和开发需求。可能缺少某些功能,也可能在扩展已有功能时受到框架本身的限制。自己开发则可以根据实际需求灵活调整。
方便后续复用:后续还会尝试开发各种 Agent
功能和相关项目。如果每次都从头实现一遍基础组件,显然没有必要。拥有一套自己的框架,就可以直接复用已有功能,在此基础上进行扩展。
cc_agent 代码仓库:https://github.com/zusda/cc_agent/tree/main
1. 框架概览
自己开发的框架总觉得应该起个名字,否则连导入包的时候都不知道该叫什么。
这里将其命名为 cc_agent。
cc_agent 是一个单 Agent
框架,以用户为隔离单位。每个用户拥有独立的会话(session)、模型配置(model_cfg)和技能(skill)。
工具统一添加到模型配置中,并通过 MCP 服务提供和调用。
源码目录(包名
cc_agent
):
config.py
:全局目录与窗口参数,包括
USER_DIR
、
MODEL_CFG_DIR
、
SKILL_DIR
、
MAX_MESSAGES
、
COMPACT_COUNT
。
users/user.py
:
User
类,负责用户相关信息的管理。
users/model_cfg.py
:
ModelCfg
类和
check_name
函数,负责模型配置及相关检查。
skill.py
:
Skill
类,负责技能的管理。
agent/agent.py
:
Agent
类,负责整合各个组件,构建完整的 Agent 实例。
message/message.py
:
Message
数据类和各种消息创建函数。
message/SessionHistory.py
:
SessionHistory
类,负责会话历史的读取、保存与压缩。
mcp_server/
:MCP 服务端与客户端,包含
get_tools
、
execute_tool
、
ToolExecuteError
等功能,以及相关工具函数。
model/
:模型层封装,统一不同模型的调用接口,提供具有统一调用方法的模型实例。
2. 组件介绍
cc_agent 主要由以下组件组成:
User(用户)
Message(消息)
SessionHistory(会话历史)
ModelCfg(模型配置)
Skill(技能)
Agent(智能体)
Model(模型类)
Tool(工具,包括 MCP 服务端与客户端)
2.1 组件组合设计
在 cc_agent 中,User
是数据隔离的最小单位,用户名全局唯一。每个用户在本地
USER_DIR
下拥有独立的目录,用于存放自己的会话、模型配置和技能。
用户的本地目录结构如下:
1
2
3
4
5
6
7
8
9
10
USER_DIR/
└── <user_name>/
├── <user_name>.json 记录 user_name、session_id、model_cfg
├── session/
│ └── <session_id>.jsonl 一个会话一个文件,每行一条消息
├── model_cfg/
│ └── <cfg_name>.json 用户自己的模型配置
└── skills/
└── <skill_name>/
└── SKILL.md 用户自己的技能
User 内部维护两个主要列表:
session_id
:记录用户拥有的会话 ID。
model_cfg
:记录用户拥有的模型配置名称。
2.1.1 Message 与 SessionHistory
session_id
用于标识用户与 Agent
之间的一段会话。每段会话对应一个 JSONL
文件,其中每一行都是一条独立的消息。
消息由
Message
类管理,负责定义和创建不同类型的消息。例如:
1
2
{
"role"
:
"user"
,
"content"
:
"北京今天天气怎么样?"
}
{
"role"
:
"agent"
,
"content"
:
"北京今天晴,气温 8 到 18 度,早晚温差较大,建议带件外套。"
}
这里的
Message
负责消息结构的定义,而
SessionHistory
负责会话历史的管理。
SessionHistory
根据
session_id
读取对应的
JSONL 文件,将当前需要使用的历史消息加载到
history_message
中,并提供消息保存、历史压缩等功能。
它并不需要始终将整个 JSONL 文件加载到内存中,只需要保留 Agent
当前运行所需的历史消息即可。
2.1.2 ModelCfg(模型配置)
ModelCfg
是模型配置类,用于管理 Agent 使用的模型、工具和
Skill。
User 的
model_cfg
列表只记录模型配置名称,而不是直接保存模型配置实例。需要使用某个配置时,再通过
User 加载对应的
ModelCfg
实例。
模型配置支持全局和用户两种归属方式:
全局配置:新创建的
ModelCfg
默认没有用户归属,即
self.belong = None
。此时调用
save()
,配置会保存到全局的模型配置目录,供后续加载和复用。
用户配置:当 User
添加一个模型配置时,会将配置复制并保存到该用户自己的
model_cfg
目录中。后续调用
user.get_model_cfg(cfg_name)
,即可根据名称加载该用户对应的配置实例。
2.1.3 Skill(技能)
Skill 的存储和归属机制与 ModelCfg 类似。
默认情况下,Skill 保存到全局目录。只有当一个具有用户归属的
ModelCfg
添加 Skill 时,才会将对应的 Skill
复制到该用户自己的
skills
目录下。
如果没有用户归属的全局
ModelCfg
添加 Skill,那么该 Skill
仍然保存到全局目录。
模型配置中会记录所使用的 Skill 名称。虽然用户目录下可能存放多个
Skill,但只有被当前
ModelCfg
记录的
Skill,才会被加载到对应的 Agent 中。
2.1.4 Agent 组合
ModelCfg 中主要记录以下内容:
使用什么模型:由 Model 层提供。
拥有哪些工具:通过 MCP 服务添加。
拥有哪些 Skill:根据 Skill 名称加载对应技能。
一次完整的 Agent 初始化,需要准备两个核心对象:
通过 User 提供的
session_id
,创建对应的
SessionHistory
实例,获取会话历史。
通过 User 提供的模型配置名称,加载对应的
ModelCfg
实例。
随后,将这两个实例和系统提示词(
system_prompt
)传入
Agent,即可完成 Agent 的初始化。
1
2
3
class
Agent
:
def
__init__
(
self, history, model_cfg, system_prompt=
""
):
...
Agent
内部包含许多辅助功能,但其核心是实现一个持续运行的交互循环(Loop),负责协调模型调用、工具执行、消息处理等操作。
loop工作循环图
2.2 一轮对话的完整示例
下面通过一个实际示例,将前面介绍的组件串联起来。
运行前需要满足以下条件:
Ollama 中已经下载
qwen3:8b
。
MCP 服务已经在
http://localhost:8000/mcp
启动。
MCP 服务已经注册了
add
工具。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
import
asyncio
from
cc_agent.agent.agent
import
Agent
from
cc_agent.message.SessionHistory
import
SessionHistory
from
cc_agent.skill
import
Skill
from
cc_agent.users.model_cfg
import
ModelCfg
from
cc_agent.users.user
import
User
USER_NAME =
"demo_user"
CFG_NAME =
"qwen3_8b"
SERVER_NAME =
"local"
MCP_URL =
"http://localhost:8000/mcp"
async
def
main
():
# 1. 创建用户
user = User.create_user(USER_NAME)
# 2. 创建 session 和历史对象
session_id = user.new_session()
history = SessionHistory(USER_NAME, session_id)
# 3. 创建模型配置并添加到用户
new_cfg = ModelCfg(
CFG_NAME,
model_name=
"qwen3:8b"
,
url=
"http://localhost:11434/v1"
,
type
=
"ollama"
,
)
user.add_model_cfg(new_cfg)
# 根据 cfg_name 获取用户自己的配置实例
model_cfg = user.get_model_cfg(CFG_NAME)
# 4. 登记 MCP 服务并添加工具
model_cfg.add_server(SERVER_NAME, MCP_URL)
tool_name =
await
model_cfg.add_tool(SERVER_NAME,
"add"
)
# 返回 "local__add"
# 目前仅支持最基础的单工具添加方式
# 5. 添加 Skill
# 结构:{name, description, content}
skill = Skill(
"answer-format"
,
"用户要求计算时使用,规定回复的格式"
,
"计算完成后,回复只能写成:计算结果:<数字>"
,
)
# 自动复制到用户的 skills 目录
# 因为当前 model_cfg 已经具有用户归属
model_cfg.add_skill(skill)
# 6. 创建 Agent
# 工具和 Skill 需要在创建 Agent 之前添加完成
agent = Agent(
history,
model_cfg,
system_prompt=
"You are a helpful assistant."
)
# 7. 发起一轮对话
reply =
await
agent.generate(
"请用工具计算 123.5 加 876.5"
)
print
(reply)
asyncio.run(main())
以上代码展示了从创建用户、初始化会话、配置模型,到添加工具和
Skill,最后创建 Agent 并完成一次交互的完整过程。
2.3 User(用户)
前面介绍了 User
在整个框架中的作用:它是数据隔离的最小单位,负责管理用户自己的会话、模型配置和技能。
采用这种以用户为单位的目录结构,主要是为了方便后续扩展。
例如:
单机个体agent:只需要固定一个 User 和一个 Session,即可作为个体
Agent 使用。
团队协作:可以在此基础上扩展用户体系,将不同用户或团队的数据进行隔离。
Web 聊天应用:类似 GPT 的聊天机器人,可以为用户创建多个
Session,并允许用户添加和管理不同的模型配置。
目前框架没有设计用户等级、权限等分级系统,主要考虑的是单用户 Agent
的使用场景。后续如果有需要,再扩展相关功能。
用户管理
以下均为类方法,可以直接通过
User.xxx()
调用。
User.create_user(user_name)
:创建用户,同时生成对应的目录结构。
User.open_user(user_name)
:打开已有用户,返回对应的
User 实例。
User.delete_user(user_name)
:删除用户及其整个目录。
User.list_user()
:打印并返回所有用户名,以
USER_DIR
下的子文件夹作为用户记录。
会话管理
new_session()
:创建一个新的空 JSONL 文件,并返回对应的
session_id
。
delete_session(session_id)
:删除指定 Session
对应的文件及用户记录中的 ID。
list_session()
:打印并返回当前用户拥有的 Session ID
列表。
模型配置管理
add_model_cfg(model_cfg, cfg_name=None)
:将传入的模型配置复制到用户自己的目录中,并返回最终使用的配置名称。
get_model_cfg(cfg_name)
:根据配置名称,加载用户自己的
ModelCfg
实例。
list_model_cfg()
:打印并返回当前用户拥有的模型配置名称列表。
delete_model_cfg(cfg_name)
:删除指定模型配置文件及其记录。
技能查询
-
list_skill_name()
:打印并返回当前用户拥有的 Skill
名称列表。
这里需要特别说明一下,
list_skill_name()
并不是直接扫描用户的
skills
目录,而是读取用户所有模型配置中记录的 Skill 名称,然后合并去重。
这样设计是因为 Skill 文件可能被多个模型配置共同使用。
例如,模型配置 A 和 B 都使用了 Skill X。如果删除模型配置 A 时直接删除
Skill X,那么模型配置 B 就无法继续使用该技能。
因此,目前删除模型配置时,只会移除对应配置中记录的 Skill
名称,不会直接删除 Skill 文件。通过汇总所有模型配置中的 Skill
名称,就可以得到当前仍被使用的技能列表。
后续可以在此基础上增加无用 Skill 的清理功能,目前暂不实现。
2.4 Message(消息)
Message 是会话历史中的最小单位,一条 Message 对应 JSONL
文件中的一行记录。
2.4.1 消息字段
Message 包含以下字段:
role
:消息角色,包括
user
、
agent
、
tool
、
context
。
type
:消息类型,包括
plain
和
tool_call
。
content
:消息内容。
plain
类型为字符串,
tool_call
类型为字典,记录工具名称和调用参数。
seq
:消息在会话中的序号,从 1 开始递增,由
SessionHistory 在添加消息时自动分配。
call_id
:工具调用
ID,用于关联工具调用消息与对应的执行结果。
start
、
end
:context
消息的摘要范围,记录本次摘要覆盖的消息序号。
2.4.2 消息类型
根据
role
和
type
的组合,目前有以下五种消息:
user
+
plain
:用户输入。
agent
+
plain
:Agent 的普通回复。
agent
+
tool_call
:Agent
发起的工具调用。
tool
+
plain
:工具执行结果,执行失败时以
tool error:
开头。
context
+
plain
:历史消息压缩后的摘要,作为当前消息窗口的第一条消息。
2.4.3 消息创建与转换
提供以下函数创建不同类型的消息:
createUserMessage(text)
:创建用户消息。
createAiMessage(text)
:创建 Agent 普通回复。
create_AiToolCall_Message(name, arguments, call_id=None)
:创建工具调用消息,不传入
call_id
时自动生成。
create_ToolResult_Message(text, call_id)
:创建工具执行结果。
createContextMessage(text, start, end)
:创建历史摘要消息。
此外,还提供两个转换方法:
toJson()
:将 Message 实例转换为 JSON 字符串。
Message.fromJson(text)
:将 JSON 字符串解析为 Message
实例。
2.5 SessionHistory(会话历史)
每个 Session 对应一个 JSONL 文件,存储路径为:
1
USER_DIR/<user_name>/session/<session_id>.jsonl
SessionHistory
是会话历史的内存管理器,负责消息的读取、写入和压缩。Agent
不直接操作会话文件,而是通过 SessionHistory 管理消息。
初始化方式:
1
messages = SessionHistory(user_id, session_id)
其中
user_id
实际上是用户名,对应的 Session
文件必须已经存在。
SessionHistory 内部维护一个
messages
列表,每个元素都是
Message 实例。初始化时逐行读取 JSONL 文件,并通过
Message.fromJson(line)
转换为对象。
Agent 与 SessionHistory 共用同一个对象引用,因此 Agent
对消息的修改会直接反映在 SessionHistory 中。
目前框架没有实现文件锁或并发写入保护,因此同一个 Session
同一时间只应由一个 Agent 写入。
2.5.1 基本方法
add(message)
:为消息分配序号,并添加到内存中的
messages
列表,不立即写入文件。
save()
:将新增消息追加写入 JSONL 文件。
load()
:重新读取 JSONL 文件,并返回消息列表。
await checkCompact()
:检查是否满足压缩条件,满足时执行压缩。
await compact()
:执行历史消息压缩。
2.5.2 JSONL 消息存储
会话文件采用 JSONL 格式,每行保存一条完整的 JSON
消息。普通保存采用追加写入,不会覆盖之前的记录。
例如,一轮包含工具调用的对话:
1
2
3
4
{
"role"
:
"user"
,
"type"
:
"plain"
,
"content"
:
"请用工具计算 123.5 加 876.5"
,
"seq"
:
1
,
"call_id"
:
null
,
"start"
:
null
,
"end"
:
null
}
{
"role"
:
"agent"
,
"type"
:
"tool_call"
,
"content"
:
{
"name"
:
"local__add"
,
"arguments"
:
{
"a"
:
123.5
,
"b"
:
876.5
}
,
"seq"
:
2
,
"call_id"
:
"a1b2c3d4"
,
"start"
:
null
,
"end"
:
null
}
{
"role"
:
"tool"
,
"type"
:
"plain"
,
"content"
:
"1000.0"
,
"seq"
:
3
,
"call_id"
:
"a1b2c3d4"
,
"start"
:
null
,
"end"
:
null
}
{
"role"
:
"agent"
,
"type"
:
"plain"
,
"content"
:
"结果是 1000。"
,
"seq"
:
4
,
"call_id"
:
null
,
"start"
:
null
,
"end"
:
null
}
2.5.3 消息窗口
JSONL 文件保留完整的历史记录,而内存中的
messages
只加载当前消息窗口。
加载规则:
如果文件中没有
context
消息,则加载全部历史记录。
如果存在
context
消息,则从最后一条
context
消息开始,加载该消息及其后面的所有记录。
context
保存此前历史消息的摘要。生成模型输入时,Agent
会将其内容拼接到 System Prompt 后面,而不是作为独立消息发送。
因此,两者的区别是:
messages
:当前窗口中的消息。
JSONL 文件:完整的历史记录,包括压缩前的旧消息。
2.5.4 历史消息压缩
当一轮对话结束并保存消息后,SessionHistory
会检查当前窗口的消息数量。如果达到
MAX_MESSAGES
,则触发压缩。
压缩范围
从当前窗口中取最旧的
COMPACT_COUNT
条消息进行压缩。如果第一条消息是上一次压缩生成的
context
,也会一并纳入本次压缩,使摘要逐级叠加。
切分位置会向后移动到下一条文字消息,避免在切分后留下不完整的工具调用记录。
生成摘要
CompressModel
只处理
plain
类型的消息,将其整理为“角色: 内容”的文本,再调用本地 Ollama 的
qwen3:8b
生成摘要。
提示词要求模型保留关键事实,并将内容压缩至原文长度的约
70%。这个比例是提示词要求的目标,并不保证实际输出长度。
压缩结果
压缩完成后,内存中的消息窗口变为:
一条新的
context
摘要消息。
尚未参与压缩的剩余消息。
随后重新分配消息序号,并将整个新窗口追加写入 JSONL 文件。
默认配置为:
MAX_MESSAGES = 300
COMPACT_COUNT = 200
因此,触发压缩时会处理约 200 条消息,保留约 100
条未压缩消息,最终窗口约为 101 条。
压缩不会删除文件中的旧记录,而是将新的消息窗口追加到文件末尾。由于加载时只读取最后一条
context
及其后续记录,因此不会重复加载旧消息。
注意事项
tool_call
类型的消息不会参与摘要,因此工具名称和调用参数不会保留在摘要中。工具执行结果属于
plain
消息,会参与摘要,但摘要可能无法准确保留它与原工具调用之间的对应关系。
压缩使用的
CompressModel
独立于 ModelCfg,默认调用本地
Ollama 的
qwen3:8b
。执行压缩前,需要确保 Ollama
服务和对应模型可用。
2.6 ModelCfg(模型配置)
ModelCfg 用于管理 Agent 的模型配置,同时记录 MCP 服务、工具和
Skill。
工具与 Skill 都绑定在 ModelCfg 上。Agent 使用不同的 ModelCfg
时,会加载对应配置中记录的工具和 Skill。
2.6.1 初始化与配置字段
初始化方式:
1
ModelCfg(cfg_name, model_name, url, key=
None
,
type
=
"ollama"
)
主要字段:
cfg_name
:配置名称,必填,同时作为配置文件名。
model_name
:模型名称。
url
:模型 API 服务地址。
key
:服务商 API Key,使用本地 Ollama 时可以留空。
type
:模型类型,目前定义了
ollama
、
vllm
、
deepseek
,但 Agent
当前只支持
ollama
。
belong
:配置归属用户名,
None
表示全局配置。
2.6.2 MCP 服务与工具管理
ModelCfg 提供以下方法管理 MCP 服务和工具:
MCP 服务管理
add_server(server_name, url)
:添加 MCP
服务,建立服务别名与 URL 的映射。
list_server()
:列出已添加的 MCP 服务。
delete_server(server_name)
:删除指定 MCP 服务。
工具管理
await add_tool(server_name, func_name)
:从指定 MCP
服务获取工具并添加到配置中,返回带服务别名前缀的工具名称,例如
local__add
。
list_tool()
:列出已添加的工具。
delete_tool(tool_name)
:删除指定工具。
to_openai_tools()
:将工具转换为 OpenAI API
兼容的工具格式。
Agent 调用模型时,会将转换后的工具列表传入模型:
1
llm.generate(openai_messages,
self
.tools)
具体的消息和工具格式处理由模型类负责。对于非 OpenAI
兼容的模型,需要在对应模型类中将这些数据转换为模型支持的输入格式。
2.6.3 Skill 管理
ModelCfg 通过以下方法管理 Skill:
add_skill(skill)
:添加 Skill。
list_skill()
:列出已添加的 Skill。
delete_skill(skill_name)
:移除指定 Skill。
ModelCfg 只记录 Skill 名称,不直接保存 Skill 内容。删除 Skill
时,也只会从配置中移除名称,不会删除对应的 Skill 文件。
2.6.4 配置文件管理
ModelCfg 提供以下方法管理配置文件:
save()
:保存配置。
delete()
:删除配置文件。
ModelCfg.load_global(cfg_name)
:根据配置名称,从全局配置目录加载配置。
ModelCfg.load(file_path)
:根据指定路径加载配置。
配置文件采用 JSON 格式,以
cfg_name
命名。
保存规则
新建 ModelCfg 对象时,配置只保存在内存中,不会自动写入文件。
全局配置需要显式调用
save()
,保存到
MODEL_CFG_DIR
。如果存在同名配置,则报错。
用户配置通过
User.add_model_cfg()
复制到对应用户目录。
配置文件创建后,后续对 MCP 服务、工具和 Skill
的修改会自动写回配置文件。
2.7 Skill(技能)
Skill 是一份提供给模型的指令文档,通常保存为
SKILL.md
。模型通过
description
判断是否需要使用该 Skill,只有调用
load_skill
工具后,才会获取完整的
content
。
1
Skill(name, description, content)
# description 必须为单行
文件格式:
1
2
3
4
5
6
---
name: code-review
description: 审查 Python 代码时使用
---
(正文,可以很长)
存放位置:
全局:
SKILL_DIR/<name>/SKILL.md
用户:
USER_DIR/<user_name>/skills/<name>/SKILL.md
每个 Skill 使用独立文件夹存放,而不是直接保存为单个文件。这样可以在
Skill
需要额外附件(如脚本、配置文件等)时,将它们放在对应目录下,方便统一管理。
方法:
save(overwrite=False)
:保存 Skill。
belong
为
None
时保存到全局目录,否则保存到指定用户目录。默认不覆盖已有的同名文件。
Skill.load(name, belong=None)
:加载 Skill,返回对应的
Skill 对象。
belong=None
时从全局目录加载;指定
belong=user_name
时,从对应用户目录加载。
Skill.exists(name, belong=None)
:判断指定目录下是否存在该
Skill。
Skill.path_of(name, belong=None)
:根据 Skill
名称和归属生成文件路径,供加载等操作使用。
copy_to_user(user_name)
:将 Skill
复制到指定用户目录。如果目标目录已存在同名
Skill,则直接复用,不覆盖。
delete()
:删除 Skill 文件。如果仍有模型配置引用该
Skill,则拒绝删除,并提示正在使用它的模型配置。
prompt_line()
:将 Skill 转换为一行提示词,用于追加到
system prompt 中。
2.8 Agent(智能体)
Agent
负责执行一轮完整的对话。它将历史消息、用户输入和工具列表发送给模型;如果模型需要调用工具,就执行对应工具,并将执行结果返回给模型,直到模型生成最终回复。
1
Agent(history, model_cfg, system_prompt="")
参数:
history
:当前会话的
SessionHistory
对象。
model_cfg
:当前用户的
ModelCfg
对象。
初始化过程:
创建 Agent 时,会读取一次模型配置,并完成以下操作:
根据配置中的
type
创建对应的模型对象。
获取可用的工具列表。
加载配置中记录的 Skill。如果对应的 Skill
文件不存在,则直接报错。
如果配置中包含 Skill,则在 system prompt 后追加 Skill
列表,并注册一个内置工具
load_skill
,供模型按需加载 Skill
的完整内容。
方法:
await generate(user_input)
:执行一轮完整对话,返回最终回复文本。内部通过循环处理模型回复和工具调用,最多调用模型
MAX_STEPS
次。超过限制后抛出异常,本轮消息不会保存。
await run_tool(call_message)
:执行一条工具调用,并返回工具结果消息。如果执行出错,不会直接抛出异常,而是返回一段以
tool error:
开头的错误信息,交给模型自行分析和修正。
parse_arguments(raw_arguments)
:将模型返回的参数字符串解析为字典。
2.9 Model 与模型类
Model 层负责封装不同服务商的模型接口,对上层 Agent
提供统一的调用方式。无论底层使用哪种模型,最终都会通过统一的
Model
接口供 Agent 调用。
Model 基类
Model
是模型基类,只定义了一个接口方法:
1
await generate(messages, tools=None)
messages
:消息列表。
tools
:工具列表,可选。
该方法负责将消息和工具发送给底层模型,并返回模型的回复内容。
OllamaOpenAI
目前实现的一个模型类是
OllamaOpenAI(model_name, base_url)
。
该类通过 Ollama 提供的 OpenAI 兼容接口调用模型,API Key 固定设置为
"ollama"
。
目前本地使用的是
qwen3:8b
,因此
generate()
内部实现 接收的
messages
和
tools
均采用
OpenAI 兼容格式。
CompressModel
另一个模型类是
CompressModel
,同样继承自
Model
,但用途不同。
它专门用于历史消息压缩,不参与 Agent 的常规对话和工具调用流程。
2.10 MCP 服务与客户端
cc_agent 内部的工具统一通过 MCP 服务提供,因此项目内部也实现了一个
MCP 服务器。需要添加工具时,只需在
cc_agent/mcp_server/functions/
目录下编写工具函数,服务器启动时会自动加载该目录下定义的工具。
例如,在
cc_agent/mcp_server/functions/calc.py
中编写:
1
2
3
4
5
def
add
(
a:
float
, b:
float
) ->
float
:
"""计算两个数之和。"""
return
a + b
TOOLS = [add]
其中,
TOOLS
用于指定该文件需要注册为 MCP
工具的函数,服务器会自动加载其中的所有函数。
启动 MCP 服务器:
1
python -m cc_agent.mcp_server.server
当然,也可以直接使用自己的 MCP 服务器。cc_agent 不依赖特定的 MCP
服务端实现,只要能够通过 MCP 协议获取工具列表并执行工具即可。
MCP 客户端
client.py
提供了操作 MCP
服务的辅助函数,主要负责获取工具列表和执行具体工具:
client.get_tools(url)
:异步获取指定 MCP
服务的工具列表,并转换为 OpenAI
格式。获取结果带有缓存,避免重复请求。
client.execute_tool(url, name, arguments)
:异步执行指定工具,传入服务地址、工具名称和参数。执行失败时抛出
ToolExecuteError
。
客户端内部还实现了
tools_cache
缓存机制。获取工具列表时,会优先检查缓存,只有缓存中不存在对应数据时,才会连接目标
MCP 服务获取。
2.11 config.py
config.py
是项目的全局配置文件,主要用于统一管理数据存储路径和运行参数。
所有路径均以运行脚本时的当前工作目录(
Path.cwd()
)为基准。因此,从不同目录启动脚本,生成的数据也会存放在不同位置。
主要配置:
USER_DIR
:
users_data/
,用户数据根目录。
MODEL_CFG_DIR
:
model_cfg_data/
,全局模型配置目录。
SKILL_DIR
:
skills_data/
,全局 Skill
存储目录。
MAX_MESSAGES
:
300
,会话历史达到该消息数量时触发压缩。
COMPACT_COUNT
:
200
,每次压缩时选取最旧的消息数量。
3. 组件之间的关系
3.1
按用户隔离,全局与用户两级存储
ModelCfg
和
Skill
都支持全局和用户两种归属,通过
belong
区分:
全局配置:
belong=None
,保存在全局目录(
MODEL_CFG_DIR
、
SKILL_DIR
)。创建时如果没有指定用户,则默认保存在全局目录。
用户配置:
belong
为用户名,保存在对应用户的目录下。不同用户的数据相互隔离。
3.2 用户、会话与 Agent
用户名唯一,一个用户可以创建多个 session,每个 session
独立记录对话历史。
每个 session 对应一个
SessionHistory
对象,通过读取对应的 JSONL 文件恢复历史消息。
创建 Agent 需要两个核心对象:当前 session 的
SessionHistory
和一个
ModelCfg
。
Agent 根据
SessionHistory
获取对话历史,根据
ModelCfg
获取模型、工具和 Skill 配置。
3.3 工具、Skill 与 ModelCfg
工具和 Skill 都需要添加到指定的
ModelCfg
中才能使用。它们绑定在具体的模型配置上,不同的
ModelCfg
可以拥有不同的工具和 Skill。
Skill:每个
ModelCfg
只记录自己添加过的 Skill
名称,不需要包含所有 Skill。Agent 初始化时根据这些名称加载对应的
Skill。
工具:只能通过 MCP 服务获取,
ModelCfg
记录 MCP
服务的别名与 URL,以及已添加的工具。
工具命名:工具名会添加服务别名前缀,格式为
服务别名__原始名称
,例如
local__add
,用于避免不同 MCP 服务之间出现工具重名。
重复添加:同一个
ModelCfg
中添加同名工具会抛出异常。
3.4 关系图
1
2
3
4
5
6
7
8
9
10
User ──┬── session(xxx.jsonl) ──── SessionHistory ─────┐
│ ├──► Agent(history, model_cfg, system_prompt)
├── model_cfg ──┬── servers (别名→url) ─────────┘ │
│ ├── tools (来自 MCP) ├─ 调用模型 (OllamaOpenAI)
│ └── skill 名字 ──► skills/<name>/SKILL.md │
│ └─ 执行工具 (mcp client)
└── skills (用户目录下的副本)
全局目录:MODEL_CFG_DIR、SKILL_DIR
用户 add 时,复制一份到用户目录
4. 使用流程
4.1 创建用户
1
user = User.create_user(
"demo_user"
)
如果用户已经存在,可以使用
User.open_user("demo_user")
打开已有用户。
创建用户后,会在
USER_DIR
下生成对应的用户目录,并自动创建三个子目录:
session/
:存放用户的会话历史文件。
model_cfg/
:存放用户的模型配置。
skills/
:存放用户的 Skill 副本。
4.2 创建 session
1
2
session_id = user.new_session()
history = SessionHistory(
"demo_user"
, session_id)
new_session()
只负责创建会话并返回
session_id
,不会直接返回历史对象。因此,还需要通过用户名和会话
ID 创建
SessionHistory
,用于读取和管理该会话的消息历史。
4.3 创建模型配置并添加工具
创建
ModelCfg
对象,通过
user.add_model_cfg(...)
将其添加到用户配置中。
使用
user.get_model_cfg(cfg_name)
获取用户目录下的模型配置对象。后续操作应针对这个对象,而不是最初创建的对象。
使用
add_server(server_name, url)
登记 MCP
服务,建立服务别名与 URL 的对应关系。
使用
await add_tool(server_name, func_name)
添加工具。每次只能添加一个工具,没有批量添加接口。
使用
list_tool()
查看已添加的工具,使用
to_openai_tools()
查看最终提供给模型的 OpenAI
格式工具列表。
准备工作: 添加工具前,必须确保对应的 MCP 服务已经启动,因为
add_tool()
需要连接 MCP 服务并获取工具列表。
4.4 添加 Skill
创建
Skill(name, description, content)
对象。
调用用户模型配置的
model_cfg.add_skill(skill)
添加
Skill。该操作会自动将 Skill 复制到用户的
skills/
目录,而模型配置中只记录 Skill 名称。
也可以先通过
skill.save()
将 Skill
保存到全局目录,再供多个用户使用;或者使用
Skill.from_file(path)
导入已有的
SKILL.md
文件。
4.5 一轮对话的完整示例
下面将前面的操作串联起来,演示从创建用户到完成一轮对话的完整流程。
运行前需要准备:
Ollama 已下载
qwen3:8b
模型。
MCP 服务已启动,地址为
http://localhost:8000/mcp
。
MCP 服务中已注册
add
工具。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
import
asyncio
from
cc_agent.agent.agent
import
Agent
from
cc_agent.message.SessionHistory
import
SessionHistory
from
cc_agent.skill
import
Skill
from
cc_agent.users.model_cfg
import
ModelCfg
from
cc_agent.users.user
import
User
USER_NAME =
"demo_user"
CFG_NAME =
"qwen3_8b"
SERVER_NAME =
"local"
MCP_URL =
"http://localhost:8000/mcp"
async
def
main
():
# 1. 创建用户
user = User.create_user(USER_NAME)
# 2. 创建 session 和历史对象
session_id = user.new_session()
history = SessionHistory(USER_NAME, session_id)
# 3. 创建模型配置,添加到用户,再获取用户自己的配置对象
new_cfg = ModelCfg(
CFG_NAME,
model_name=
"qwen3:8b"
,
url=
"http://localhost:11434/v1"
,
type
=
"ollama"
,
)
user.add_model_cfg(new_cfg)
model_cfg = user.get_model_cfg(CFG_NAME)
# 4. 登记 MCP 服务并添加工具
model_cfg.add_server(SERVER_NAME, MCP_URL)
tool_name =
await
model_cfg.add_tool(SERVER_NAME,
"add"
)
# 5. 添加 Skill(自动复制到用户的 skills 目录)
skill = Skill(
"answer-format"
,
"用户要求计算时使用,规定回复的格式"
,
"计算完成后,回复只能写成:计算结果:<数字>"
,
)
model_cfg.add_skill(skill)
# 6. 创建 Agent,必须先完成工具和 Skill 的配置
agent = Agent(history, model_cfg,
"You are a helpful assistant."
)
# 7. 执行一轮对话
reply =
await
agent.generate(
"请用工具计算 123.5 加 876.5"
)
print
(reply)
asyncio.run(main())
一轮对话内部的执行过程:
Agent 将 system prompt(包含 Skill
列表)、
history.messages
(当前窗口内的历史消息)和本次用户输入组合成
OpenAI 格式的消息列表。其中,context 消息(旧消息摘要)会合并到 system
prompt 中;连续的多条
tool_call
消息会合并为一条带有
tool_calls
的 assistant 消息。
Agent 调用模型。如果模型返回了工具调用请求,则进入工具执行流程:
模型可能先调用
load_skill
获取 Skill
的完整内容,也可能直接调用
local__add
等工具。
Agent 通过
run_tool
执行每条工具调用。对于带有
服务别名__
前缀的工具,根据前缀找到对应的 MCP 服务
URL,再通过 MCP Client 执行;对于
load_skill
,则直接返回对应 Skill 的正文。
工具执行结果封装为
tool
消息,追加到本轮消息中,再次发送给模型。
重复上述过程,直到模型返回不包含工具调用的普通回复,作为本轮对话的最终结果。
将本轮缓存的全部消息(用户输入、模型的工具调用、工具执行结果和最终回复)一次性添加到
history
,然后调用
history.save()
保存到 JSONL
文件,最后执行
await history.checkCompact()
,检查是否达到压缩条件。
本轮产生的消息示例:
以下是实际测试中的一次对话记录:
1
2
3
4
5
6
7
===== 对话之后(窗口
6
条)=====
[
seq=
1
]
role=user type=plain content='我是 IT 部门的,电脑坏了,请帮我登记一个报修工单。'
[
seq=
2
]
role=agent type=tool_call content=
{
'name'
:
'load_skill'
,
'arguments'
:
{
'skill_name'
:
'test-ticket-format'
}
call_id=call_lpfdu0d7
[
seq=
3
]
role=tool type=plain content='登记工单时,严格按以下步骤执行:\n1. 先调用 local__get_ticket_id 工具获取工单号,参数 department 填用户所在部门的英文小写缩写(例如 it、hr)。\n2. 拿到工单号后,回复只能用下面这个格式,不要添加任何其他内容:\n工单已登记 | 编号:<工单号> | 处理人:Orion-Desk' call_id=call_lpfdu0d7
[
seq=
4
]
role=agent type=tool_call content=
{
'name'
:
'local__get_ticket_id'
,
'arguments'
:
{
'department'
:
'it'
}
call_id=call_j3sihoj3
[
seq=
5
]
role=tool type=plain content='TICKET-IT
-4821
' call_id=call_j3sihoj3
[
seq=
6
]
role=agent type=plain content='工单已登记 | 编号:TICKET-IT
-4821
| 处理人:Orion-Desk'
4.6 再次使用已有用户和会话
程序重启后,可以通过用户信息和已有的 session ID 恢复之前的对话:
1
2
3
4
5
6
7
8
9
user = User.open_user(
"demo_user"
)
session_ids = user.list_session()
cfg_names = user.list_model_cfg()
history = SessionHistory(
"demo_user"
, session_ids[
0
])
model_cfg = user.get_model_cfg(cfg_names[
0
])
agent = Agent(history, model_cfg,
"You are a helpful assistant."
)
reply =
await
agent.generate(
"我刚才问了什么?"
)
恢复过程:
User.open_user()
打开已有用户。
list_session()
获取该用户的会话 ID 列表。
list_model_cfg()
获取该用户的模型配置名称列表。
SessionHistory()
根据用户名和 session ID
加载对应的历史消息。
get_model_cfg()
获取已有的模型配置。
使用历史对象和模型配置重新创建 Agent,即可继续之前的对话。
需要注意,
SessionHistory
只会加载当前窗口中的消息,即最后一条
context
消息之后的完整消息。模型实际能够看到的是压缩摘要(context)以及摘要之后的完整对话历史。
ai_agent工具mcp
@zusda
原文 ↗
Measuring how AI agents handle everyday wet-lab science | OpenAI
November 6, 2026
Publication
Measuring how AI agents handle everyday wet-lab science
Introducing BioBasicsBench, a benchmark measuring how well AI agents interpret raw experimental data across 100 routine wet-lab tasks.
Listen to article
10:37
Our mission is to ensure that artificial general intelligence benefits all of humanity. As part of that mission, we want to help scientists accelerate biological research—from understanding disease to developing new treatments. For AI agents to contribute, they need to reliably interpret the experimental evidence scientists work with every day in wet labs, where they run experiments with biological samples. Everyday wet-lab work involves failed experiments, ambiguous images and conflicting observations. Existing benchmarks are built around charts and images selected for scientific papers may miss this ambiguity and the troubleshooting it requires. That’s why we’re introducing BioBasicsBench, a new benchmark developed with Ligo, a biotech company focused on enzyme design, to measure how well agents handle routine wet-lab analysis.
The benchmark’s 100 tasks test whether agents can interpret raw experimental data, identify errors, and recognize when the evidence cannot support a conclusion. Getting these tasks right matters because misreading an image or overlooking a problem with the controls used to check an experiment’s results can lead scientists to draw the wrong conclusion and spend time on experiments that won’t help answer their question.
At the ‘xhigh’ setting, which allows more reasoning per task, GPT‑6 Astra earned an average of 66.7% of the available credit, compared with 43.8% for GPT‑5.6 Sol. The results show progress on routine lab analysis, with room for further improvements in interpreting images, checking controls and applying experimental context.
Inside BioBasicsBench
BioBasicsBench uses raw experimental evidence, from gel images and colony plates to microscopy and instrument readouts.
BioBasicsBench has 100 tasks across five areas of routine wet-lab work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. Tasks range from simple requests, such as reading the molecular weight of a DNA sample in a gel, to detailed assay analysis, but all involve routine work that wet-lab scientists should be able to complete reliably.
Scientists created the tasks using data from their own experiments, pairing each question with expert-written grading criteria. Each task underwent an internal review by a Ligo scientist who had not written it, an external review by a PhD-level scientist working in a wet lab, and a final internal review.
Agents return a structured answer with the requested conclusion, measurement or recommendation and, where required, supporting reasoning. Answers receive credit against individual rubric criteria, allowing the benchmark to distinguish a correct conclusion from a partially correct analysis. Most tasks test several skills together, including biological knowledge, interpretation of images and other data, error detection and protocol understanding.
Tasks by domain and tested skill
100 unique tasks · each task can test multiple areas
“Unique tasks” counts each task once. The skill columns count tasks with each skill tag. These categories overlap: one task can appear in several columns, so the skill counts should not be added to obtain a task total. Color identifies the domain; stronger shading indicates a larger share of that domain’s tasks testing a skill.
The inputs preserve the forms of evidence scientists work with: gel and blot images, plate photographs, microscopy stacks, sequencing reads, plate-reader exports and videos of bench work. A task can include several kinds of data. Reference sheets, such as ladder charts and plasmid maps, support the interpretation of the experimental evidence.
What the agent is given
Kind of data
Tasks
What it includes
Gels & blots
31
Agarose DNA gels, SDS-PAGE gels, Western blots
Protocols, notes & specs
27
Protocols and SOPs, lab notebooks and run records, photographed handwritten notes, catalogues, instrument specs, published methods
Tables & instrument exports
20
CSV and Excel exports, lane layouts and inventories, raw flow cytometry (FCS) files
Bench, apparatus & reagent photos
18
Benches, gel rigs, reagent and culture tubes, a PCR plate, tube racks, pH strips, a concentrator, a freezer, a liquid-handler deck
Microscopy
14
Raw brightfield and fluorescence images, multi-channel sets, z-stacks, time-lapse movies
Instrument screens & plots
13
Photos of flow cytometer, microscope, NanoDrop, plate reader and balance screens; exported standard curves, chromatograms and plots
Sequencing files
12
Raw long-read FASTQs, FASTA sequences, GenBank maps
Protocol videos
7
Streaking, inoculation, serial dilution, SDS-PAGE tank and incubator setups, weighing on a balance
Agar plates & colonies
7
Transformation plates, blue/white screens, antibiotic-disc plates
Reference sheets
21
Ladder charts, plasmid maps, pH charts (supporting material)
Number of tasks that include each kind of data. Categories overlap; reference sheets are supporting material.
Scientists wrote the tasks using data from their own experiments, pairing the experimental evidence with a question and an expert-written grading rubric. Each task went through an internal review by a Ligo scientist who had not written it, an external review by a PhD-level scientist working in a wet lab, and a final internal review. Reviewers checked the prompt, experimental evidence, rubric, author notes and a graded model attempt.
Answers are scored against the task’s rubric, which assigns credit to individual criteria. Depending on the task, these criteria assess the conclusion, numerical results, supporting reasoning or recognition that the evidence is insufficient.
Results
We evaluated seven models on BioBasicsBench: GPT‑5.2, GPT‑5.4, GPT‑5.5, GPT‑5.6 Sol, GPT‑6.1 Sol, GPT‑6 Luna and GPT‑6 Astra. The evaluation covers 37 model and reasoning-effort settings, with five attempts on each of the 100 tasks per setting, for a total of 18,500 results.
We report
mean reward
: the average task score, expressed as a percentage of the available credit. Error bars show 95% confidence intervals calculated by resampling whole tasks, keeping the five attempts for each task together.
Mean reward by model and reasoning effort
GPT-6 Astra
GPT-6.1 Sol
GPT-6 Luna
GPT-5.6 Sol
GPT-5.5
GPT-5.4
GPT-5.2
Mean reward versus average output tokens per task, shown on a logarithmic scale. Each point represents a tested reasoning-effort setting; lines connect those settings in order. The x-axis doubles at each tick. Dashed orange lines show GPT‑5 models, and solid blue lines show GPT‑6 models. Faint whiskers show 95% whole-task bootstrap confidence intervals. Each setting includes all 100 tasks and five attempts per task. Effort settings differ across models.
GPT‑6 Astra substantially improves on GPT‑5.6 Sol, even at low reasoning effort. These gains suggest that
stronger native multimodal capabilities may be key to improving performance on everyday wet-lab
analysis.
Performance at xhigh reasoning effort
Model
Mean reward
95% CI
Output tokens / task
GPT‑6
Astra
66.7%
60.3–73.1%
4,704
GPT‑6.1
Sol
62.0%
55.6–68.4%
5,238
GPT‑5.6
Sol
43.8%
37.6–50.0%
9,099
GPT‑6
Luna
38.3%
32.3–44.5%
13,120
GPT‑5.5
35.4%
29.7–41.0%
11,135
GPT‑5.4
33.2%
28.3–38.4%
13,717
GPT‑5.2
31.2%
26.2–36.3%
27,620
Xhigh is the highest reasoning-effort setting reported for all seven models. Each row includes all 100 tasks and five attempts per task. Max-effort results, where available, are included in the chart above.
GPT‑6 Astra improvements from GPT‑5.6 Sol
Recognizing what the data can and cannot show
A common trap in science is over-interpreting the data, especially given the nature of bias that occurs when testing a hypothesis. Astra demonstrated a better ability to discern the limit of what the data shows and not draw unwarranted conclusions.
In this example, GPT‑5.6 Sol classified cells as dead based on the signal from a dye used to detect dead cells. But the detector could also pick up light from a second dye, so brightness alone could not justify that conclusion. Astra recognized this ambiguity and called for better controls to separate the signals.
The same cells, three detection channels
Brightfield · transmitted light
Channel 2 · SYTOX-labelled · 470–540 nm
Channel 3 · FUN-1 green · 525–580 nm
The same cell regions in three channels; the detection windows overlap at 525–540 nm. The shared window permits interference; its amount has not been measured. Each channel is displayed with a separate linear intensity scale, fixed across both regions; brightness cannot be compared quantitatively between channels.
Better multimodal handling
Scientific research is highly multimodal, requiring constant visual interpretation and decision making. Astra displayed a significant improvement at handling multimodal data which was given in the form of photos and videos, as well as raw outputs from sequencing, plate-reader assays, microscopy and flow cytometry.
At the most basic level, this improvement was noticeable through GPT‑6 Astra’s ability to pick up on finer detail from images taken during experiments and therefore draw more accurate conclusions as shown in this example. This task asked agents to draw conclusions from a DNA cloning experiment, where controls are essential. GPT‑5.6 Sol saw no colonies on the negative control plate and based its conclusions on that observation. GPT‑6 Astra noticed that the plate was labeled with the wrong antibiotic and correctly rejected it as a reliable control.
Original photograph
The label on the rim
The original benchmark photograph, with the rim label circled and enlarged. “LB Kan50” identifies kanamycin, a different antibiotic from the ampicillin used in the experiment.
Keeping experimental context in mind
Scientists must be able to hold tacit biological knowledge and experimental context in mind whilst analysing data in order to correctly interpret what it shows. Astra demonstrated an improved ability to maintain this context and apply it throughout its analysis.
One task showed cells coming into and out of focus in a series of microscope images. While GPT‑5.6 Sol looked for a biological explanation for the cells’ apparent disappearance, GPT‑6 Astra correctly recognized a z-stack: a series of images taken at different depths.
Changing visibility across a depth scan
0 s
3 s
6 s
9 s
12 s
15 s
Six frames from the original recording. Dashed boxes mark the same location as the microscope scans through depth. Times refer to playback position in the supplied video.
Where models still fall short
Higher benchmark scores do not yet mean models can make reliable judgments in the lab. These Astra examples show how assumptions, expected answers and supplied labels can override the evidence, and how a correct intermediate observation can fail to guide the final decision.
Letting an assumption override the controls
Astra lets a familiar assumption override the interpretation supported by the experimental controls.
It can distinguish the two groups of colony bands, but assumes that a larger PCR product would identify successful cloning. On that condition it proposes lanes
4, 6, 7 and 8
; the reference’s control comparison instead selects lanes
5, 9 and 10
.
The reference selects colonies
5, 9 and 10
; Astra proposes
4, 6, 7 and 8
.
1: marker · 2: cloning mixture · 3: original plasmid · 4–10: colony samples
Reading the controls
Lane 3 contains only the original plasmid PCR. Its band provides the old-plasmid reference position.
In the reference interpretation,
lane 2 contains both patterns
: incomplete assembly leaves old template/backbone alongside the new product.
The resulting colonies can carry either old or new plasmid.
Compare their bands with these reference patterns.
Reference selection:
5, 9 and 10
Astra’s selection:
4, 6, 7 and 8. Astra nominates this group suggesting the larger PCR product represents the intended construct.
The visual grouping is correct but the experimental identities attached to those groups are reversed relative to the reference. Astra therefore selects the wrong colonies from the colony PCR results which would result in the down-stream purification of the incorrect plasmid.
Discounting data that contradict its preferred interpretation
Astra discounts observations that contradict its preferred interpretation.
It concludes that vacuoles, compartments inside cells, are generally darker than the surrounding cell, even though some appear lighter in the same image. Both appearances matter: changing the microscope’s focus can reverse the contrast, so the darker examples do not establish a general rule.
Darker appearance
Lighter appearance
Astra’s darker-vacuole generalisation omits the lighter vacuoles visible in the same field.
What that answer misses
Some cells have an opposite contrast. Both appearances occur because different focal depths can reverse the contrast.
Astra inspects several regions and even comments that contrast varies with focus. It then gives greater weight to the cells it calls better focused and retains only their darker-vacuole description in the final answer. The contradictory lighter regions have been inspected but are not considered in the conclusion.
Miscounting colonies in a routine laboratory measurement
Astra’s counting marks do not reliably correspond to individual colonies: it fails to resolve close neighbours, misses visible colonies and marks empty agar.
The net result on this plate is an overcount of approximately 12% relative to the benchmark reference.
A. Closely spaced colonies
Original image
Astra’s counting marks
B. Visible colonies left unmarked
Original image
Astra’s counting marks
C. Empty agar counted as a colony
Original image
Astra’s counting marks
A: one counting mark between neighbouring colonies. B: two visible colonies left unmarked. C: empty agar counted as a colony. Red circles indicate Astra’s counting positions. Black arrows are editorial guides.
An inaccurate colony count directly affects estimates of how efficiently DNA was introduced into cells. With the plated volume and dilution held fixed, an overcount inflates that estimate by the same proportion.
Building toward reliable lab assistance
Accelerating biological research requires getting the everyday work right. BioBasicsBench makes that work a target for improvement: interpreting raw experimental data, checking controls and recognizing when a conclusion is not supported. Progress on these tasks could help scientists spend less time troubleshooting and more time deciding which experiments to pursue next. Tracking these capabilities will also help us understand their implications for biosafety and biosecurity.
2026
Life Sciences
Author
OpenAI
Keep reading
View all
A thousand fewer paths to root
Security
Nov 4, 2026
Accelerating engineering for the physical world
Research
Nov 3, 2026
LifeVal
Research
Nov 2, 2026
ai_agentllmbenchmark
@openai-research
原文 ↗
01
A three-step program
AI for Science
grantees
Jakob Macke
,
Daniel Gedon
, and
Stefan Wahl
are researchers at the
ELLIS Institute Tübingen
and the University of Tübingen, where they build AI tools for developing scientific models in neuroscience and beyond. In this guest post, they discuss how AI is starting to change scientific modeling.
In 1951, Cambridge University’s computer went out of action for six months. This was inconvenient for physiologist Andrew Huxley, who, alongside his colleague Alan Hodgkin, had just built a set of equations to describe how a particular squid nerve cell fired. The scientists wanted to understand the “action potential” of these nerve cells: the electrical pulses they use to communicate, which underlie everything brains do, from perception to intelligence. The only way to find out whether the equations worked was to solve them and compare the solutions to data.
Huxley had hoped to use the newfangled computer to solve the equations. Instead, he was forced to solve them using a hand-cranked mechanical calculator. It took him over three weeks.
The basic structure of their model was dictated by known facts about the cell. Hodgkin and Huxley modeled its membrane as an electrical circuit. One part of the model, however, was just an educated guess. In order to make the curve fit the data they had measured, they found that they had to raise one variable to the fourth power—that is, to multiply it by itself three times. At the time, they did not know why. They just knew that otherwise the model didn’t predict the observed data as well.
They did offer a hypothesis, however. They cautiously suggested that the variable in question, which governed how particular gates in the cell membrane opened, might require the fourth power because multiple charged particles in the membrane had to move into place before the gate would open.
Decades later, long after both Hodgkin and Huxley had won the Nobel Prize in Physiology or Medicine in 1963, another team
found the structure of a related potassium channel protein
. And they discovered that, yes, it had four identical pieces arranged
around a central pore
. The model was fit to the data in 1951. Even then, it predicted how fast the impulse travels relatively accurately. More than that, it turned out to have helped predict the structure of potassium ion channels themselves, which it had never been fit to.
Today, an AI system could plausibly find an equation like Hodgkin and Huxley’s in an afternoon, and perhaps it would fit the experimental data even more closely. But the point of the story is that simply knowing the fourth power did the job was not enough; what mattered was knowing that a simpler version of the model did not work. An AI system that merely hands you the model might give you an answer, but it does not give you understanding. Reproducing the data with a model is insufficient, because any number of models could do the same. What matters is being able to make claims about the real-world system itself, and ideally about other, similar systems. To do so, you need to know which features the model shares with other models that work, and which set it apart from models that don’t. As the mathematician Richard Hamming wrote in 1962, “The purpose of computing is insight, not numbers.”
A three-step program
More than 70 years on,
the Hodgkin-Huxley model
remains the standard description of electrical activity in neurons. The same kinds of equations, with different constants, describe nerve cells across the animal kingdom, from squids to humans, even though our evolutionary lineages separated more than half a billion years ago.
Even more important than its longevity, however, is what the Hodgkin-Huxley model tells us about how models work. Hodgkin and Huxley’s approach can be broken down into three steps, and most scientific modeling involves some version of them.
The first is
building
the model. This means deciding what the model is made of and what shape its equations take. In Hodgkin and Huxley’s case, that meant the parts of the electrical circuit—the membrane and the gates—and the form of the equations, such as raising the underlying variable to the fourth power.
The second is
constraining
the model—tuning its adjustable parameters to fit real measurements. For Hodgkin and Huxley, that included settings like how often the gates open under different voltages.
The third is
interpreting
the model—asking what the model says about the real system, such as suggesting that the fourth power meant the gate had four moving parts.
In practice, scientists
go through these steps repeatedly
. If the numbers fail to fit the measurements, or if the interpretation makes no sense, they go back and revise.
All three steps were once done by people, but AI systems are increasingly involved in each of them. Machines have arguably made the most progress in constraining models, so we’ll start there.
Splitting the bill, or how to constrain models
When Hodgkin and Huxley constrained their model by eye and intuition, they ended up with a single set of numbers. This told them that the model
could
match the data. It didn’t tell them whether other numbers would have worked just as well or even better.
Imagine you know that a group dinner added up to $120, but not who ordered what. Many different combinations of orders could have added up to that amount—perhaps everyone ordered a mid-priced burger, or perhaps one person had steak and lobster while everyone else had soup. Models of nerve cells work on the same principle: different combinations of gate parameters can produce near-identical electrical activity. Simply showing one combination that fits well hides all the other plausible parameter settings, and other combinations might have led to different predictions. This is true of almost any phenomenon being modeled. Many different sets of parameters can explain the observed data equally well.
An illustration of constraining the Hodgkin-Huxley model. (1) A measurement from a nerve cell. (2) How likely the measurement is for each combination of potassium and sodium channel settings. Darker means a better fit. The blue dot is interactive. (3) Simulations of the model at three settings, compared to the measurement (gray). Some fit well, while others do not.
The solution is a statistical method called Bayesian inference. It gives you every set of numbers that works reasonably well, as a probability distribution that takes into account their fit to the data and your prior knowledge about the parameters. The distribution takes the shape of a curve, with the best fits at the peak and the weaker ones at the fringes.
But this method has usually been impractical for complex models, because it involves calculating how likely your measurements would be under every possible set of numbers. In Hodgkin and Huxley’s model, that would still be feasible, but for anything more complicated, it would be computationally impossible. For example, if each of the thousands of ion channels opens and closes at random, you would have to work out every combination. Returning to our restaurant bill analogy: it would be easy to answer, “If person A had the burger and person B had the steak, what would the bill come to?” but much harder to show which combinations could have led to the final bill, especially as the number of people and menu options grows.
A different approach called
simulation-based inference (SBI)
overcomes this reverse-engineering problem with sheer computation. You pick random parameters, run the model, and record the output. You do it again and again—
perhaps 100,000 times for a model like Hodgkin-Huxley
, and millions of times for larger, more complex ones. Then you train a neural network on the results so that, given an output, it can return the range of parameters that could have produced it.
All SBI requires is that you can run the model: put in a set of parameters and get back the kind of data you would expect to see. A model you can run like this—parameters in, data out—is called a
simulator
, hence the name “simulation-based inference.” You never need to use a formula to calculate how likely your measurements are, which is the step that makes standard Bayesian inference often impractical. These days a simulator is usually a computer program, but it doesn’t have to be: Huxley, cranking through the equations by hand, was running one. Many simulators are easy to run forward but impossible to reverse. Some can’t even be written out as a tidy equation—their assumptions are built into the rules they follow—but they are models nonetheless. For many complex phenomena, from networks of neurons to colliding galaxies, simulators are often the only kind of model we have. SBI lets us constrain those models with data.
Running all those simulations is expensive and can take weeks. But once it is done, a trained network can perform inference on new data in seconds, without requiring further simulations. That speed has changed what’s possible. When
two neutron stars collide
, SBI can work out roughly where in the sky the collision happened, fast enough to point telescopes at it while the signal is still detectable. The same approach has been successfully applied to problems in
cosmology
,
structural biology
,
geophysics
, and many other domains.
And SBI has been applied to
nerve cell models
,
including the Hodgkin-Huxley model
. Because it returns a distribution rather than a single fit, we can see which parameters are well constrained by the data, where more than one explanation is plausible, and where more data—and thus another experiment—is most needed.
The bill, one level up, or how to build models
Before you can constrain a model, you need a model to constrain. The equations of the Hodgkin-Huxley model were not derived from first principles. Instead, the fourth power of the underlying variable emerged
empirically
from the scientists’ biological insight, and from finding that simpler versions didn’t work as well.
For most of the history of science, this is how models have been built. Scientists used their expertise and assumptions to write down a candidate model. Then they tested it, revised it, and tried again. The problem is that the space of possible models is essentially infinite, and a scientist proposing a handful of models from experience will very likely miss models that could have explained the observational data equally well.
AI systems can now help with, or even take over, this step. Given some data and a natural-language description of a system, an LLM-based agent writes code for a candidate model or a set of candidate models. Then it runs the model, constrains its parameters, compares the output to real-world data, and revises it, perhaps by modifying the form of an equation. This loop repeats until the model replicates the data accurately enough. Several such systems have appeared in the past two years.
1
The details vary, but all use an LLM to design the model itself. Until recently, this was thought to be something only humans could do.
But the fact that a model replicates data doesn’t mean that it provides insight. As with the restaurant bill, or the Hodgkin-Huxley parameters, many different combinations of parameters within a single model can provide the same output. The same is true one level up, for the model itself: many different
models
, with differently shaped equations, can explain the same data equally well. Run an agent twice on the same data, and it might return two different, and equally good, models. An agent that returns only one of them is hiding the rest, just as a single best-fit set of numbers does.
Building models by searching the space of all models. Left: Each dot is a possible variant of the Hodgkin-Huxley model; a darker background indicates models that are more likely. We keep both likely and unlikely models, as all models can carry insight. Right: Simulations of models built by AI systems from each cluster, and each cluster’s share of all models.
So it is better to treat building a model the same way we treat constraining one: you ask for the whole range of models that could plausibly explain the data.
2
The AI system proposes many candidates. Each is then weighted by how much it is supported by the data, and the ones that need a lot of flexibility to fit are given less credit than those that fit naturally. The weak candidates drop out, and the AI system proposes new variations on the stronger ones.
AI agents can thus provide a distribution of plausible models, rather than a single best fit, and that distribution tells you things a single model can’t. If every surviving model shares a feature, the data likely favors it. If the survivors disagree, that shows where the evidence is thin, and points researchers in the direction of the next experiment.
We shouldn’t get too far ahead of ourselves. So far, these systems have mostly been tested on rediscovering models that scientists already know. Evidence that they work on problems with
no
known answer is sparse. Still,
newer LLMs perform better at research tasks
than older ones, and we expect these systems to keep improving.
Lost in translation, or how to interpret models
Once a model has been built and constrained, it still has to be interpreted. What does it tell us about the real system being studied? Hodgkin and Huxley’s four-particle hypothesis came about because equations involving the fourth power are what you would expect if four separate things all had to happen before a gate could open. So they suggested that this might be what was happening inside the membrane. As we saw above, they turned out to be largely right.
That interpretation only carried weight because of what they had ruled out. If a simpler model had worked just as well, there would have been no reason to think the gate had several moving parts. It was the failure of the alternatives that allowed them to make that claim. There was some uncertainty about the exact number: Hodgkin and Huxley noted that a third or fourth power was needed and settled on four.
They wrote
that higher powers might even have fit slightly better, but that the potential “improvement was not considered to be worth the additional complication.”
The equations also let Hodgkin and Huxley reason out several other implications. For instance, the equations explained why nerve cells have a voltage threshold below which they don’t fire and above which they do, and why nerve pulses all have the same shape. They also predicted that a current rising slowly enough would never make the cell fire. Philosophers of science
call a theory “intelligible”
if it lets you work out its consequences without having to run it. That test works for a model with a handful of variables, like Hodgkin-Huxley. For a larger model with perhaps thousands of variables, the equivalent might be whether we can boil it down to a few crucial mechanistic ideas that are enough to rebuild it. The larger the model, the harder that becomes.
Of the three steps, interpretation is the hardest to hand over to AI. The goal is harder to pin down: we can measure how well a model fits the data, but not whether it leads to a better understanding of the system, and it’s not clear how to tell an AI system to make “understanding” its objective.
One proposal
is to judge an AI system’s understanding by seeing how well it predicts what a system does when you disturb that system. Another is to sidestep the question altogether, and instead ask whether the agent’s outputs
help human scientists better understand the problem
. One example is an AI system for
designing quantum experiments
. Rather than providing a single setup, it was trained to write a short program that generates the setups for a whole family of related experiments. A physicist can then read the underlying design principle straight off the program instead of reverse-engineering it from one experimental setup.
In both approaches, having access to a full distribution of suitable models, rather than a single best-fitting one, is crucial for interpretation. A single model cannot tell us which of its parts are actually constrained by the data, so it’s easy to read meaning into parts that are not. In Hodgkin-Huxley, as we’ve seen, the hypothesis about four particles only makes sense if you know that lower powers don’t work as well. A range of models also makes predictions more robust. Say you ask your models what happens when you disturb a system—for instance, by blocking all, or only some, of the sodium gates on the cell membrane. If all the models that fit the data return the same answer, as you would expect when every gate is blocked, then you can be fairly confident in it. If they disagree, as they might when only some gates are blocked, that can provide ideas for the next experiment.
Cells and scales
Most famous models, including Hodgkin-Huxley, are compact. They consist of a handful of variables and equations, small enough for a human scientist to write down and reason about. That’s no coincidence: they needed to be compact, because people had to build, constrain, and interpret them. Their smallness is also what made them intelligible. But as AI takes over more of the work, the size limit will start to lift. We can begin to model systems that are far too complex to write down by hand, even if we can no longer understand them just by reading the equations.
Brains are an obvious use case. The Hodgkin-Huxley equations describe a single nerve cell. But a fruit fly’s brain contains more than 100,000, and a human brain tens of billions. And it is the interplay between those neurons that gives rise to behavior and thought. For the fruit fly, we now have something that was impossible for previous generations of scientists: a complete wiring diagram of its brain, showing every neuron and how it connects to the others.
3
Researchers have started
building large-scale models on top of it
, including one that predicts the activity of neurons across the fly’s visual system. That opens up questions nobody could sensibly ask before. How does a whole brain turn its wiring into behavior? How much of an animal’s behavior can be predicted from its wiring alone? And which connections actually matter?
But at these scales it will not be possible to find a single correct model. Indeed, in most cases there is no such thing. As the statistician
George Box wrote
, “All models are wrong but some are useful.” The data will always be compatible with many models. The question will no longer be which model is the right one, but which properties all data-compatible models share, and which predictions can be derived from them. The most useful experiments will be those that rule out the greatest number of remaining models, even if they are not always the ones a human scientist finds the most interesting.
Scaling from one neuron to a whole brain. The fruit fly brain, drawn from its wiring diagram with 1, 10, and 255 neurons, and with 124,635 cells displayed. Rendered from the
male CNS connectome
(HHMI Janelia, University of Cambridge, MRC LMB, and Google Research), CC BY 4.0.
This isn’t unique to neuroscience. Climate scientists already work like this: they build many gigantic numerical simulations, each based on slightly different assumptions, and report their projections as the spread across all of them. Particle physicists at the Large Hadron Collider work in a similar spirit: when they look for a new particle, they don’t just declare whether a particular theory worked or not; they produce a map of which versions of the theory are ruled out by the data and which remain open.
The painstaking work Hodgkin and Huxley did—building the model, fitting each curve by eye, trying the powers that didn’t fit, and, in Huxley’s case, spending three weeks hand-cranking the mechanical calculator—is not something we want to go back to. But it was not wasted. Much of the insight that a model provides comes from the process of creating it; Hodgkin and Huxley understood what the model was telling them because they had built it themselves, and they knew what hadn’t worked. This understanding let them formulate their interpretation of gating variables—part of the discovery that ultimately earned them the Nobel Prize.
An AI system could now plausibly do much of the same work in an afternoon. But to offer understanding, it needs to deliver what the two scientists’ slow work produced: not just the model that fits, but an account of which parts are pinned down by the data, which are still open, which alternatives failed along the way, and what new puzzles were uncovered in the process. A scientific model may provide an answer, but it is the process of constructing it that provides the insight.
Footnotes
01
See
Romera-Paredes et al., 2024
;
Shojaee et al., 2025
;
Ma et al., 2024
;
Novikov et al., 2025
;
Xia et al., 2026
; and
Ziaei Bideh and Gryak, 2026
, for example.
02
See
Wahl et al., 2026
;
Mishra-Sharma, 2026
; and
Murphy, 2026
.
03
See
Dorkenwald et al., 2024
;
Schlegel et al., 2024
; and
Azevedo et al., 2024
.
Authors & acknowledgements
Contributors
Jakob Macke, Daniel Gedon, and Stefan Wahl
Acknowledgements
We thank Xander Balwit, Tom Chivers, and Siddharth Mishra-Sharma for review and editorial guidance.
llm科研
@anthropic-research
原文 ↗
OpenAI has announced a partnership with Hacktron to investigate vulnerabilities in its systems. Our researchers will work with OpenAI’s engineers to prioritize testing of connections with third-party services, with broader frontier-model access through Daybreak Red.
The collaboration follows
our work in OpenAI’s security bug bounty program
.
“The future of cybersecurity won’t be defined by who has the largest security team, but by who can best combine human creativity with AI-driven capabilities.
OpenAI is setting a strong example by putting frontier cyber models in the hands of security researchers, enabling them to uncover complex vulnerabilities at the boundaries between interconnected systems.
We’re excited to deepen our partnership with OpenAI, combining our offensive security expertise with their frontier models to help secure OpenAI and the broader software ecosystem.”
Zayne Zhang
CEO, Hacktron
Read
OpenAI’s announcement
for the full details.
红队产品动态
@hacktron-blog
原文 ↗