AI 自动化侦察利用的宣称缺乏实证,检测工程需区分貌似合理与证据
ai_agentllm报告检测
Oct 12, 2026
Absence of Evidence: What an AI-Orchestrated Intrusion Harness Teaches Us About AI Detection Engineering
SecFlow, an AI-powered orchestration framework that uses AI models to automate reconnaissance and exploitation, provided a real-world lesson about why plausible is not the same as proof. Read SafeBreach expert Adrian Culley’s thoughts on the take-away for defenders
.
Summary
What happened:
Hunt.io researchers documented SecFlow, an AI orchestration harness run by a Chinese-speaking operator (“nie”) that used Claude, Qwen, and DeepSeek as interchangeable workers for recon, exploitation, and reporting.
The failure:
One worker claimed a successful Apache Shiro exploit that was never verified. Because the harness treats prior output as ground truth, more than 27 follow-on tasks were built on it, and all of them failed.
The capability is still real:
The same harness dumped LSASS credentials from a Beijing district government environment, accessed 800+ database accounts, and generated steganographically concealed web shells.
The infrastructure layer:
Separate Lumen Black Lotus Labs research, not linked to SecFlow, describes an “Infrastructure Quartermaster” that rents reconnaissance (QScan) and relay networks (Fast Labyrinth) to multiple China-nexus groups.
Attribution:
Neither report names China’s Ministry of State Security. Targeting is consistent with state intelligence collection, but formal attribution isn’t established.
The defender lesson:
AI-generated detection rules have the mirror-image problem. A rule that loads without errors is not a rule that fires. Only live execution against your controls proves coverage.
Author:
Adrian Culley, Offensive Security Expert & Senior Sales Engineer
“Absence of evidence is not evidence of absence.” Carl Sagan popularized the line, drawing on a much older statistical idea, and it’s one of the most misapplied sentences in science. It may also be the most important sentence for anyone putting AI anywhere near offensive or defensive security work.
The use of AI in security is moving quickly from supporting the process to taking action. Defenders use it to speed up detection engineering:
drafting detection rules
, mapping attack techniques, and writing playbooks. Attackers use it too, and increasingly they aren’t just prompting a chatbot. They’re wiring several models together into automated pipelines, known as AI harnesses, that plan, execute, and report on an operation with minimal human review.
That shift creates a new kind of risk for attackers and defenders alike. When a model says something worked, its output often looks exactly like success: well-formed, confident, and in the expected format. But looking right and being right are different things. A system that never checks the difference will keep building on mistakes it doesn’t know it made.
Recent research into a China-nexus intrusion framework called SecFlow shows how this plays out in the real world. The harness got genuine access to a government network. It also spent effort on more than 27 follow-up tasks that were doomed from the start, because one earlier step claimed a success that nobody ever verified.
The same failure is quietly showing up in defenders’ SOCs, where AI-generated detection rules can look ready to deploy without ever being proven to fire. To understand why, it helps to start with how an AI harness actually works.
PREFER TO LISTEN?
Check out episode 80 of The Cyber Resilience Brief, “The AI Harness That Never Checked Its Work,” where host Tova Dvorin is joined by SafeBreach expert Adrian Culley to unpack Hunt.io’s SecFlow research incident.
Listen now
.
First, what is an AI harness?
A large language model on its own answers one prompt at a time. An AI harness (sometimes called an orchestration framework or agent harness) is the software around the model that turns it into something that can complete multi-step work. A harness typically:
Breaks a goal into tasks
, often following a predefined workflow or playbook
Routes each task to a model
, sometimes spreading work across several models or several instances of the same one
Gives models tools
, like the ability to run commands, call APIs, scan networks, or read and write files
Carries state between steps
, so one task’s output becomes the next task’s input
Decides what happens next
based on what the previous step reported
Harnesses are now common in legitimate work, from coding assistants to security automation. They’re what lets AI move from answering questions to carrying out a sequence of actions with little human involvement.
That last point is also the weak spot. A harness is only as reliable as the results it passes along. If it accepts a model’s report of success without checking it, a single wrong answer can shape every step that follows. That’s exactly what happened recently with SecFlow.
SecFlow: A Harness That Never Checked Its Own Homework
Hunt.io’s research
describes SecFlow as an AI orchestration framework built by a Chinese-speaking operator using the handle “nie.” SecFlow farmed out reconnaissance, exploitation, and reporting to Claude, Qwen, and DeepSeek instances, which ran as interchangeable workers.
The architecture was straightforward. A master workflow file (a SKILL.md policy file) pushed each target through initialization, reconnaissance, exploitation, post-exploitation, and reporting. Workers shared a file system, so one worker’s output became the next worker’s input.
That design is where things broke down. At one point, a worker reported that it had successfully exploited an Apache Shiro vulnerability. No one confirmed it. Because the harness treated the prior worker’s output as ground truth, the unverified success was carried into more than 27 subsequent tests, all of which failed because the original exploit had never worked.
The system wasn’t lying, exactly. The output was plausible: a well-formed “exploitation successful” message in the shape of a good result. But no one, human or machine, went back to check what mattered. Did a shell actually open? Did a single byte of data leave the box?
Plausible-looking success was treated as verified success.
The Mirror Image: The Hidden Risks in AI Detection Engineering
What is AI detection engineering?
AI detection engineering is the use of large language models (LLMs) and other AI tools to build and maintain the rules that spot malicious activity in a security environment. Typical tasks include writing Sigma or SIEM queries, mapping detections to
MITRE ATT&CK
techniques, tuning thresholds, and drafting response playbooks.
How does this incident relate to AI detection engineering?
This isn’t only an offensive AI problem. It’s the same failure detection engineers face, seen from the defender’s side.
Ask an LLM to write a Sigma rule for a specific attack technique and you’ll have one in about four seconds. It will almost certainly be valid YAML, its field mappings will look plausible, and it will probably load into your SIEM without a syntax error. None of that tells you whether it fires when the real technique runs against your real endpoint configuration. That’s because syntactically correct is not operationally correct. A rule that passes is not a rule that detects.
AI-assisted
detection engineering
makes it easy to produce a large volume of rules that pass. That can create a false sense of coverage that is arguably worse than having no rule at all, because now someone believes they’re covered. “We have a rule for that” and “that rule fires when it should” are two very different sentences, and only the second describes a security control.
This is where Sagan’s line cuts both ways. If a rule never fires because no one tests it against a live execution, that’s not evidence the technique isn’t happening in your environment. It’s evidence you never looked properly.
The absence of an alert is not evidence of the absence of an attack.
SecFlow’s Targets & Tradecraft
Hunt.io researchers identified the SecFlow operator by the handle “nie,” which appeared consistently across a model service namespace, proxy credentials, and a family of domains hosting private relays for Claude, Qwen, and DeepSeek.
The targeting spanned six countries, mostly government, political, and telecom organizations, including:
A Fengtai District Government office automation environment in Beijing, China
An attempted intrusion into the Kuomintang Party History Archive in Taiwan
A web shell attempt against Indonesia’s Ministry of Foreign Affairs
Port scanning against an Afghan telecom’s SMPP infrastructure
That list reads like intelligence collection, not fraud. The tradecraft was real, too. SecFlow included a dedicated capability called Glutton for generating web shells in Java, .NET, and Node, delivered steganographically: XOR-encrypted payloads hidden in the RGB channels of PNG files to evade static signatures.
The confirmed Fengtai compromise included LSASS credential dumping and extraction of 822 user account records from its OA database. The exploit chain covered eight named CVEs, including Shellshock, Log4Shell, Spring4Shell, Ghostcat, and the Apache Shiro deserialization flaw (CVE-2016-4437) behind the false success. The oldest, Shellshock, dates to 2014, yet these flaws are apparently still useful against unpatched estates in 2026.
The Infrastructure Quartermaster: Recon & Relays as a Service
Two separate pieces of research, published about a week apart in late summer 2026, describe what may be two halves of an emerging model. Neither report links the two operations.
The first is Lumen Black Lotus Labs’ work on what they call the
Infrastructure Quartermaster
, a private technical entity whose administrative activity Lumen traced to Nanjing. It doesn’t carry out intrusions itself. It builds and rents out the plumbing, including:
QScan:
automated reconnaissance at scale
Fast Labyrinth:
an encrypted relay network running on legitimate commercial proxy subscriptions
QT Proxy:
an admin panel managing it all
Multiple downstream
Chinese threat groups
appear to use this shared infrastructure. Instead of every crew building its own obfuscation network, a specialist quartermaster handles target discovery and traffic laundering for multiple customers. QScan profiled research universities (including aerospace and bioinformatics departments), government, defense, and critical infrastructure targets across the US, UK, Europe, and Asia-Pacific, rotating subnets on a 30-day cycle to avoid detection. Lumen tracked it for about a year before publishing.
A Careful Word on Attribution
Attribution isn’t for the faint-hearted, so let’s be precise. Neither report names China’s Ministry of State Security (MSS). Lumen describes the Quartermaster as China-nexus and its downstream users as Chinese threat groups. Hunt.io stops short of naming an intelligence service. There’s no government attribution statement and no MSS branding on the infrastructure.
What we can say with more confidence is that the combined targeting (defense networks and research universities scanned by QScan; government systems, a party political archive, a foreign ministry, and telecom infrastructure targeted by SecFlow) looks like a state intelligence collection list, not a criminal one. The fairest framing: state-nexus, moderate confidence based on targeting pattern; formal intelligence-service attribution not established in public reporting.
Dangerous & Sloppy at the Same Time
SecFlow is real access and real waste. The harness that pulled LSASS dumps out of a government network also burned effort on more than 27 doomed tasks because of one unverified claim. The capability is real. The discipline inside it is not.
That may be the most important strategic point in the story. People want
AI-driven APT operations
to be either terrifying and flawless or overhyped and harmless. The evidence says neither. It’s a real capability uplift for a
state-nexus actor
: faster reconnaissance, faster exploit customization, and faster web shell generation with evasion built in, all running inside a harness that inherits every epistemic weakness of the underlying models.
That combination is what defenders need to plan for, not a caricature of either extreme.
What Security Leaders Should Do Differently
Whichever side you’re on, the takeaway is the same: don’t trust a system’s self-report of success.
If you’re a detection engineer and an AI assistant hands you a rule, a mapping, or a playbook, treat it as a draft. It isn’t a verified control until you’ve executed the actual technique against it and watched it fire. If you’re
red teaming with AI tooling
, apply the same discipline in reverse: don’t let a model’s “exploit successful” claim flow into your next phase without confirming impact.
Three Immediate Questions for SOC Leads
Has it actually been executed?
For every detection rule generated or substantially assisted by an LLM in the last six months, has anyone run the corresponding technique against it and confirmed that it fires? Not reviewed. Executed. Right now, this is one of the industry’s quiet problems.
Are you separating “deployed” from “validated”?
Is there a process gap where a rule sitting in the SIEM gets reported upward as “we’re covered”?
When a detection stays silent, what do you conclude?
When a rule stays silent in production, do you treat that as “we’re clean” or “we don’t know”? These are very different postures, and only one of them is honest.
That third question is Sagan’s line in practice. A quiet SIEM isn’t proof of a quiet network.
Validate Continuously, Not Once
The defenders who get this right run
continuous validation
. They execute technique chains against their own controls on a recurring schedule instead of checking that a rule exists and looks reasonable. That’s the validation layer of
Continuous Threat Exposure Management (CTEM)
in practice, and it’s how
SafeBreach customers close this gap
: confirming that a rule fires against a live execution, not just that it compiles.
The principle holds whatever platform does the testing. The question to take back to your team is:
Can we prove, with evidence, that our top 10 detections fire against the real technique today, not six months ago when the rule was written?
Ask the Same Question of Your AI Offensive Tooling
If you’re evaluating AI tooling for offensive testing, red team automation, adversary emulation, or anything that generates and executes attack logic, ask it the same thing you’d ask a detection rule:
Does it verify its own claimed outcomes, or does it just report what it was told to report?
SecFlow shows what happens when the answer is the latter. Twenty-seven-plus wasted cycles is almost funny, until you remember that the same architecture pulled a real LSASS dump out of a real government network.
What to Watch Over the Next 12 Months
Whether harness architectures mature.
If the next version of something like SecFlow adds a verification step, a worker that independently confirms exploitation before tasking follow-on work, the wasted-cycles problem disappears quickly. That would be a meaningful capability jump.
Whether the Quartermaster model spreads.
Shared, rented recon and relay infrastructure serving multiple downstream crews is a scaling pattern. If it works for this operator, others will copy it.
Whether defensive AI gets the same scrutiny.
Hunt.io gave us a rigorous failure-mode analysis on the offensive side. There’s comparatively little public scrutiny of whether AI-assisted
detection engineering
is quietly shipping the same unverified-success problem into SOCs.
The Bottom Line
Absence of evidence is not evidence of absence. That’s true for the state actor you haven’t formally attributed yet, and it’s true for the detection rule you’ve never actually fired. In both cases, the only honest answer to “are we covered?” is one you’ve already tested.
SafeBreach runs real attack techniques against your security controls, safely and continuously, so you can see which detections fire, which ones fail silently, and where the gaps are. That includes detections written by your team and ones drafted with AI.
Request a demo
to see how SafeBreach helps security teams move from “we have a rule for that” to “we’ve tested it.”
About the Author
Adrian Culley
is Head of Engineering, EMEA for SafeBreach. He has over 35 years of computer security experience, particularly specialising in security operations, artificial intelligence, offensive security, threat intelligence, and malware — in short, cyber resilience. He is a core member of the OWASP Large Language Model Top Ten risks working group.
He is a graduate of the Information Security Group at Royal Holloway, University of London, and an occasional visiting lecturer there. Adrian was previously a Detective with Scotland Yard, working around the world with their Special Operations Directorate, and latterly served with the Computer Crime Unit there. He has also worked directly for the UK Central Government in a global role. He worked for Palo Alto Networks for five years as a Consulting Engineer and Head of Industry 4.0. Previously, he has also worked for PwC, Trellix, Cybereason, Guidance Software, AccessData, and Damballa. Adrian has a strong background in digital forensics and has worked with government, NGO, military, intelligence, law enforcement, and commercial clients around the world.
Frequently Asked Questions
What is AI detection engineering?
AI detection engineering is the use of large language models (LLMs) and other AI tools to build and maintain the rules that spot malicious activity in a security environment. Typical tasks include writing Sigma or SIEM queries, mapping detections to
MITRE ATT&CK
techniques, tuning thresholds, and drafting response playbooks. AI can cut rule creation from hours to seconds. But a rule AI generates has only been checked for syntax, not proven to work. It becomes a real security control only after the matching attack technique has been run against it in your environment and the rule has fired.
What is an AI harness?
An AI harness is the software layer that wraps one or more large language models (LLMs) so they can complete multi-step work. It breaks a goal into tasks, assigns them to models, gives those models tools like command execution or network scanning, and passes each step’s output to the next. Harnesses power legitimate tools such as coding assistants and security automation, and threat actors now use them too.
What is SecFlow?
SecFlow is an AI orchestration harness documented by Hunt.io researchers. Built by a Chinese-speaking operator using the handle “nie,” it assigned reconnaissance, exploitation, post-exploitation, and reporting tasks to Claude, Qwen, and DeepSeek instances running as interchangeable workers, with each worker’s output feeding the next through a shared file system.
Why did SecFlow waste more than 27 follow-on tasks?
One worker claimed it had exploited an Apache Shiro vulnerability, but the claim was never verified. Because the harness treats prior output as ground truth, more than 27 subsequent tasks were built on that false success, and all of them failed.
What is the “Infrastructure Quartermaster”?
It’s Lumen Black Lotus Labs’ name for a private technical entity, likely based in Nanjing, that doesn’t carry out intrusions itself but rents infrastructure to multiple
China-nexus threat groups
. That includes QScan for automated reconnaissance, the Fast Labyrinth encrypted relay network, and the QT Proxy admin panel.
Is SecFlow attributed to China’s Ministry of State Security?
No. Neither the Hunt.io nor the Lumen reporting names the MSS. Lumen describes China-nexus infrastructure and Hunt.io a Chinese-speaking operator; the targeting in both is consistent with state intelligence collection. That supports a reasonable inference of state alignment, not a confirmed intelligence-service attribution.
How does this relate to AI-assisted detection engineering?
It’s the mirror-image failure. LLMs can produce syntactically valid detection rules in seconds, but a rule that loads into a SIEM isn’t proven to fire against the real technique in your environment. Treating plausible output as verified coverage creates a false sense of security.
How can security teams validate AI-generated detection rules?
Execute the corresponding attack technique against your controls and confirm the rule fires, then keep doing it on a recurring schedule.
Continuous security validation
, part of a
CTEM program
, separates rules that are deployed from rules that are proven.
What should I ask vendors of AI-driven offensive testing tools?
Ask whether the tool independently verifies its claimed outcomes, such as confirming that a shell opened or data actually left the system, or whether it simply reports what it was told to report.
You Might Also Be Interested In
Podcast
Videos
Podcast: Anthropic’s CISO Guide to Agentic AI: Your Next Insider Threat Is an AI Agent
Listen Now
Partner Solution Brief
SafeBreach & Anvilogic
Read More
China State-Sponsored Cyber Threat Actors: A Comprehensive Guide
Read More
Get the latest
research and news