The Morpheus Challenge
The Morpheus Challenge: judge us on evidence
Every vendor in this market says their AI learns. Here’s a test any of them can take — including us.
The Morpheus Challenge is a free, vendor-neutral evaluation protocol for agentic SOC platforms. It has two tests. Test one runs the same alert ten times and diffs both the verdict and the reasoning. Test two takes the evidence away mid-investigation and records what the system does when it cannot know. You run it yourself, on your own alerts, against any platform on your shortlist. Morpheus is the accountable agentic SOC platform, and this is the test we want to be judged on.
Demos are built to be passed. Both tests here are built to be failed, which is what makes the results worth reading. Neither one requires a proof of concept, a data-sharing agreement, or a vendor in the room.
Protocol v1.0 · Published 10 August 2026
Test one
Run it ten times
Same alerts, ten runs, any platform. Diff the conclusions and the reasoning. Consistency is what deterministic learning looks like; variance is what you’ll be explaining to your auditor.
What you do
Pick five alerts from your own queue where you already know the right answer. Run each one ten times under the same conditions, with no operator intervention between passes.
What you record
Capture the disposition, the evidence cited, and the reasoning path for every run. The disposition alone hides the interesting part. Two runs can reach the same verdict by very different routes.
What it separates
A system replaying validated experience returns the same answer the same way. A system regenerating its reasoning each time drifts, and that drift is what you will be asked to account for when an auditor picks one closed alert at random.
Test two
Cut the evidence
Kill a log source, time out a tool. Watch what each system does when it can’t know. Morpheus will grade its evidence, return “Open — a human should look,” and show you the audit trail. We’d love to know what the others do; so would you.
Starvation case one: a log source stops answering
Disable a connector or revoke a token partway through an investigation. The system now holds an incomplete picture and has no way to confirm it. Here is what to watch for, and what Morpheus does.
Failure
The evidence goes missing
A branch of the investigation can no longer be resolved. The tempting behavior, and the one worth testing for, is a confident verdict produced from what is left.
Detection
Evidence is graded, not assumed
Morpheus grades every piece of evidence it holds as CONFIRMED, INFERRED, or GAP. A gap is recorded as a gap. It does not get quietly interpolated into a conclusion.
Recovery
Open, routed to a human
“Open — a human should look” is a first-class disposition, not an error state. Weak evidence never closes an alert. When Morpheus is uncertain, it defers to a human.
Starvation case two: an enrichment tool times out
Point an enrichment call at an endpoint that will hang, or throttle it until it fails. This is the quieter version of the same question, because a timeout looks like a slow success right up until it doesn’t.
Failure
The query returns nothing usable
An empty result and a failed result are different facts. Systems that treat them the same will report an absence of findings as a clean bill of health.
Detection
The query layer fails open
The deterministic query layer reports the failure rather than substituting a plausible result, and rule sanity-checking catches queries that came back structurally wrong.
Recovery
Escalate with the gap attached
The investigation escalates carrying its own evidence ledger, so the analyst inherits the reasoning and can see exactly which branch was never resolved.
The point of the test
Performance on a good day is the easiest thing in this market to demonstrate, and every platform in your evaluation folder can demonstrate it. The alert that decides whether you can defend the deployment is the one where the honest answer is “I don’t know.” Grade AI on its worst day, not its best. When Morpheus is uncertain, it defers to a human.
The protocol
Five steps, half a day, no vendor required
The download is ungated. Take it into every demo on your shortlist and score them the same way.
What’s in the download
A short PDF, no form, no email required. It is written so a senior analyst can run it without help from any vendor, including us.
- Setup. What to instrument before the first run, and what to hold constant so the results mean something.
- Alert-set guidance. How to choose five alerts that are representative rather than flattering, including at least one you expect any system to get wrong.
- The scoring sheet. A one-page grid covering verdict consistency, reasoning consistency, evidence citation, and behavior under starvation.
- What to log. The fields to capture per run so two platforms can be compared side by side weeks later.
- How to read the results. What a passing pattern looks like, and the failure signatures worth raising in a procurement review.
How to use this
We will run both tests live, on your alert set, in any competitive evaluation. Bring the protocol to every vendor demo, then book ours. If a platform on your shortlist will not sit for either test, that result is also worth writing down.
Related
For the governance model that makes an “Open” disposition safe to ship, see the accountable agentic SOC and what an AI SOC does when it’s wrong. Every capability named on this page carries a ship date in the Morpheus release history.
faqs
Frequently Asked Questions
What the challenge is, who it’s for, and why there is no accuracy number on this page.
What is the Morpheus Challenge?
The Morpheus Challenge is a free, vendor-neutral evaluation protocol for agentic SOC platforms. It has two tests. Test one runs the same alert ten times and diffs both the verdict and the reasoning, which measures whether the system is repeatable. Test two removes evidence mid-investigation, by killing a log source or timing out a tool, and records what the system does when it cannot know. The protocol is an ungated download and names no vendor.
Do I need D3 to run it?
No. The protocol works against any platform that will let you re-run an alert, and most teams run it across a whole shortlist and compare the scoring sheets afterward. We will also run both tests live on your own alert set in any competitive evaluation.
Why is there no accuracy number on this page?
Because an accuracy figure describes the good day, and the good day is not where SOC risk lives. Any vendor in this category can produce a strong number on a curated alert set. What an auditor asks about is the closed alert that should not have closed. The two tests here measure repeatability and behavior under missing evidence, which is what a percentage cannot show you.
What does Morpheus do when it cannot reach a verdict?
It grades the evidence it holds as CONFIRMED, INFERRED, or GAP, then returns “Open — a human should look” as a standard disposition with the full audit trail attached. Weak evidence structurally cannot close an alert. When Morpheus is uncertain, it defers to a human.
Watch it fail toward a human
Book 30 minutes. We’ll run the ten-run test and the evidence test on your alert set, live, and you can keep the scoring sheet.
