Why our coverage number cannot be inflated by typing
Any vendor can raise a coverage figure by adding a technique tag to an alert. It costs one line and nothing checks it. This note is the set of build gates we adopted so that ours cannot move that way, the one asymmetry that makes them work, and the place where the strongest gate can quietly fail to run.
A coverage claim is just a string until something refuses it
Most platforms map a technique identifier onto an alert as metadata. The identifier is a string in a struct. Nobody has to prove the detection works, nobody has to prove the identifier is real, and the published coverage total is whatever the sum of those strings happens to be. Coverage inflation is not usually dishonest — it is what happens by default when nothing refuses a claim.
So we treat a detection as a source artefact with mandatory metadata, and we let the build reject it. Each detection carries a stable identifier, an owner and version, a lifecycle status, severity and confidence, the techniques it claims, the data sources it needs, false-positive guidance, the response it pairs with, and the names of the tests that prove it. That last field is the one that does the work.
The keystone, and why one gate is not enough
The rule is that a detection in production status must name at least one backing test. On its own that rule is decorative: the field is a list of strings, so a detection can satisfy it by naming a test that does not exist.
The second gate closes that. It walks the whole module tree for real test functions, collects their names, and asserts that every test name cited by every detection is in that set. A fabricated or renamed test name fails the build. The consequence is the sentence we actually wanted:
Two gates for one claim is the general shape here. A rule that states a requirement, and a second rule that makes the first one impossible to satisfy on paper.
What fails the build, and what only warns
Twelve rules fail the build and four only warn. Where we drew that line is itself a judgment, so it is published rather than left implicit. The referential check described above is a separate test rather than one of the twelve, and it fails the build too.
Any detection not marked deprecated must claim at least one technique, and each claimed identifier must resolve in the matrix it declares. A typo cannot become coverage — T1071 mistyped is a build failure, not a covered technique.
The keystone plus its referential check, above. Also fatal: a missing description, an invalid severity, a confidence outside zero to one, a duplicate identifier, an unrecognised lifecycle status.
Data lineage should be declared, but a missing declaration is a documentation gap rather than a false claim, so it does not stop a release. This is the warning we are least comfortable with, because a detection with no feed is uncovered in practice.
A superseded detection should say what replaced it. Advisory, because the detection is already out of service and cannot contribute to a coverage total.
The asymmetry that makes it work
A separate gate reconciles the detection corpus against the published coverage declaration, and it is deliberately one-directional.
If a production detection claims a technique the coverage map marks uncovered, that is an error and the build fails. The corpus would be contradicting the published number, and we would rather find out at build time than in a customer's evaluation.
If the coverage map marks a technique covered but no documented, test-backed detection record exists for it, that is information — a backlog line, not a failure. It means we have the detection and owe it the paperwork.
The direction is the whole design. Overclaiming breaks the build; underclaiming produces a work queue. A symmetric gate would have been tidier and would have created a standing incentive to close gaps by editing the coverage map, which is exactly the move the gate exists to prevent.
Partial coverage is tracked as its own state to keep both directions honest. It does not count toward covered, so the backlog stays truthful; but a production detection claiming a partially covered technique is consistent rather than drift, because partial means a real detection exists with a scope we have written down.
A small decision with a real reason
The backlog is emitted in sorted order. It was not, once. The set is built from a map, and ranging a map yields a different order every run, so two identical corpora produced backlogs that appeared to differ. The backlog is read as a work queue, and a queue that reshuffles cannot be diffed. Sorting it costs one line and makes the output comparable between runs.
Lifecycle, not a switch
A detection moves through four states: experimental, test, stable, deprecated. What matters is that autonomy is gated on that state rather than granted with a flag. An experimental detection never acts on its own — it produces telemetry only. Only a detection in production status, which by the rules above must be backed by a real test, is eligible to drive an automated response.
This is the same principle as the approval gate described in the anatomy of an event, one level earlier: the question of whether a detection is trustworthy enough to act is answered by evidence in the build, before the question of whether a specific action is safe is answered at runtime.
Why publish the gates rather than the number
A coverage percentage tells you what a vendor counted. The counting rules tell you whether the number could have been otherwise. These gates are checkable claims: each one either fails a build or it does not, and the ones we call advisory are named as advisory. That is a smaller claim than a big number, and it is the one we can defend.
The check that cited tests really exist has to walk the source tree to find them, and a broken or partial checkout would make every citation look fabricated. To avoid that false alarm it skips itself when the walk finds implausibly few tests. That guard means the gate can silently not run in a degraded environment, which is the one soft spot we know about and have not yet closed. Reporting it is cheaper than pretending the guard is free.
The gates prove a named test exists and runs. They do not judge whether it exercises the detection meaningfully, and no automated rule can. A weak test satisfies the keystone exactly as well as a strong one.
The published coverage declaration describes the compiled platform, so content installed at runtime is treated as additive rather than as drift against it. Installed content passes the same metadata gates at install time, but it is not reconciled against the published number the same way.
Every check here runs in our build, reported by us. No independent party executes them. They make our number hard for us to inflate carelessly; they are not third-party attestation, and nothing on this site claims otherwise.
None of this widens what the platform detects. It constrains what we may say about it. What is measured, and what remains only compiled, is on the proof page.