Resources / Engineering notes
Engineering notes
Six decisions we would defend, and what each one cost.
Every one of these made the product harder to build or harder to sell, which is why they are worth writing down. A design note that describes only the upside is a feature announcement wearing a lab coat.
01
Absence gets its own visual language, and it cost us every dashboard convention
Four different facts can empty a panel: nothing happened, nothing is connected, you are not permitted to see it, or the field was never recorded. Every other platform draws all four as a zero, because a zero fits in a cell and looks like data. We gave absence its own treatment — dashed, faint, always label-carried, and deliberately outside the healthy-to-failing colour ramp.
The cost was real. It means no summary tile can show a single number without qualifying it, no estate-wide percentage survives review, and every chart needs a fourth state its library does not have. Reviewers consistently ask us to simplify it, and the answer is consistently no.
We hold the line because the alternative has a body count. An unmonitored subnet reported as clean for a quarter is not a display bug; it is the mechanism by which organisations discover they were never watching. Silence read as safety is the most expensive misreading in security operations, and it is produced by a rendering decision.
What we gave up: single-number dashboards, and the demo that fits on one slide.
02
The approval chain does not vote
Every world-changing action crosses a chain of independent guards, and a single hold stops the action. Not a majority, not a weighted score, not a confidence threshold. Three guards passing and one holding is stopped, and the guard that held is the sentence that appears in the review.
Scoring was the obvious design and we prototyped it. It fails in a specific way: the most cautious check is the one that most often disagrees with the others, so under any voting scheme it is the check that gets systematically outvoted. You end up with a safety mechanism that is reliably overruled precisely when it is doing its job.
The price is that the gate holds things people find obvious, and every hold is a person interrupted. We think that is the correct trade for a system that can isolate production infrastructure, and it is the reason teams leave the automation switched on after the first month rather than disabling it after the first surprise.
What we gave up: a fully autonomous response story, which is the thing most buyers say they want.
03
Analytics cannot act, and execution cannot judge
The layer that decides something is suspicious has no ability to do anything about it. It produces a proposal with its reasoning attached, and a separate layer decides whether that proposal becomes an act. Merging them would remove a network hop, several seconds of latency and a good deal of code.
Keeping them apart buys one property: you can audit a decision independently of the thing that acted on it. A system that both concluded and enforced has no record left that was not produced by the component under investigation — which is exactly the situation you are in when you most need an independent account.
The same reasoning drives the rule that collection does not decide anything. Sealing evidence and ruling on it are separate jobs, held by separate layers, so that the evidence survives disagreement about the ruling.
What we gave up: the lowest possible time-to-contain, by a margin we can measure.
04
Two timestamps on every event, forever
Every event carries when it happened on the machine and when we first saw it. Storing both costs space on the hottest path in the system, and every query has to declare which one it means — which is a papercut in roughly every piece of code that touches events.
The single-timestamp version cannot answer a question that comes up in every serious incident: was this quiet, or was it late? A host that was offline for six hours and a host that did nothing for six hours are indistinguishable if you kept one clock, and the difference is frequently the whole finding.
It also makes an entire class of argument tractable. Most disputes about whether a platform "had a gap" turn out to be disputes about which clock a dashboard was ordering by, and having both means the question is answerable rather than a matter of opinion.
What we gave up: storage, and a simpler query surface.
05
A refusal has no override, and we shipped it without one on purpose
When a guard refuses an action, no approver can release it. There is no elevated role that can, no emergency path, and no button that appears under sufficient authority. Changing the outcome means changing the policy that produced it, deliberately, in a different place, with that change itself recorded.
Every reviewer asks for the override. The argument for it is always the same and always plausible: a real emergency, a senior person, a decision that has to happen now. The argument against it is that an override exists on the worst day, in the hands of whoever has the most authority and the least context, and that its existence is exactly what an attacker with a compromised admin account is looking for.
So the design records the refusal as a decision already taken, and puts the remedy where deliberation happens rather than where panic does. It is the single most argued-about property of the platform, and it is the one we are most confident in.
What we gave up: an answer to "but what if we really need to", which we will keep being asked.
06
No uncited sentence ships
Every claim in a generated narrative links to the record that supports it. Not most claims, and not the ones a reviewer flagged — every one, enforced in construction rather than checked afterwards. A sentence that cannot cite its evidence does not get written.
This rules out a great deal of fluent prose. Summaries that characterise a week, sentences that connect two findings into a trend, anything that reads as insight rather than record — all of it fails the rule, and all of it is what people expect a report to contain.
The reason to accept a duller report is that the alternative degrades in one direction only. A report where most sentences are cited teaches its reader to skim the citations, and once they skim, the uncited sentence carries the same weight as the proven one. The rule is absolute because a partial version of it is worse than none.
What we gave up: the executive summary everyone asks for, in the form they ask for it.