Four Marks That Separate What the Architecture Measures From What the Architecture Merely Finds Easy to Count.
Most architectures track what is easy to measure rather than what is structurally true.
This is not a failure of intelligence. It is a failure of instrumentation, and it is nearly universal, because the easy number is always available first, always cheaper to produce, and always more comfortable to report. The structurally true number frequently requires work the organisation has not yet built the capacity to do, and arrives later, if it arrives at all. Given a choice between an available number and a true one, most dashboards quietly choose available, and no one ever decided this on purpose. It accumulated, metric by metric, over years.
The architect who has never personally identified and removed a vanity metric from their own dashboard is still being led by one, even now, even having read this sentence. The absence of a known vanity metric is not evidence of a clean dashboard. It is usually evidence that the audit has never been performed.
This essay introduces the framework. The Instrumentation Doctrine. Four marks. Structural Metric. Leading Indicator. Lagging Indicator. Vanity Metric to Purge. Every number the architecture tracks belongs to one of these four categories, whether the architecture has classified it or not. The discipline is to classify deliberately, keep the first three in view, and find the fourth and remove it.
This is Volume X’s second framework, and it pairs directly with Monday’s Contingency Architecture. Detection, in that framework, depends entirely on the quality of what the architecture is actually measuring. An architecture with poor instrumentation cannot detect an emerging risk quickly, no matter how well the later stages are engineered, because detection is instrumentation applied under pressure.
Why the convenience framing fails
Three reasons the instinct to measure what is easy fails the architect.
First reason. Ease of measurement and structural relevance are uncorrelated. Some of the most structurally important facts about an architecture, the true health of a key relationship, the actual depth of a moat, the real state of team morale under strain, are difficult to reduce to a number and are therefore quietly dropped from the dashboard in favour of whatever can be pulled from a system automatically. The dashboard becomes a record of what the software makes easy, not a record of what the architecture needs to know.
Second reason. Easy metrics tend to be lagging by construction, because the easiest thing to count is usually a completed event, a closed deal, a shipped feature, a filed report. An architecture that instruments only what is easy therefore instruments only the past, and discovers problems only after they have already produced the lagging number that finally reveals them. By then the cost has already been incurred.
Third reason. Convenient metrics accumulate reputational inertia. A number that has been on the dashboard for years, reported quarterly, cited in board packs, becomes politically difficult to remove even after everyone privately suspects it has stopped meaning very much. The metric survives not because it is useful but because removing it would require someone to explain, in a room, why it was ever there.
The structural framing corrects all three. It asks not whether a number is easy to produce but what category of truth it belongs to, and it treats the presence of an unclassified number on the dashboard as an open question rather than a settled fact.
Mark One. Structural Metric
A Structural Metric is a number that describes the underlying architecture itself, not the architecture’s recent activity.
Structural Metrics change slowly. They describe things like customer concentration, the ratio of recurring to one time revenue, the proportion of institutional knowledge held by a single irreplaceable person, the actual duration of the sales cycle rather than the target duration. They are the numbers that would still be true, roughly, if the organisation stopped all activity for a month. They describe the shape of the thing, not its recent motion.
The architect’s discipline on structural metrics is to track them even though they rarely move quarter to quarter, precisely because the rare movement is the signal that matters most. A structural metric that shifts materially in a single quarter is usually reporting something more important than any of the faster moving numbers on the same dashboard.
Four properties to audit at the structural layer.
First. Slow cadence, deliberately tolerated. The metric is reviewed quarterly or annually, not weekly, and the architecture resists the temptation to demand more frequent movement from a number that is structurally supposed to be stable.
Second. Genuine descriptive power. The metric, if it changed sharply, would actually change how the architect thinks about the business, not merely register as an interesting data point.
Third. Independence from any single team’s incentive. Structural metrics are typically owned at the architecture level, not by a department whose quarterly bonus depends on the number moving in a particular direction.
Fourth. Resistance to gaming. The metric is difficult to improve through short term manoeuvring, because its value depends on the actual shape of the underlying business rather than on any single period’s activity.
The dominant failure mode at the structural layer is absence. Most dashboards have none. The architecture measures activity exhaustively and has never once written down the handful of numbers that describe its actual shape, because those numbers do not update often enough to feel like they belong on a dashboard designed around weekly rhythm.
Mark Two. Leading Indicator
A Leading Indicator is a number that moves before the outcome it predicts, giving the architecture time to act while the outcome is still avoidable.
The value of a leading indicator is entirely in its timing. A number that correlates perfectly with a future outcome but only becomes visible the week before the outcome occurs has almost no leading value, because there is no longer time to act on it. The discipline is to find indicators that move weeks or months ahead of the outcome, even if the correlation is less than perfect, because imperfect early warning beats perfect late warning every time the warning is meant to change the outcome.
Four properties to audit at the leading layer.
First. Genuine lead time. The indicator moves meaningfully before the outcome, tested historically rather than assumed. An indicator claimed to be leading that, on inspection, moves at the same time as the outcome is not leading at all.
Second. Actionability within the lead window. The architecture actually has a mechanism it can deploy in the time the indicator provides. A leading indicator with no corresponding response capability is an early warning of a problem the architecture still cannot solve any faster.
Third. Ownership assigned to whoever can act on it, not merely whoever can report it. The person watching the leading indicator needs the authority to do something with it inside the lead window.
Fourth. Periodic recalibration. Leading relationships between numbers erode as the business changes. An indicator that led reliably three years ago may have stopped leading anything, and the architecture needs a discipline for testing this rather than assuming past correlation still holds.
The dominant failure mode at the leading layer is false confidence in a stale correlation. The architecture continues watching an indicator that once predicted an outcome well, unaware that the underlying relationship has quietly broken, and is therefore blindsided by an outcome the dashboard should have flagged and did not.
Mark Three. Lagging Indicator
A Lagging Indicator is a number that confirms an outcome has already occurred.
Lagging indicators are not inferior to leading indicators. They serve a different, necessary purpose: verification. Revenue, churn, profit, these are lagging by nature, and the architecture needs them precisely because they are the ground truth against which every leading indicator and every structural metric is eventually checked. The error is not in tracking lagging indicators. The error is in tracking only lagging indicators and mistaking a full dashboard of confirmations for a functioning instrumentation system.
Four properties to audit at the lagging layer.
First. Accuracy of confirmation. The number genuinely represents the outcome it claims to measure, without definitional drift that makes quarter to quarter comparison misleading.
Second. Explicit pairing. Every lagging indicator on the dashboard is paired, deliberately, with at least one leading indicator that should have predicted it. An unpaired lagging indicator is a confirmation with no corresponding early warning.
Third. Post hoc review discipline. When a lagging indicator delivers a surprise, the architecture reviews which leading indicators should have caught it and did not, feeding directly into the recalibration named above.
Fourth. Resistance to becoming the only conversation. Lagging indicators, because they are the most concrete and the most comfortable to discuss, tend to dominate review meetings at the expense of the leading and structural metrics that would have prevented the surprise the lagging number is now reporting.
The dominant failure mode at the lagging layer is the well confirmed surprise. The architecture has excellent, accurate, timely lagging indicators, and is still routinely surprised, because the dashboard confirms outcomes beautifully and predicts almost nothing.
Mark Four. Vanity Metric to Purge
A Vanity Metric is a number that moves in a direction that feels good, correlates with nothing structurally important, and survives on the dashboard because removing it would require an uncomfortable conversation about why it was there.
Vanity metrics are not always obviously frivolous. Some are quite sophisticated looking. A number can be precisely calculated, professionally presented, and still be a vanity metric, if its actual relationship to the architecture’s structural health has never been tested and would not survive the testing.
Four properties to audit at the vanity layer, which together form a test rather than a definition.
First. Untested correlation. No one has actually checked whether this metric’s movement corresponds to any outcome the architecture cares about. It has simply always been reported.
Second. Emotional rather than decisional utility. The metric’s primary function is to make a review meeting feel good or bad, rather than to change any decision that gets made in that meeting.
Third. Political protection. The metric has an internal sponsor whose function or history is associated with it, making removal socially costly independent of the metric’s actual value.
Fourth. Survivorship through inertia alone. When asked directly why the architecture tracks this number, the honest answer is some version of because it has always been tracked, rather than any structural justification.
The dominant failure mode at the vanity layer is not tracking a vanity metric. It is failing to ever ask the question that would identify one. The metric survives indefinitely, not through any deliberate defence, but because no one in the room has been assigned the job of asking whether it deserves its place.
The integration
The four marks form one Instrumentation Doctrine. Structural Metrics describe the shape. Leading Indicators warn ahead of outcomes within the shape. Lagging Indicators confirm what actually happened. Vanity Metrics are the accumulated residue of a dashboard that has never been audited against the other three.
A well instrumented architecture has few structural metrics, watched patiently. A moderate number of leading indicators, each paired to an action the architecture can actually take. A disciplined set of lagging indicators, each paired back to the leading indicator that should have predicted it. And, ideally, zero vanity metrics, achieved not by never having any but by removing them as soon as the audit identifies them.
This connects directly to Monday’s Contingency Architecture. Detection depends on leading indicators that genuinely lead, watched by someone with the authority to act inside the lead window. An architecture whose dashboard is dominated by lagging indicators and vanity metrics will detect its risks late, regardless of how well the containment, decision, and recovery stages have been engineered, because the detection stage was never actually watching the right numbers.
The audit cadence
The Instrumentation Doctrine is reviewed twice yearly, and the review is performed by asking, of every single metric on every dashboard the architecture maintains, which of the four marks it belongs to.
The architect asks four questions. Which numbers on this dashboard would we still need if the business stood still for a month, and are those structural metrics being watched at the patient cadence they deserve. Which numbers claim to be leading, and when did we last test whether they actually lead anything. Which lagging numbers have no leading indicator paired to them, meaning we will be surprised by them again. And which number on this dashboard, if we are honest, has never once changed a decision, and exists purely because it has always existed.
The fourth question is the one architects avoid, because answering it honestly usually identifies a metric someone in the room has personally championed. The Voice Architecture’s restraint component (Volume VI) is relevant here in an unexpected way. The discipline of refusing what does not belong applies as much to a dashboard as it does to a sentence. A metric that does not hold its place under honest questioning is removed, regardless of how long it has been there or who will be uncomfortable seeing it go.
A note on the architect’s own resistance
The architect reading this essay is likely, at this moment, confident their own dashboard is reasonably clean. This confidence is itself worth examining, because it is the exact confidence every architecture with an unaudited vanity metric also holds, right up until the audit is actually performed.
The honest test is specific, not general. Pick the single metric on the architect’s own primary dashboard that has been there longest, that no one has questioned in at least two years, and that would be mildly uncomfortable to defend if a sceptical peer asked, directly, why it matters. That metric is the one to test first. Most architects, performing this test honestly for the first time, find at least one number that fails it.
This is not a mark of a poorly run architecture. It is the ordinary condition of any dashboard that has existed long enough to accumulate metrics faster than it has removed them. The distinguishing discipline is not having a perfect dashboard. It is being the kind of architect who actually performs the test rather than assuming, indefinitely, that the test would come back clean.
A closing observation
The dashboard the architect has never audited is not neutral. It is actively shaping decisions, quietly, in the direction of whatever is easiest to measure rather than whatever is structurally true. This shaping is invisible precisely because the dashboard looks complete. It has numbers. The numbers move. Something appears to be measured.
The Instrumentation Doctrine does not ask the architect to measure more. It asks the architect to know, for every number already being measured, which of the four marks it belongs to, and to have the discipline to remove the one mark that survives on inertia alone.
Doctrine close
The Instrumentation Doctrine separates what the architecture measures from what the architecture merely finds easy to count.
Structural Metric. Leading Indicator. Lagging Indicator. Vanity Metric to Purge. Reviewed twice yearly, every number classified, the fourth mark removed on discovery rather than tolerated on inertia.
The Circle is a room of architects who have each, personally, found and removed a number from their own dashboard that had survived purely because no one had asked why it was there.
Structure. Access. Positioning.
The Architect Circle is application only. Two hundred and fifty seats globally. createyourownmarket.com.au.