A rubric version cut without a public changelog produces a score a customer can’t fully trust — if the rubric changes between audits without disclosure, a score movement becomes ambiguous: did the site improve, or did the measurement shift? The changelog is how Obaron makes that distinction possible. It is a measurement contract, not a release note.
Docs Readiness Audit v5.0 shipped on April 30, 2026, with a public changelog and a customer crosswalk table. This post covers what those artifacts commit us to.
What makes a score trustworthy
A Docs Readiness Audit score is a number between 0 and 100. For that number to be actionable — for a team to put engineering effort behind moving it — they need to know it’s measuring what we say it’s measuring, consistently, over time.
Two things break that trust:
-
Model variance. If the same page scores differently on two runs without anything changing, the score isn’t measuring the page — it’s measuring noise. The Obaron rubric is deterministic. Same URL, same raw HTML, same score. No sampling variance. No LLM non-determinism in the scoring path.
-
Silent rubric changes. If the rubric changes between audits without disclosure, a score movement becomes ambiguous: did the site get better, or did the rubric shift? Both interpretations are possible if you can’t see the diff.
The changelog addresses the second problem. Every version cut is documented publicly, with the structural delta isolated from any content or wording changes. Customers who ran an audit under v4.3 can look at the v5.0 changelog and determine exactly which signals moved categories and which weights changed.
What the crosswalk does specifically
The crosswalk is the side-by-side table: every category, with its v4.3 weight and v5.0 weight, and the change annotated. For v5.0, that means nine rows — one for each category — with the Cat 7 split and the meta-description migration spelled out explicitly.
The crosswalk is not a marketing document. It doesn’t frame changes as improvements. It shows the structural delta and lets the customer reconcile their prior score against the current rubric. If they ran an audit in January and want to run another in May, the crosswalk tells them which differences in score to attribute to rubric changes vs. site changes.
Why customer reconciliation crosswalks get deferred-but-named
When a customer runs an audit under one rubric version and later runs another under a different version, they need a crosswalk for their specific score — not just the generic structural delta, but “your site’s Cat 7 was 8/10 under v4.3; here is how that maps to 7a + 7b under v5.0.” That’s a per-audit artifact, not a per-version artifact.
That work is named in the changelog but not yet built. The structural crosswalk ships with each version cut because it’s straightforward to produce. The per-audit reconciliation requires tooling to generate at scale. Naming it creates the commitment; deferring it acknowledges the sequencing.
The convention going forward
Every time the rubric versions, the same pattern repeats: public changelog, structural crosswalk, provisional flags updated if weights change, prior-version annotations updated on affected categories. The changelog section at /methodology#changelog is the permanent, append-only record.
The score is only as useful as the confidence in what it’s measuring. The changelog is part of the measurement contract.