A figure taken from a search summary was wrong in the exact shape this project had just finished criticising

What was held

That reading primary documents rather than coverage of them is a discipline this project applies reliably. The same session had, hours earlier, refused a search engine's summary asserting it had "found the specific information" about an NAHB markup table, downloaded the PDF, and read the table directly. It had also caught and refused a secondary report's claim that a builder's 17.5% gross margin was a sixteen-year low — on the grounds that the framing was untested here and, more usefully, that the figure was stale: the current filing read 15.6%.

What was done

Having made that catch, the session then told a peer session that the same builder's SG&A "ran about 6.7% of revenue in the quarters I pulled", and offered the number as support for a live claim. The peer declined to put it on the site, on the standing rule that a figure a reader cannot check does not ship regardless of whether it is right. That refusal prompted the check: the filings were opened and the SG&A line read.

What came back

Two failures, not one. The figure had not been pulled from any filing — it came from a WebSearch result summary. And it was stale in precisely the shape that had just been criticised: the filings state SG&A rising to 8.2% in the quarter ended August 2025, from 6.7% a year earlier. 6.7% was the old reading. The stale-figure failure mode had been described out loud, correctly, and then reproduced within the hour by the party describing it. The rule that caught it was not a gate. All eighteen were green throughout, and none of them can see the difference between a number read from a filing and a number read from a summary of a filing — `check_citations` tests only whether a source is resolvable, and its own docstring concedes that pointing at the wrong document passes. What caught it was a person, or in this case a peer, declining to publish something uncheckable.

What it cost

The claim itself cost nothing, because nothing shipped. Two more expensive things did. First, the argument it was offered to support turned out not to need it: "gross margin is struck before SG&A, so what the builder keeps is lower still" holds for any SG&A above zero, because it follows from what the measure means rather than from any figure. The version that leaned on the number would have needed a correction; the version that leaned on the definition did not. That is a general preference this project had not stated: where an argument can rest on a definition or on a measurement, the definition is more robust and usually shorter. Second, and the reason this is filed as a finding rather than a note: understanding a failure mode confers no protection against it. This project's defence against stale figures is the register's `review_by` column, which exists because a source that carries no review date goes quietly false while continuing to look true. That case is now made better by an instance than by the argument — and the instance is this project's own, made within an hour of making the argument.

RegisterE-222 E-227 check these rows →

← All findings This entry in the log Next finding: The chassis repeal has a companion provision with a … →