Information Gain

Information gain asks a blunt question of any page: what does this add that is not already known? The idea is borrowed from information theory, where it measures how much a new piece of evidence reduces uncertainty. Applied to a website, it separates the pages that tell a machine something it did not have from the pages that restate what it has already read a thousand times over.

This mattered less when search ranked documents. It matters enormously now, because the system reading you already holds the general facts. A model does not need your page to learn what a heat pump is, what conveyancing means, or why insulation matters. It reaches out to the web for what it cannot know from the inside: what something actually costs this year, in this town, from you.

Most pages score badly, and they score badly for a reason that was once sound advice. The definitional opening paragraph. The reassuring throat-clearing about a fast-changing world. The service page assembled from the same six benefits every competitor lists. The article written by surveying the top ten articles, which by construction can contain nothing the top ten did not already have. All of it reads as perfectly competent, and all of it is invisible.

What carries gain is specific and slightly uncomfortable to publish. Real prices, and what moves them. The constraint you work under. The job you turn down, and why. The thing you changed your mind about after the twelfth year. Numbers from your own work rather than from the report everyone cites. It tends to be exactly the material an owner assumes is too particular to interest anyone, which is what makes it worth retrieving.

The plain version: if this page disappeared tonight, what would be lost that could not be found elsewhere by morning? If the honest answer is nothing, the page is not badly written. It is simply not needed — and being not needed is now the more expensive of the two failures.