We audited our own scoring weights, and got two of them wrong

Any scoring system is an argument about what matters, and the weights are where the argument is made. The Local Digital Visibility Index gives Speed & Core Web Vitals 20% of the Digital Visibility Score and Content & trust 10%. That was a judgement made before we had the data to check it.

We now have 1371 businesses across 34 indices, so we can check it. This article is what happened when we did, including the finding that our own weighting is close to backwards.

The test

For each pillar we asked a simple question: does it move with the things the score is actually trying to capture? Those two things are already measured separately — whether a business appears in search results at all (the Visibility pillar), and whether an assistant names it (AI presence).

A pillar that matters should rise and fall with those outcomes. A pillar that does not is measuring something real about a website that has little to do with whether anyone finds it.

Pillar	Share of score	Moves with Visibility	Moves with AI naming
Speed & CWV	20%	-0.09	-0.07
Technical	20%	0.21	0.12
Local presence	20%	0.40	0.29
Visibility	15%	—	0.65
AI search presence	15%	0.65	—
Content & trust	10%	0.32	0.25

Those figures are correlation coefficients. Zero means no relationship; 1 would mean the two move in perfect lockstep. For this kind of observational data, anything above about 0.3 is worth noticing and anything under 0.1 is noise.

Each pillar's share of the Digital Visibility Score, against how strongly it moves with the two things the score exists to capture — appearing in results, and being named by an assistant. If the weights matched the evidence, these points would climb to the right.

Vertical axis is the mean absolute correlation with the Visibility and AI presence pillars, across 1371 businesses. Association inside one cohort, not proof that changing a pillar changes an outcome.

The two we got wrong

Speed & CWV is 20% of the score and correlates -0.09 with being findable. That is not a weak positive relationship. It is nothing, very slightly leaning negative. The fastest sites in this dataset are not the ones that get found.

A correlation is an abstraction, so here is the same claim without one — median findability within each band of speed score:

Median Visibility score within each band of Speed score, across the 1246 businesses measured on both. If speed drove findability this would climb from left to right.

Left axis is the Speed pillar score. Bars marked · hold fewer than 10 businesses and should not be read as a trend on their own.

If speed drove findability, that would climb from top to bottom. It does not climb at all — it wanders. The bands run 8, 13, 4, 4, 0 as speed improves, which is to say the fastest sites in the index are the least findable ones, and the band with the best median findability is the slow one.

Do not over-read that inversion either. The slow band holds 137 businesses against 792 in the middle, and a median drawn from that few moves if three of them change. The safe conclusion is not “slow is better”. It is that within this range, speed carries no usable signal about findability in either direction.

Content & trust is 10% of the score and correlates 0.32. It is the strongest predictor we have that is not itself an outcome, and it carries the least weight of any pillar.

So the heaviest pillar predicts least, and one of the lightest predicts most. Put plainly: a business could improve on the thing we weight most and move barely at all on the thing it came to us for.

Why we have not just changed the weights

The obvious response is to reweight and republish. We are not doing that this quarter, for two reasons.

Changing the ruler mid-measurement destroys the comparison. The whole point of publishing quarterly is that a firm can see whether it moved. If we change the weights at the same time, every score changes, and nobody can tell whether their business improved or we redefined the test. That is the failure mode the changelog exists to prevent, and we would be committing it deliberately.

One quarter of correlational data is not enough to rebuild a scoring system on. Which brings us to the part that argues against our own headline.

The case against this finding

Three reasons to be careful with what you have just read.

The confound is real and it points the wrong way. Established firms tend to have larger sites, more pages, image-heavy portfolios and years of accumulated third-party scripts. Those sites are slower. They also rank better, because they have been around longer and have more content. That alone could produce a slightly negative speed-to-visibility relationship without speed being harmless — the same firms that rank are dragging the speed numbers down.

The range is narrow. Almost every site here scores somewhere between mediocre and decent on speed. A dataset containing no catastrophically slow sites cannot tell you what happens to a catastrophically slow site. It would almost certainly be bad.

Correlation between two pillars is not causation, and both are measurements we made. If our Visibility pillar has a bias, this analysis inherits it. We are checking our ruler with another part of the same ruler.

Local’s row deserves a specific note, because it is the one pillar we have already acted on. Its review-velocity component was miscounted for most of this quarter, then re-measured, then changed again the same day — from a fixed threshold to one set by each business’s own cohort, because a fixed bar was largely measuring which trade a firm was in.

What happened to its correlation when we did that is the interesting part. It fell — from 0.37 to 0.40 against Visibility, and from 0.22 to 0.29 against AI naming. A chunk of Local’s apparent predictive power was not the businesses at all. It was the sector: high-review trades scored well on Local and also happened to rank, and the pillar was quietly taking credit for the correlation between those two facts.

That is worth holding onto when reading the rest of this table. If one pillar’s association shrank by a third once we removed a confound we had not noticed, the others have not been audited that hard — and Speed’s figure could move in either direction under the same scrutiny.

What happens next

We will run this same audit against Q4-2026 when it lands in September. If Speed still shows no relationship across two independent quarters, the weights change — announced in advance, dated in the changelog, with both the old and new scores published side by side for one quarter so nothing moves silently.

If it turns out the confound above explains the result, we will publish that too, and this article will carry a correction.

Why publish this at all

It would be straightforward not to. No client has asked to see the correlation matrix behind our scoring, and the finding makes our own methodology look under-considered.

But an index whose author never publishes an inconvenient result is not an index, it is marketing with a table in it. The data is CC BY, the pipeline is open source and the method is documented, which means anyone sufficiently motivated could have run this analysis and published it themselves. We would rather find our own mistakes in public than have someone else find them.

The practical takeaway for a business owner is narrower than the headline: do not buy a performance project on the promise that it will make you findable. Speed is worth fixing for the people already on your site. On this evidence, it is not what decides whether they arrive.