You check your home page, get 0.21 g and a D, change nothing, check again the next day and get 0.20 g and a C. Nothing is broken. A live web page is not a fixed file, and the test that measures it runs in a real data centre over a real network. This article explains where differences between repeat checks come from, and how to compare checks so that the changes you make are the ones you see.
This is a different question from mobile versus desktop. If the two devices in one report disagree, see why desktop and mobile results differ.
The page itself can change between loads
GreenTracer's estimate comes from the data your page transfers on one initial, uncached load. If the page sends different files on different visits, the estimate changes with it. Google's Lighthouse variability guide calls this “page nondeterminism”, rates its impact as high and gives the examples of an A/B test that changes the layout and assets loaded, and a different ad experience depending on how a campaign is going. Common sources on small business sites include:
- Rotating content. A slider that picks a random first slide, a “featured product” that changes, or a news feed. A different image on top can mean a different number of kilobytes.
- Ads. Each ad slot can be filled with a different creative, from a small image to a video.
- A/B tests. If your marketing tool shows two versions of a page, each check may get either one.
- Personalisation. Content chosen by location, time of day or campaign can change what loads.
- Third-party updates. A chat widget, font service or tag manager can change its own files without you doing anything.
- Your own edits. A new blog post on the home page, or a plugin update, counts too. It is worth remembering before assuming the test has drifted.
Where and how the test runs
GreenTracer uses Google PageSpeed Insights, which runs a lab test of your page. Google's PageSpeed Insights documentation says lab data is “based on a simulated load of a page on a single device and fixed set of network conditions”, and that the test runs in a Google data centre, reporting one of North America, Europe or Asia as its location.
Location matters if your site behaves differently by region. If your consent tool shows a different banner, or loads different tags by default, depending on where the visitor is, two checks run from different regions can see two different pages. Our article on cookie banners, tags and your carbon test explains why consent settings change what the test counts.
Speed varies more than bytes
The same documentation lists local network availability, client hardware availability and client resource contention as common sources of variability, and the variability guide adds that web servers “do not always respond with the same delay”. Google's performance scoring guide also names A/B tests, ads being served and internet traffic routing changes among the reasons scores fluctuate.
Much of this affects how fast the page loads. GreenTracer's carbon estimate doesn't use timings at all: it multiplies the bytes transferred by the Sustainable Web Design Model's figures. So if your PageSpeed performance score jumps around while your bytes stay put, the carbon estimate stays put too. When the estimate does change, the first question is what the page sent, not how quickly.
Reading your check history
When you check a page again, the report keeps the earlier checks as history. Each row shows when the page was checked, the grade, the grams per page view, the data measured and which device was used for the headline figure. A few things to look for:
- Data measured. This is the number that drives everything else. A change of a few kilobytes is rarely worth chasing; a change of hundreds usually has a cause you can find.
- Device used. The headline uses the heavier of mobile and desktop. If one check used desktop and the next used mobile, you are comparing two different loads.
- “Reused measurement”. This label means PageSpeed Insights returned a measurement it had already taken, with the same measurement time, so that check isn't an independent new test.
- The grade near a boundary. A page at 0.208 g is a C and one at 0.210 g is a D. Two letters can hide an almost identical result, so read the grams. The grades page lists every limit.
Compare like with like
- Check before and after a change. Check the page just before you publish a change and again once it is live. Nothing re-checks automatically, so make it part of the job.
- Check the same address. Enter the page exactly as before, so the history compares the same page rather than, say, a campaign link with extra code on the end.
- Hold still what you can. If you are testing an image change, pause the A/B test or slider randomisation for that page if you can, so only one thing differs.
- Look for a lasting change. One different figure proves little. The Lighthouse variability guide recommends running tests several times and using the median, and says the median score of five runs is twice as stable as one. You don't need five checks every time, but a second check before drawing conclusions is cheap.
- Find the cause in the browser. Open the page in a private window with Chrome DevTools' Network panel, tick “Disable cache” and reload a few times. If the transferred total moves, sort by size and see which file changes.
Check your page, then come back to the same report after your next change and compare the data measured.
