How We Discovered a Google Experiment

beaker-512pxHere at Hall we tend to pride ourselves on being knowledgeable, data driven marketers, but perhaps more than anything we’re curious. We analyze customer data daily trying to find out what’s changing in the online landscape even before it shows up in a Google press release or a marketing blog. Here’s an example of our process from just this past week.

Discovery

Recently our team noticed some anomalies in a client’s search results while reviewing data pulled from the Google’s Search Query API. We were seeing cases where Google had three or sometimes four different landing pages being used for the same query even with all other parameters equal. They all had an average rank of 1, and they all had the exact same number of impressions.

The query in question was a branded query similar to the company name with a significant amount of traffic. Google had obviously chosen this site as the correct domain to serve as the top result, but it seems as though they were testing which landing page within that domain is the best option.

The Home Page, the About Us Page, and the Product Overview Page were the three pages on this day that were competing with each other in this “Google SERP Experiment”. My guess is that Google was experimenting with landing pages and will eventually pick a winner based on bounces and clicks.

Expanding Our Search

We were originally only looking at a single day worth of search data, but became curious to see if this was a trend or an anomaly. We started looking for the same pattern moving backwards in time and quickly recognized it on a number of occasions. They weren’t always the same landing pages, but Google had been “experimenting” with different landing pages for quite some time.

There seemed to be cases where landing pages would fall off after a period of time, as if they had “lost” the experiment and Google determined it wasn’t a worthy page for the query. There were also cases where new pages were added to the experiment and removed after having “lost” the experiment as well.

What Does It Mean?

Though we can’t speak with any certainty, what this seems to suggests is that, even though Google’s algorithm determines which site it should serve in a position for a certain search query, it is now experimenting within that site to determine the correct landing page is best for the query.

How Can We Use This Information?

If we detect these experiments on an individual level, we can start to track the results. Once we find that Google has determined a winner, we can review the results over the time leading up to the decision to try and see if there are any determining factors as to why a certain page won over the others. We can compare the different pages in the experiment, and try to figure out why certain pages performed better than others.

For Paid Search purposes, these experiments can also help point you in the right direction as to which landing pages to use for certain paid terms. We can reasonably assume that pages winning these tests organically are more likely to deliver a higher quality score in the long run.

Google hasn’t officially said that they are running these tests, but the data does suggest that this is just another attempt to refine their algorithm in order to serve the best SERP experience for users.