Before you start: what do you know about SEO split-testing? If you’re unfamiliar with the principles of statistical SEO split-testing and how SplitSignal works, we're suggesting you start here or request a demo of SplitSignal.
First, we asked our Twitter followers to vote:
This time our followers weren’t lucky. The result was negative.
Let's find out why.
The Case
Over the years, many articles have been written about “best practices” for writing the perfect title tag. A topic that you often see is the use of capital letters. Should you sentence case or title case your title tags? In many of these articles, you will read that you have to format your title tags in title case for readability reasons, but is that strictly true? We decided to test this for a major Canadian software company.
Before we move on to the case study, let’s consider the difference between sentence case and title case. Sentence case is the capitalization of (only) the first letter of the first word (for example, “SEO ranking factors: does page experience matter?”). The title case is the capitalization of the first letter of most words (e.g., “SEO Ranking Factors: Does Page Experience Matter?”).
The Hypothesis
The website in question had its page titles formatted in title case.
We wanted to validate if the use of title case is impacting organic traffic (CTR). So, for the variant group, we changed the formatting to sentence case.
In the spirit of testing, we hypothesized that sentence casing would have a negative effect on organic traffic to the variant pages, as it makes page titles harder to read and harder to scan.
The Test
So, we used SplitSignal to set up and analyze the test. 70 article pages were selected as either variant or control. We kicked off the test and ran it for 28 days. We were able to determine that Googlebot visited all the tested pages.
The Results
The above image shows the development/progress of the variant (orange line) compared to the predicted control group (blue line). We saw that the traffic to the variant pages did not perform as well as predicted.
Note that we are not comparing the actual control group pages to our variant pages but rather a forecast based on historical data. We compare this with the actual data. We use a set of control pages to give the model context for trends and external influences. If something else changes during our test (e.g., seasonality), the model will detect and take it into account. By filtering these external factors, we gain insight into what the impact of an SEO change really is.
The cumulative view not only shows additional or lost organic traffic to the tested pages but also shows that the test performed is significant. When all three curves perform below (negative) or above (positive) the y=0 (cumulative gradient) axis, the test is statistically significant. That means we can be sure that the decrease we are seeing is due to the change we made and not due to other (external) factors.
After running the test for 28 days, we saw a 2.4% decrease in organic clicks to the tested pages, with a confidence level of 96%.
Why
Even if you’re very confident in the outcome, it’s always good to be able to validate it with data in a statistically significant way. For the website in question, we can now be sure that title casing the title tags of its article pages is better than sentence casing them.
Analysis of Search Console data shows that this test impacted the click-through rate (CTR) to the variant pages. Compared to our modeled control group, rankings and impressions remained fairly stable. While title casing is not a ranking factor, we can definitely say that it is a CTR factor.
People with informational intent often scan the search results to see if the listed webpage can answer their questions. Within search, you always compete with other search results. If your result is not easy to read and/or scan, a user can simply click on another search result.
This test is another great example that small changes can have a significant impact, especially when you can scale this out on templated pages.
Keep in mind that results can differ per site. Given the impact this can have on your overall search performance, thoroughly testing these “small” optimizations is always a good idea. For the website in question, this was confirmation that title casing does indeed matter.
Have your next SEO split-test analyzed by OrangeValley Agency.