Skip to content

SEO Testing: The Secret To Maximize Your Website Traffic and Profits

Maximize Your Website Traffic and Profits cover image

Search engine optimization (SEO) requires constant testing to be effective. If you don’t understand what changes drive growth (or why visitors respond the way they do), you’ll waste your time guessing at solutions while your competitors leave you in the dust.

In this article, as someone with significant SEO expertise, I will provide a comprehensive understanding of SEO testing, the challenges of acquiring actionable data from Google Analytics, and how to conduct SEO tests that effectively improve your crucial performance indicators.

Upon concluding this article, you will not only recognize the transformative potential of SEO testing but also benefit from three of my recent test results that defy traditional SEO principles.

To begin, let’s briefly define SEO testing and examine its different manifestations.

What is SEO Testing?

SEO testing refers to the experiments you perform to measure how search engines and users react to your site.

There are many tools you can use to get data about each web page (such as Surfer, Ahrefs, and other free tools), but no Google Analytics tool or SEO checker can tell you what effect changes will have.

The purpose of SEO testing is to manipulate different search traffic factors to determine if the changes result in an increase or decrease in organic traffic or your position in search results.

There are many ways you can run SEO tests, including:

  • Creating different sets of content
  • Adjusting keywords, meta descriptions
  • Trading out different links

Like your high school science classes, SEO testing relies chiefly on utilizing experimental and control groups’ concepts.

experimental groups and control groups

The experimental groups are the cases (pages or entire sites) that experience your test variable (such as trying a new link strategy).

The cases in the control groups are untouched.  Their rankings and organic traffic will give you the reference point to determine whether or not the experiment worked (or not).

Great testing depends on well-built tests.

Instructions on how to create great SEO tests will be covered a little later…

But first, it may help you precisely understand what kind of benefits you can enjoy if you do the work to develop reliable SEO tests.

What Impact Does an SEO Testing Have on SEO?

The impact that testing has on your SEO strategy is immeasurable—no pun intended. It provides all of the following concrete benefits:

Testing Gives You the Power to Be First

Whenever you run an SEO test, you put yourself in a limited category of people on the industry’s cutting edge.

Power to Be First in SEO testing

Some of the most effective SEO tactics ever discovered have been neutered by Google search console shortly after they reached the wider community.

If you’re a tester, you get to be in the rare company of people who use the latest techniques while still being powerful. That enables you to operate in the most profitable niches.

Testing Helps You Develop a Ranking System That Works

Repeated testing gives you the power to separate what you can and can’t know about SEO, based on experience.

SEO ranking system

This allows you to skip time-wasting techniques and move directly to those with the most significant, most reliable SEO performance impact.

That makes you a far more efficient SEO. This ability is one of the critical differences between amateur SEOs, and the kind who can successfully dominate even the most competitive niches.

Testing Can Make You Algorithm-Proof

If you test SEO changes often enough, you can develop a second sight for the trends that are guiding search engine updates.

You’ll spot trends before they’re officially announced, and adjust your practices to avoid penalties.

What are the Challenges of Performing Accurate SEO Tests?

The biggest challenge with SEO testing is collecting untainted data. Your results become tainted when they become affected by outside factors or wrong assumptions.

These obstacles make it impossible for you to say which results are a product of your experiments.

seo challenges meme

Challenge #1: You Must Make Sure Your Variables Are Stable

To understand what’s happening on each web page in the experiment, you need to isolate what changes affect your data. Stabilizing your variables means working to prevent changes from happening that are outside of your test.

single variable testing seo

For example, if you are trying to measure the effect of content upgrades, you need to prevent your link profile from changing during the testing phase.

If a lousy link gets disavowed or a useful link gets added, your site’s changes could easily mask any result of the content upgrades.

Challenge #2: You Need to Make Test Cases as Similar as Possible

The first thing you need to do is make each experimental and control group nearly identical. This will allow you to start measuring the changes once you start altering the test site.

seo split testing illustration

When you come back to check website changes, you’ll have a better idea of how they happened.

Here are just a few examples of why you’ll have trouble with that:

  • No two web pages can have the same page URL, yet the URL is itself a ranking factor.
  • Those links still won’t be the same age if you manage to get the same links for both sites. It’s also highly unlikely that you’ll be able to gather them at an identical velocity.
  • If you create two identical sites for your SEO split testing, one is now filled with duplicate content. Duplicate content is a factor that Google can detect and may choose to penalize. In some cases, google search console may refuse to index the new material at all.

Challenge #3: You Need to Expect the “Random Ranking Factor”

The random ranking factor is a phenomenon coined by SEO Terry Kyle (founder of WPX hosting). It is best illustrated with an example…

Imagine that you launch five identical sites or landing pages on the same day. They are in the same niche, use corresponding keywords, and employ the same design style.

You will likely notice the following:

  • Three sites tend toward average performance
  • One site tends toward exceptional performance
  • One site tends toward inexplicable sluggishness

seo testing random ranking factor

This is not a precise rule, but many SEOs have recorded the effect over the years.

For reasons that are nearly impossible to measure efficiently, some sites simply behave as if they are blessed while others act as if they are cursed.

What’s the Solution?

There is one primary solution to the challenges that SEO testing poses.  You need to increase the number of test cases.

The solution is to simply increase the number of test cases.

One control page is never going to be enough, and neither is one SEO test page.

Adapt to this challenge by creating larger groups for each test group and control group. Have 10 URLs in the control group, and 10 in the experimental group…or better yet, 50.

seo large test cases

Then, you’re going to measure the results by taking the average of the changes in both groups.

Maximizing the number of test cases helps to resolve our three biggest challenges in the following ways:

Keeping variables stable

It’s impossible to completely control external variables in SEO because none of it takes place in a vacuum.

For example, if we have 50 test cases, then we know that a single random backlink hitting one of them isn’t going to throw off our test.

Keeping test cases similar

Once again, creating similar test cases for SEO is nearly impossible. A large amount of test cases helps us smooth out fluctuations.

For example, it helps you control all the minor differences caused by URLs, age, and other factors that play a role in SEO.

Controlling the random ranking factor

With a large number of test cases, we know that we’re at lower risk of our having our results colored by a page that’s uniquely blessed or cursed. The random ranking factor gets averaged out.

We now have enough information to identify sites that behave differently. This will help prevent us from either getting overconfident or giving up on a great technique just because it failed randomly.

Let’s look at all this through an example.

Example Test

Pretend you have a blog in the wellness niche, and you want to test the effect of double-counting keywords in the title tag.

If you have 30 posts with organic traffic, a simple experiment might involve creating two groups:

  • 15 posts preserved as a control group (titles unchanged)
  • 15 posts to test your idea of an ideal title

wellness website seo test

If you’re curious about how a test like this might play out, don’t miss the ‘clickbait title’ test that will be revealed near the end.

For now, let’s focus on what roles these groups play:

  • Group 1 will help you follow any changes that happen outside your experiment. If your control group experiences sudden SEO shifts, then you know the same effect in your other groups isn’t a result of your SEO tests.
  • Group 2 will test your hypothesis. Once you’ve subtracted any SEO shifts from the control group and random group—you’ll have an idea of the effects of your real changes.

Naturally, testing like this is going to depend on the funding you have. If you can afford it, testing ten sites instead of 5 (or 20 instead of 10) will allow you to create a more accurate average.

If your budget is very limited, you can test across multiple pages with only 1 or 2 sites.

However, there will be more room for error with those numbers than most experienced testers will tolerate.

The more web pages you can create for a test, the more effectively you can cancel out noise. Sometimes, the most effective way to cancel out noise is to remove important factors altogether.

Best Practices for SEO Testing

Follow these rules to draw some better data from each SEO test:

Set Aside Enough Time

seo strategy schedule

Beyond your money budget, you need to put a significant time budget in place. Once again, the more time that you can give your experiment, the more sure of the accuracy of any data that you collect.  Some results (especially offsite related) may take months to yield a difference.

Exercise Attention to Detail

If you can’t measure it, you can’t manage it. The more information you’re tracking, the more awareness you’ll have of how sites are different, so you must be tracking even when you’re not testing.

peter drucker quote

I can illustrate this with a real-life example. At one point, I had a great site in the Brazilian testosterone niche. One week, we suddenly jumped from page two to page one.

I was only able to figure out why, because of my tracking. It turns out, some links I snagged from a local citation package were worth a lot more than I imagined it would be.

Thanks to my research, I caught the factor. Thanks to that insight, I now had a new strategy to use for many other sites.

Recent Findings in SEO

Nothing illustrates the impact of SEO testing, like having access to information most of the industry doesn’t yet possess.

The following experiments were performed as personal research. What follows are the theories, tests, and findings from research built using the rules above.

Test #1: Measuring the Effect of Inserting NLP-Friendly Sentences Into the Content

This test involved a look into whether implementing Natural Language Processing (NLP)-friendly language would change the way that Google responded to a site. NLP refers to the ability of software to interpret and manipulate language. This technology is improving but still limited.

Theory: If we can make it easier for NLP to process our content, Google will be more likely to reward us with snippets and better rankings.

We aimed to optimize our content for NLP by doing the following:

  • Echoing back the question
  • Giving the answer
  • Mentioning the correct unit

how to write nlp friendly content

For example, here’s a typical way of answering a question to another human, compared to the NLP-friendly style.

Question: What is the best temperature to brew beer?

Standard answer: “68 to 72 degrees.”

NLP-friendly answer: The best temperature to brew beer is between 68 to 72 degrees Fahrenheit (20 to 22 degrees C).

Test Structure:
  • 50 page were chosen, spread over ten websites
  • Each page had flat rankings
  • NLP-friendly content replaced query answers for a selection of pages, and the sites were submitted for recrawl
  • Finally, we looked at average % increase or decrease in rankings and rich snippets

Results:

BeforeAfterImprovement
# of featured snippets4694+104.3%
Average ranks15.211.2+26.3

Implications: NLP-friendly language measurably improves rankings and attracts snippets. The number of snippets we controlled doubled after our work.

Test #2: Is It True That Clickbait Titles Stopped Working in 2024?

clickbait sensational title
This test measured whether the conventional wisdom that clickbait titles were “over” held any water. It involved changing 40 clickbait titles on existing blog posts.

Theory: People have started to avoid clickbait titles, and may prefer more straightforward descriptions of what they’re getting.

We understand clickbait meta titles as titles that tease exciting results, but may not explain the article’s content:

Example:

Clickbait title: 7 Strategies that will increase your organic search traffic by 200%

Straightforward title: How to increase organic traffic to your blog

Test Structure:
  • We gathered 40 URLs that met the following criteria: Each one:
  • Used a clickbait title (that’s how we normally write all of them)
  • Ranked in the 1-3 spot
  • Had been flat over last 30 days
  • The Click Through Rate (CTR) stats from the past two weeks were gathered from Google Search Console.
  • The SEO titles were changed to straightforward descriptions of what the content was about.

Results:

Total ClicksTotal ImpressionsAverage CTR
BeforeAfterBeforeAfterBeforeAfter
40X URLs11,4207,75031,848426,1093.73%2.90%

Implications: The new titles had far fewer clicks. People are still more responsive to clickbait titles than they are too straightforward descriptions of the content. 

Test #3: Does Interlinking Irrelevant Pages Hurt Your Rankings?

This test examined whether irrelevant internal links were noticed or acted upon by Google. The SEO site that inspired this experiment was a beard oil shop interlinked freely with pages that were slightly, but not directly, related (such as beard shampoo).

beard oil starter sample

Theory: Irrelevant internal links hurt the potential to rank

We understand “irrelevant” internal links as those that link to pages with unrelated topics. We were pretty strict in defining links as irrelevant.

For example, for the topic of Beard oil, we purged links from pages that covered:

  • Beard shampoo
  • Beard wax
  • Beard balm

These topics are related to one another, but they don’t serve the same need, so we considered them irrelevant for testing purposes.

Test Structure:
  • Eight real medium-sized websites were chosen
  • 40 URLs were selected from the set. They included:
  • Irrelevant pages: These were chosen because they had no keyword overlap whatsoever. They were linked to our test page.
  • Test pages: These were chosen because they had decent link equity. The irrelevant pages were linked up with standard anchors.

The test involved creating internal links to the irrelevant pages from the test pages. 80 keywords were tracked to determine changes

experimental group test phases

Results:

Ranking ImprovementTraffic Improvement% of keywords that went down
+9.9%+12.73%2

Oddly enough, the pages that were given links from irrelevant pages experienced short-term ranking and organic traffic improvement. Only 2% of all keywords went down.

Implications: Internal links from irrelevant pages are not that harmful, and may even result in short-term improvements.

This video outline everything you need to know about interlinking, watch this video.

What Types Of SEO Tests Are There?

There are several types of SEO tests that can be performed to optimize a website and improve its ranking on the google search console. 

1. SEO Serial Testing 

Serial testing involves modifying all of a website’s pages simultaneously and observing the results. This is not recommended for three reasons:

  • If the update has a negative effect on SEO, it affects all of your pages, not just a select handful.
  • It is often more time-consuming to implement site-wide changes and considerably more time-consuming to undo them.
  • It does not account for seasonality or uncontrollable circumstances.

2. SEO Time-based Testing

In time-based testing, you modify a single page and observe its performance over time. It is unwise to imply causality with a sample size of one; hence, we do not advocate this form of SEO testing approach.

3. SEO A/b Testing (Split Testing)

A/B testing (also known as split testing) entails modifying certain web pages but not others. Then, you compare the performance of the modified pages to that of the unmodified ones. The group of unaltered pages is referred to as the control group, while the group of altered pages is referred to as the variation group.

The SEO split testing form of SEO testing is recommended because:

  • If the adjustment adversely affects SEO, only a tiny fraction of pages will be affected.
  • It is less time-consuming than serial testing since just a subset of pages must be modified.
  • Seasonality and elements outside of your control are accounted for since they affect both sets of pages.

Success Through SEO Testing Is Within Your Reach

SEO testing is one of the most effective ways to improve your sites and your skill as an SEO. You can learn the most effective techniques by understanding the role of testing, meeting the challenges head-on, and setting up your SEO tests the right way.

As the SEO tests above showed you, the conventional wisdom about what works for SEO is often flawed, or only applicable in some cases. SEO checker or Google Analytics tools will only tell you where your website is now. Testing empowers you to see where it could go.

By taking the time and expense of testing seriously, you can discover a more productive path for your sites, and plan for the changes google search console
make. Armed with the knowledge that only you possess, you can carve out a niche for yourself in the SEO world.

If you want to learn more about tested SEO tactics and the strategies that work, you can join The Affiliate Lab. You’ll get access to all my private SEO tests as soon as I have results, and the chance to discuss your own with the most exclusive affiliate SEO Facebook group around.

Got Questions or Comments?

Join the discussion here on Facebook.

Matt-Author-Img

Article by

Matt Diggity

Matt is the founder of Diggity Marketing, LeadSpring, The Search Initiative, The Affiliate Lab, and the Chiang Mai SEO Conference. He actually does SEO too.

 

Want to rank easier, higher and faster?

Sign up and join 100,000+ other subscribers and get SEO test results sent straight to your inbox.

moz sidebar

"One of the most effective SEOs I've ever met"- Cyrus Sheppard

As Seen On...

Search Engine Journal
New York Post
ahrefs brand