Home » Keyword research » Keyword Difficulty Index – What is it’s Relevance in Keyword Research

Keyword Difficulty Index – What is it’s Relevance in Keyword Research

Palla Sridhar

Keyword Difficulty score is proposed by several different tools to understand the competition of keywords.

A high index indicates that the keyword is very tough to rank in search engine results. A low score can be used to target those keywords in your content marketing and outrank your competitors in SERP rankings.

Search Engine Optimization tools can identify the ranking position of web pages for a particular keyword. Some of these tools are Moz, Ahrefs and SEMRush.

Keyword Difficulty score results of various SEO tools like Moz, SEMRush and Ahrefs vary based upon their statistics. The amount of data collected by each tool can be different from the other.

A keyword difficulty checker tool should be able to give accurate reasonable score which will make the publisher understand easily whether to target those keywords in his content marketing campaign.

Though several tools give an index or score for each keyword, how useful each one of them is to be seen. Each keyword difficulty checker tool gives a different score based on its own definition.

In this post, we are going to see the relevance of keyword difficulty score by different tools of Ahrefs, Moz and SEMrush.

SEMRush Keyword Difficulty Index

Keyword Research is finding those long term words and phrases to write content which answers the user’s query.

The SEMrush difficulty score is based on factors like authority of domains present on the first page. It analyzes those pages and gives an estimate on how a new website can outrank the current domains on the SERP. The score percentage is between 0 to 100.

The semrush difficulty index is divided into 3 zones.

1. High Difficulty : Above 80%
2. Medium Difficulty: Between 60 to 80%
3. Low Difficulty : Below 60%

Google sees each web page as an answer to each search query. So you have to focus content that is relevant to the user query and can gather good links from other sites.

SEMrush Keyword Difficulty Score Definition

As per SEMRush norms, KD refers to the following –

It is an indicative figure allowing users to estimate the difficulty with which which you can catch up with competitor’s positions in organic search results with a particular keyword. The score is based from 1-100. The metric is simply based on the competition between the domains ranking in the Google’s top 100 results.

In simple terms, you can say that SEMRush concentrates more on competition as a “difficulty” indicator.

How to Check SEMrush Keyword Difficulty?

It is a 4 stage process for keyword mining using the SEMRush difficulty tool.

1. Choose Keywords – Find all the relevant ones from Search Console or GKP and enter in the text box. A maximum number of 100 keywords can be input at the same time.
2. Estimate Keyword Difficulty – A lot of metrics are shown along with the KD score like volume, results, trend, serp source etc.
3. Analyzed Domains – Clicking the KD percentage score, one can see which domains are used to calculate the difficulty score. You can drill down to find further keywords.
4. Export data- If you want the values of KD score exported to an XLS or CSV, just use the Export button.

How does SEMrush calculate Keyword Difficulty Score

To calculate Keyword Difficulty, SEMRush takes into account the authority of the domains that are showing up in the results page.

Then using some “untold” formula it will give an index percentage on how hard for a new website to beat its competitors on the SERP.

As mentioned in the article, SEMRush also gives an indication of the difficulty level that can be used an indicator to go after particular keywords.

To calculate the difficulty of a keyword, we analyze the top domains that are ranking in the first twenty organic search results for the given keyword. Based on each website’s SEMrush Rank, we calculate its domain strength and define its keyword difficulty.

Using the Keyword Difficulty tool, you can enter up to 100 keywords at once. Then it will shop up corresponding metrics and data in a table format. These include volume, results, SERP features, Trend, SERP.

By sorting your results according to difficulty score, you can easily determine whether a keyword is worthy of writing good article or not.

Keyword Difficulty Score Display on the SERP

Install SEOQuake extension for any of your favorite browser and you can see several metrics related to each search results. It also provides the important statistic “Keyword Difficulty” as per SEMRush.

Note : Sometimes you may not get KD results for all of your keywords. They cease to exist in the SEMRush google.com database.

To prove that blindly believing in KD score always of any checker tool is like a pitfall, here is an example.

Let’s see the KD relevancy with a particular keyword – “how much data does skype use per hour”. This has a Keyword Difficulty of 76.75% for google.com database.

The results shows that KD is between “medium’ and “high” competition levels. So if you straight forwardly think that it is not worth the effort, you are on the wrong foot. But at the same time, you cannot discard the score.

For this keyword, my site page ranks as a “featured snippet”. If you observe the DS (Domain Score) and TS (Trust Score), these are zero for my site.

If you see the second result, it is a “people also ask” question which dominates the next 4 results. The following are the domains in this list –

1. quora.com
2. smallbusiness.chron.com
3. iphonefaq.org
4. quora.com

These have high TS of 64 and good SEMRush rank.

But the most important thing is that my site page has “zero” backlinks. On the other side, some of the other results have good number of links pointing to the page, good DS and TS.

Is SEMrush Keyword Difficulty Score Relevant

According to SEMRush, a KD around the range of 80% is high to rank for. But even for such high difficulty keywords I could rank for without much authority and links.

For some other keywords you may find that there is a relation between keyword difficulty index and ranking.

How does this happen? As we don’t know Google’s method of showing the top results, it is still a mystic idea.

Since it is my page I can explain a few things for this.

• I sent some paid traffic around an year ago.
• I did deep keyword reserach study and included relevant terms.
• The information provided is particularly useful and answers the user query.
• I used the queries in Search console during the initial days to include and modify the content several times.

Check SEMRush Keyword Difficulty Score Accuracy with Search Console

In order to test the relevancy of the keyword difficulty number given by SEMRush to different keywords, let’s do a case study.

I have taken a web page from my own site which is getting organic traffic. Now from Search Console data you can easily find all the queries that are ranking for this web page.

I exported that data into Excel sheet and made into two columns.

1. Queries
2. Position

Now we have different keywords ( “queries” to exactly say) ranking from 1 to 160 position in the Google.com search engine rankings.

These can be ordered ascending or descending using the Excel sheet, Data > Sort functionality.

From this list, I have taken 3 category of keywords –

1. Top most position #1   ( let’s call this “Easy”)
2. #11 to #12 position    ( Medium)
3. Least position #97 to #160.  (Hard)

Now I have taken around 10 keywords and put them in the Keyword Tool of SEMRush. As said before, it only allows 100 at a time. Now when I click “Submit” I will get the KD score for these queries.

Relation between “ranking” and “keyword difficulty score”

As my web page has “zero” backlinks, very less DA, PA, DS, TS, less number of social shares they should be directly related.

But as I found out, I could not infer any conclusive result based on this study.

The “easy” keywords should be having less KD score as they are ranking on the top for my less authoritative domain and page. Similarly, the “hard” keywords were meant to have high KD score as they are down the position lane. The same can be said that of “medium” keywords.

Most of my keywords were having a KD score between 60 to 80 for all positional ranking keywords.

High Difficulty Keywords

Medium Difficulty Keywords

Low Difficulty Keywords

Keyword Difficulty Index of SEMRush – How to Interpret it

The best method of analogy is to see other metrics in relevance to the keyword difficulty score.

For example, if a knowledge graph, featured snippet, news are present for a particular keyword with branded domains ranking for them, you can take for granted that ranking for that particular keyword is difficult.

Though the score may show below 30 or 40.

You must also take into account the number of searches and trend of the keyword.

As you know, even if you rank for a keyword that doesn’t have uniform traffic trends, it is a waste.

As this will definitely not bring you traffic and revenue.

SEMRush Keyword Difficulty Tool – Tweet about its Significance

Moz’s Keyword Difficulty Tool

To make things clear Moz has discontinued the Keyword Difficulty tool from Oct 2017. This is because the old KW difficulty tool heavily relied on PA and DA, age of domain etc kind of metrics which are outdated in the current year.

https://moz.com/community/q/bye-bye-keyword-difficulty-tool

But now in 2018, the difficulty metrics score from the Keyword Explorer point to a more reliable solution.

https://moz.com/tools/keyword-difficulty

Moz Keyword Difficulty Definition

Understand how difficult a keyword may be to rank for based on key indicating factors. Moz’s Difficulty score takes into account the Page Authority (PA) and Domain Authority (DA) scores of the results ranking on the first page of Google’s search engine for the given query. It also intelligently modifies for projected click-through-rate (CTR) of a given page (putting more weight on high-ranking, more visible pages and less on low-ranking, less visible pages). The algorithm also accounts for newer pages on powerful domains that may have DA scores but not yet assigned PA values.

https://moz.com/community/q/what-is-a-good-keyword-difficulty-score

There were some interesting notes from Rand Fishkin on Moz forum, about “keyword difficulty”. The question was “What is a Good Keyword Difficulty Score?”.

He clearly mentions that the answer is “simple and frustrating”. We cannot blindly depend on the score is high or low.

The above post received around 98k views.

KW Difficulty score Indicator

It gives the level of competition for a “medium” range website to rank for that keyword.

As explained before, Moz used to display a score for relevant keywords in their Keyword Difficulty tool. But the precision of these marks against observed ranking in Google was not that high.

https://moz.com/community/q/keyword-difficulty-and-time-to-rank

Another difficult task is to assess the correlation between keyword difficulty score and the amount of time it takes to rank high in Google.

But it is found that the “amount of time” for 1st ranking in Google depends on various SEO signals. Your competitors may be building links more quickly than you.

Also the relevancy of the keyword to the time is important. For example, a new item may quickly rank higher if it has the right target user intent.

From this perspective, it is to be observed that the “time” factor mainly depends on three things.

1. Content quality
2. Domain Strength
3. Backlinks to your site

Unless you produce highly engaging content which will make the user affix to your website, it will keep going down.

Initially you may have high ranks, but if your visitors are not sticking to your page, automatically Google will give less value to your page and site.

For example if you are in the 8th position of ranking for the keyword “seo research tools” and a branded site like searchengineland.com is ranking 1st position with awesome content.

Keyword Difficulty Score Relevance

Then what do you think is the relevance of keyword difficulty score? Do you think a KD = 5 or 10 will make sure to get you 1st ranking, if you write relevant and optimized content?

The chances are less. This is because the branded website has good authors who have good expertise and authority to write SEO optimized content. How much you struggle to keep up the pace you will be left behind. Unless the content you write mesmerizes the visitors and are ready to engage with your site.

So just by looking at a single metric like keyword difficulty it is not possible to say whether you can rank quickly and highly on Google, Bing, Yandex or Baidu.

When you target a keyword, first of all observe the current pages that are ranking on the first page. At that instant, Google thinks those are the pages that answer your query.

Important factors other than Keyword Difficulty

1. Number of backlinks to each page are less
2. The content is not lengthy (It is not true always)
3. Information provided is not relevant
4. Domains ranking are not branded ones
5. SEO factors are not implemented well

You can easily make up for the gaps in the SEO analysis and wait for Google algorithm to think yours is the best. This way we can deduce that whether a keyword is difficult to rank or not.

Other factors that decide Keyword Difficulty Relevance

1. Number of searches resulting for that keyword.
2. Less or High competition for the keyword
3. Long tail queries have less keyword difficulty
4. DA and PA of the page
5. Volume of the keyword in GA
6. Good Internal Linking structure
7. Number of shares on Social Media and their Influencers
8. Amount of Organic traffic the keyword is “actually” generating

Keyword Difficulty Checker using Google (Free Checker Tool)

We also use Google as a Keyword Difficulty tool to find whether a keyword is really complicated to rank high or not within less time. There are special operators to use in Google.

1. allinurl – Displays all results with keyword appearing in URL.
2. allintitle – Displays all results with keyword appearing in Title.
3. allintext – Displays all results with keyword appearing inside the article.

From the above, if we can get the following metrics to show up for a particular keyword in Google

allinurl = around 1,000,000 to 1,500,000
allintitle (and) allintext = around 1000 to 1500

you have a very good chance of getting on 1st page of Google.

The following SEO factors are also Important

1. Length of the article = around 2000 words
2. High Flesch Reading Score
3. Quick Load Speed
4. High number of Social Shares
5. High PA, DA backlinks
6. Good Image Optimization with Alt attributes

https://moz.com/help/guides/keyword-explorer/keyword-metrics

Keyword Explorer from Moz gives important metrics which are more relevant. These include the following –

1. Monthly Volume
2. Difficulty
3. Organic CTR
4. Priority
5. My Score

Here the two parameters – Priority and My Score reflect the true difficulty a keyword researcher has to face. Priority combines all the other metrics and spits out a single factor which is more indicative of a keyword having high traffic and easy to rank with.

My Score is the value you want to set, to make Moz know how important the keyword is to the client or a website. You can adjust its value. It is in the range of 1-10. This has an influence during keyword research.

Ahrefs Keyword Difficulty Score

Keyword Difficulty from Ahrefs

It does not take any on-page factors into account. Today these incorporate a lot more than keyword density, content length, exact keyword matches in titles/URLs/subheaders, etc.

That is why it is extremely complicated (if not impossible) to analyze the impact of the on-page SEO factors on Keyword Difficulty in every particular case.

Ahrefs’ Keyword Difficulty evaluates the chances of getting into Top 10 of search results (not Top 3 or Top 1). Apart from backlink profiles and content relevancy, many additional ranking factors come into play among the results of the first page of search.

https://ahrefs.com/keyword-difficulty

In the words of Ahrefs, this handy metric will be useful to know how hard it is to rank in a search engine during your keyword research process. There is a difference between Keyword Difficulty and Keyword Competition.

One is for organic search results and other is for paid search results respectively. You can rely more on Ahrefs metric, as it takes into account the number of relative domains pointing to the link. This has more authority than the Moz Keyword difficulty score.

Ahrefs Keyword Difficulty – How does it Fare

Links and Content relevancy are more connected than Page Authority (PA) and Domain Authority (DA). In a study conducted by Ahrefs for over 2 million keywords, it was found that a strong connection exists between the number of domains referring to a page and its position in Google search results.

This is more like the Page Rank factor that Google used to use in the initial days. If a page A has three page a1,a2,a3 linking to it, than based on algorthmic forumula some relevance score is given to page A. It depends on how much PR authority each one of the pointing pages have.

Ahref’s metric is more accurate than others in the industry.

It does not take into account on-page factors like keyword density, content length, exact keyword matches in titles/URLs/subheaders, etc. alone.

Keyword Difficulty Scale

Just like Moz, Ahrefs also integrated this factor into the Keyword Explorer tool. When you are going through a list of keywords suggested by the above tool, you can make informed decisions by looking at the keyword difficulty score.

This metic is integrated into other tools of Ahrefs like – Site Explorer, Content Explorer, Rank Tracker.

https://ahrefs.com/blog/keyword-difficulty/

It is also not clear with Google algorithm which has over 200 signals for ranking pages. But prominently it mainly relies on three factors – quality content, backlinks and Rank Brain algorithm.

Now-a-days it is also testing knowledge from machine learning and artificial intelligence algorithms.

Google Keyword Planner tool also shows a typical metric known as “Competion” in its keyword explorer feature.

Sometimes it is easy to be mistaken for the “Keyword Difficulty” score.

But this is different. It is more lenient to the amount of advertisers bidding for that particular keyword.

So “competition” indicator is more of a revenue speciality. If it is “low”, you can say that the CPC of that keyword is also less. Similarly the other way. A keyword can have a high difficulty score, but a poor competition level.

Low competition keywords with High search volume

This is to indicate that it is more easy to rank for such keywords. In a way, it also shows the difficulty level with which you can rank for in Google.

If we can link the competition level, search volume, backlinks and come to an weighted average, that can be a better indicator for keyword difficulty.

Another correlation found by Ahref regarding keyword difficulty is that, the number of referring domains to a page has a better link to Google rankings then just the number of backlinks pointing to it.

For example, if we can have 100 domains pointing to a page, than 100 links pointing, we can say that page has more chances of ranking high.

Factors determining Keyword Difficulty are –

1. Domain Authority.
2. Page Authority.
3. Number of referral domains.
4. Content relevance to search query.
5. Content depth of top-ranking results for a search query.

Source :  brafton.com/blog/strategy/how-to-use-keyword-difficulty-in-your-keyword-research-process/

The importance of keyword difficulty score lies in filtering your suggested phrases or relevant keywords. For example, if you type a seed keyword in Ahrefs and there are 3,000 semantic keyword spitted out.

Moz vs Ahrefs vs SEMrush – Which is Good

In a recent professional study conducted by Moz, it estimated the relevancy of keyword difficulty score of different tools based upon 50 keywords from 50 blog posts. This keyword research data was pulled from a single instant of time.

They have used the Pearson Correlation Coefficient (PCC) to find out the relationship between ranking vs organic difficulty. The final results stood like this –

As you know different SEO and SEM tools provide a keyword analysis tools which have their own meaning as mentioned above.

Ahrefs and SEMrush Keyword Difficulty Score

It should be seen from the earlier perspective that a 40 score in Ahrefs tool is not  the same as in SEMRush tool.

So if you are doubtful to use which tool to do keyword research based upon keyword difficulty score, you can use the other metrics in conjunction with keyword index.

Based upon this test, initially the PCC scores favored Moz and SEMRush.

Corrections to PCC scores –

1. remove the three most egregious outliers
2. resampling done to include randomness
3. adjustment based on keyword matching of practical functionality
4. usability adjustment of reliability

the final tally looked something like this –

Referential score estimated on the PCC scores

The final correlation was, what value of keyword difficulty score should be used for each tool that gives a better indication of toughness of ranking high in google search engine rankings. This chart can now be taken as a baseline, until further research shows up –

As we can see from the above values, the score given by each tool is different for ranking on the first page and also for ranking greater than or equal to 11.

So, if Ahref’s tool spits out KD of 20, it is not the same as SEMRush’s 60. 

Another important point to note is all the remaining tools have almost near score for pages less than 10 and above, but Ahref’s tool shows a good variance of 12.

Source : https://moz.com/blog/ranking-keyword-research-tools

Ahrefs vs SEMrush vs Moz

Since the keyword difficulty score by different tools have a varied meaning, it is difficult to understand their relevnace. You can filter out your keyword results in Ahrefs, for example, with KD < 30. This may give a keyword list of 100 to 200 keywords which are easy to include in your article.

Source: shivarweb.com/9822/how-to-judge-keyword-difficulty/

Before we dwell in KD score it is important to understand the user intent behind a keyword. KD helps partly as a resource to do your on-page SEO analysis.

Factors to Rank High on the SERP

1. Find out the suggested keywords from Google Instant search, just by typing the main keyword.

2. Think what Google thinks the user expectation of the keyword is. This can be done by looking at the top 10 posts. For a particular keyword like “omlette recipe”, you can get diverse answers in the form of commercial, informational, transaction, educational, community, local, resource etc.

3. Decide from reading those articles, are they currently answering the user question. Do you really think so? is the question.

4. If you can put a different perspective to the given query, make it a big post based on the parameters of the existing posts. Here you can take help of SEO tools like SEMRush, Ahrefs etc.

Conclusion

At last, Keyword Difficulty score is not a particular number you can just blindly believe on and go forward. Of all the three tools, Moz vs Ahrefs vs SEMRush, each one has a different way of calculating the difficulty parameter.

For example if you are following the Moz keyword difficulty rank, then your approach to keyword research and analysis should be different from that of Ahrefs.

Similarly if you take SEMRush difficulty factor into account, the way you decide which keyword to target should also be different.

Finally, thanks for reading this length article. If you find this useful, please share it among your friends and link to it. Also please comment in the section below on your views about “keyword difficulty” relevance in present period and how it can be useful as an SEO metric.