A How to Guide on Identifying Bad Links and Pruning Them Using Google’s Disavow Tool

Navneet Kaushal

November 14, 2012

When the long-awaited Disavow Tool from Google was announced finally at the PubCon Las Vegas, search professionals around the globe breathed a sigh of relief. There is a way now with which they can deal with the penalties laid on them by the Penguin Update due to ‘un-natural links’. They can now ‘disavow’ these links and get their site back in Google’s good books.

If you want to use this tool, you will have to identify your bad links first and then work on pruning them. Once you submit the list of links you have to disavow, you will be given a choice of comments or links or domains to block. Of these options you must choose accordingly. But remember that you must use the tool only on the links which you haven’t been able to remove manually.

Google itself has stressed that the tool is a last resort after you have exhausted all manual efforts to get bad links removed. Because once Google disavows them, you may even lose the good links and eventually your ranking may fall even further. What are these bad links and why Google penalizes you for them are discussed in the next section.

The Update That Changed It All: Penguin

It would be safe to say that Google’s Penguin Update has shaken the very basis of SEO practices: link building. Until recently, sites linking to your site were not much of a problem. But now, if you have ‘unnatural links’ (as Google calls them) linking to your site, it can hurt your ranking severely.

The update has come down heavy for businesses selling links, as well as sites which have built a spammy link profile by using exact match anchor text extensively. This activity is flagged off by Google as ‘unnatural link building tactics’ and falls under heavy spamming.

Many sites which have not been intentionally buying links have also been hit by the update, simply because they have links from ‘untrusted sites’. Webmasters are struggling to find a way to fix these link issues and get their site back up. The urgency to do the same is intensified because the Penguin can strike again anytime.

It’s Not Over Yet

At the recently concluded SES San Francisco, Google’s webspam team head, Matt Cutts, gave a clear warning against the Penguin. He said that the algorithm was in its experimental stages. Google was fine-tuning it and ’till they get it right—the Penguin roll outs will give ‘huge jolts’.

The Panda Update is smoother and subtler now (after more than a year of the first roll out), but the Penguin can smack hard. This announcement by Matt sent nearly the entire SEO fraternity scurrying to find: “penguin proofing techniques.”

The panic attack has not been helped by the link warnings being sent via Google Webmaster Tools. Google has been sending ‘unnatural link’ warnings to webmasters since February 2012. Now and after the Penguin smack, people finally understood that their entire effort of building links over the years could be the reason for their site falling in SERPs.

The message from Google is loud and clear: There is more to come. And the ball is in your court now, as to how you will identify and deal with those ‘unnatural’ or bad links.

The Best Way Forward: Backlinks Check

You have to take up the mammoth task of a backlink audit to see where are the toxic links coming in from and how you can get rid of them. Analyzing your link profile can be a time taking and tedious task, but you will have to do it—the sooner the better. Bad links may include:

Links coming in from poor quality/spammy sites
Disproportionate anchor text links
Links from blog networks
Links from paid posts
Links from dead & rotting directories
Links from irrelevant & ‘untrustworthy’ sites

A great way to spot an unnatural link profile is to compare your site with your competitors. A thorough comparison will let you know of the major differences between their link profile and yours. To identify the ‘unnaturalness’ of your back links profile you will have to see:

Site-wide links
Anchor Text Distribution
Link Networks
Blog Links
Blog Comments/Forum Links
Article Directories links
Web Directories links
Footer Links

Whether you have been hit by the Penguin or have been warned by Google for having unnatural links, you will have to analyze the backlinks mentioned above and make comparative analysis with your top five competitors (at least).

Top Backlinks Evaluation Tools

There are a number of SEO tools out there you can use for deep evaluations for backlinks. Some of these are free, while some will require subscription. Let us see, what each tool can deliver:

cognitiveSEO

This tool allows you to research your backlinks profile quickly and simply. It is a paid tool and offer Backlinks Reports; inBound Link Analysis; Link management and daily rank tracking.

The inBound Link Analysis module allows you to analyze all incoming links using the filters like:

Link Position
Link Context
Majestic AC Rank
inBound Links

The results will show the poorest quality pages first. Reviewing is a breeze with the link preview feature, and you will have a data of the most toxic links coming back to your site. The tool advises you to manually review the links and then add comments in to the links and then importing the data into the Google Disavow tool, and disavow the negative links.

Link Research Tools

It is a paid tool with a number of functionalities, but their Backlink Profiler is perhaps the best way to source and identify backlinks from various domains and get every detail about them. The list you will get will already be pruned to remove dead links. You will get to know the nature of the link, follow or no follow etc. as well. The tool will also tell you the domains giving you quality link juice.

You will have an easy to read bar chart with the report and further digging will tell you the anchor texts of these links and all the data can be saved as a CSV file. The tool comes with a one-click export function which immediately creates a suited export file for the Google Disavow links tool. And using the file, you can get rid of the links you see unfit.

Google Webmaster Tools

Google Webmaster Tools is probably the most effective and free backlinks research tool out there. The latest functionality being added is that one can download backlinks by date. To know the links to your site, you will have to go to your Dashboard > Links to Your Site > Download Latest Links.

You can download all the link data in a Google docs form or as a CSV file, as well. The data will have the newest links on top. Interpreting the data is easier, as you can track the links pointing to your site by sorting them on date basis.

Once you have the list of links from Google Webmaster Tools, you can then export the data on to the other tools mentioned above and get a final report of the links that are bad. The next step would be to use the disavow tool to get rid of them.

SEOMoz Open Site Explorer

This tool is perhaps the most loved after the Yahoo Site Explorer retired. The tool is easy and quick to use and has both free and paid option. The best thing about this tool is that you can compare with other sites using domain authority benchmarks.

Of course, there is a lot of data made available for anchor text analysis and you can take the data out on a spreadsheet to mull over it. You can drill and drill further in your link evaluation process and can also find links to certain chosen pages.

The link data includes the top pages on your site, anchor texts used in links, the links to a site and the shares on social sites. You can study the links your competitor is getting and see where your links are going wrong or shall we say- ‘unnatural’.

ahrefs Site Explorer

ahrefs is a paid tool (with plans starting from $790/year), but offers two months’ free usage if you sign up for an annual subscription. It has a number of services/tools on offer, which are great for researching the competitor, as well as finding the toxic links to your site.

Their ‘Site Explorer’ functionality is the one you can use to study backlinks profiles. Simply keying in your domain will give you a tabulated view of the referring links to your domain:

Number of Links
Types of Links
Number of Unique referring domains
Most extensively used anchor texts
Domains directing the biggest number of links
New links
Lost links

The tool’s ‘Standard Reports’ functionality will allow you to research in a different manner. You can sort and filter the data as per certain parameters like social sites metrics. There is a very nice graphical representation of the changing links, which you can refer to for in-depth analysis.

Majestic Site Explorer

This tool is free as well as paid, and for deep analysis and comparisons like finding site-wide links, you do need to take up a subscription (there are silver, gold, and other subscription plans). The tool’s ‘Bulk Backlink Checker’ is perhaps the most recommended for checking links for many domains.

You can instantly compare your backlink profile with your competition’s. The tool lets you find:

Site-wide links: To see the list of sites linking to yours in a descending order of the number of links they send at you. You need to Activate Historic Index > Key in your domain in Site Explorer > Explore > Referring Domains.
Over optimized anchor text: See the anchor texts used most widely and you can further filter the report as well to see links for any particular anchor text. Simply hit on “Generate Report” to see the list.
301 Redirects: Find the links you are getting from parked domains/articles.
Weeding out bad link networks: The functionality called Clique Hunter allows you to get a list of links pointing from penalized sites.
Identifying weakest backlinks: You can get a report of the links to your site in a descending order of their strengths. You can download the data too for further use.

Using Google’s Disavow Tool:

Once you have the data, you can log in to cognitiveSEO using API and get all the data from Majestic SEO and then import the data into the Disavow Tool.

Judging the Domains Linking to You

OK, so you have a list of suspicious domains linking to you, but there is no need to immediately embark on a ‘link removal request’ spree right away. You can judge the domain on your own to see if it is a healthy one or not. Judge the site on these parameters:

Is the site of good quality and relevant to your site?
What is the Alexa rating of the site?
Have they passed on a site-wide link?
What is the index status of the domain in Google? And what is its PR? A penalized site can be identified easily. If it has a number of links pointing to it but has zero PR, then it has been penalized by Google.
Does the site’s server have many domains hosted on it?

Pruning Those Bad Links: Link Removal Requests

You now have gathered the data and analyzed your backlinks reports to know the toxic links, as well as the anchor text distribution. You have the list of links that need to be gotten rid of, but how do you go about it? The obvious way is to request a removal by the domain webmaster.

The fastest way would be to create a standard email to be sent across the all webmasters requesting link removal. The request should include information in this order:

You are an SEO or webmaster.
Your site has been penalized by Google.
You are working on recovering from the penalty.
For the same, you need them to remove the links.
Then list the URLs of the links they send and the URL of your site they point to, and also the anchor text used.

If you are lucky, you will get a reply and the link removed. If it is your bad day, there will be no response/or there could be demand for payment for the link to be removed. At this stage, you must keep a record of all your correspondence details with every webmaster and attach that spreadsheet to the reconsideration request you send to Google.

In fact, you can also save, as well as take screen shots, of the emails you have sent to webmasters. This will convey to the search engine all about the efforts you have been making.

There are certain other things you can do to keep afloat after a Penguin smack:

Keep checking your link profile regularly.
Take care of the links from your own sites.
Update your anchor texts.
Aim for a varied backlinks profile by getting links from social media, and high-quality blogs, forums, video sites, and Q&A sites etc.

Penguin Recovery Success Stories and What They Teach Us

Web experts went on to work overtime to figure out why their site was smacked by the penguin and then finding the route back to all-is-well land. Research conducted by many has helped arrive at this synopsis: A site with 65 percent or more exact anchor text links was hit for sure.

One of the sites that recovered from the Penguin was WPMU.org. It could get back on its feet only after removing a whopping 15,000 site-wide footer links. They also dealt with problems like canonical URLs.

This recovery story led people to question the possibility of recovering from the penguin by getting rid of old URLs and copying that content into new URLs. Ian Howells did a study of the same on slideshare.net and came out with an affirmative answer. He experimented to see if a 301 redirect from a gutted page could pass on the ranking hit to the fresh URL and concluded that it did happen within five days.

What this means for you is:

Check your penalty level. If at page level, same domain will work for you. But if your home page is hit, you are in highly troubled waters. Only way forward is to get the links removed
In case your homepage has not been penalized, you can shift the affected pages to newer URLs but do not use the 301 redirect.
Build links from the new URLs.
Get the good links to the penalized pages transferred to your new URL.

Before Using the Disavow Tool

You should not use the disavow links tool unless you are absolutely sure that the links are negative. Even Google says warns you when you use the tool:

You have to understand that it is not an easy job. Determining the quality of the site and the links, that is why you must research well and at least get a rough understanding of the quality of the site before taking the call.

Secondly, don’t think of the disavow tool as a shortcut to repairing your link profile. There is a possibility, as Google says, that the tool doesn’t come with a 100 percent guarantee to get links removed.

Thirdly, you should not disavow an entire domain until and unless you are completely sure that every single link is useless. They may have old links from reputed sites which may work good for you. So, the bottom line is to research thoroughly before finally sending off the links.

The Road Ahead

You have done everything in your capacity and of course you want to know how effective all of this would be. This question has no direct answer. This is not a yes or no situation. Your site’s ranking was good prior to the Penguin because of these very links, and now you are in the pits because of them.

After Google removes your penalty even them there is no guarantee that you will get the same rank you were at earlier. The rise in your SERPs depends on finding out the link benefits, which came from the now removed bad links. You will now set new standards of traffic and with a clean links profile you can look for better ranking secured against any further ‘Penguin jolts.’

Navneet Kaushal

CEO at PageTraffic

Navneet Kaushal is the founder and CEO of India’s leading search marketing agency PageTraffic. You can follow him at twitter @navneetkaushal.

Alex Rojas Riva

Pages

Friday, February 27, 2015