While performing SEO (Search Engine Optimization) on your website, you may notice a sudden sharp decline in visitors from a particular search engine. This can happen for any number of reasons, but the number one suspect is usually a search engine penalty has been applied to your site.
Types of Search Engine Penalties
Duplicate Content – The search engine algorithm will filter out what it considers to be duplicate content. This can happen if a certain number of words or phrases exceed a pre-determined level as specified by the algorithm. The filter works by dropping either the newest content, or replacing the older content if the page has been determined to have more “weight”.
Keyword Spam – If a page is found to have been created solely for the purpose of drawing visitors with keywords not related to the topic of the page, the page in question would be dropped from the index. This is usually applied manually by a human reviewer, but can be automatically applied if the spider detects it.
Cross Linking / Interlinking – This involves link schemes set up between sites that are closely related, links are created solely for the purpose of increasing link popularity. These schemes are detected usually because the sites reside on the same group of servers which contain similar IP addresses.
Cloaking / Masking – Pages created specifically for spiders containing keywords or phrases, human visitors are redirected to a totally different page containing different content than what the search engine spider sees. These are usually caught by human reviewers after a complaint, or if a sophisticated search engine spider is smart enough to catch on.
Sandbox – The sandbox is a sort of incubator, new sites or pages might be added to the sandbox to give them time to develop to ensure that the pages will be around long enough to warrant a place in the index. This penalty, if you can call it that is hotly a debated topic however as nobody has been able to positively confirm that it exists.
How to avoid being penalized
This part is simple, you can avoid being penalized by the search engines by simply practicing good old fashioned “White Hat” SEO techniques. Build good pages with useful content and you will be rewarded, use the “Black Hat” SEO techniques and you may experience good results, but it will be short lived.
If you are going to dabble in some of the black arts of the SEO world, at least try to keep in minimal and not obvious, remember to not only consider the human visitors, but the spiders as well.
How to recover from a penalty
If you have found yourself penalized, don’t panic. First find if you have surely been handed a penalty. Do a search for your site name, and also a site search to see if you have been totally removed.
Second, narrow down the reasons for which you may have been penalized:
- Have you added a new link recently? – Make sure you are not linking to a “bad neighborhood” or that one of your link partners has not changed their site’s topic that may be considered bad.
- Have you added a new page? – Check to make sure you have not added a page which might have offended the spider, either with keyword spamming or duplicate content.
- Have you changed your directory structure? – Changing the naming structure of your site without adding redirects to the old pages will have an almost instant response from the search engine. Always make sure to add a redirect for any pages that you are going to rename or move.
- Is the spider able to access your site? – Run a DNS trace route to determine if your site is visible to the entire internet. Also make sure that your robots.txt file is up-to-date and is not disallowing the spiders visit to either your entire site or certain directories.
If you have found any one of these to be true, rectify the problem as quickly as possible and wait to see if the spider returns and your site re-appears in the index. If after 1-3 weeks your site has still not returned to the index, notify the search engine operator to inquire the status and if there are further problems that need to be corrected.