Google's Supplemental Index, and How to Get Out

I think firstly you should watch the video, as you can see it's from SEOmoz, which is a top SEO Company based in Seattle, Washington. I think the video demonstrates a great basic understanding of the supplemental index and we'll go forward from there.

Lets begin by defining the purpose of the Supplemental Index. Webopedia says:

In search engines, a supplemental index is comprised of Web results compiled with fewer restraints on sites than sites that are crawled for a main search engine index. Google is one search engine that uses both a main index and a supplemental index. In some instances restrictions for inclusion in the main index may be ignored allowing your Web site to be crawled by the supplemental index, or the supplemental index crawls those pages that the main index had difficulty with. Web pages that are
listed in a supplemental index usually are not returned as results in Web search
queries as often as those included by the main search index.

So we can see it is an index that is easier to get in to, especially if you messed up your pages in some way. Additionally, they don't return in general search results. I see it as a staging area for pages that require repair.

Now lets identify your sites indexed pages and pages in the supplemental.

Load Google Search and type in for example site:seocog.com
this will show you all of your indexed pages in the MAIN search
index.
Now add *** -sjpked or *** -s to the end and it like this site:www.seocog.com *** -sjpked now you see the pages contained in the
supplemental index.

Lets try to figure out how and why they ended up there. There are a great deal of ways to end up there, many people will try to tell you it is a staging area pages go to before regular indexing when they are new, Google just does not support this theory at all. So how does it happen then, well there are some proven theories and some things Matt Cutts and even Vanessa Fox has said that will land you there.

Duplicate content: I can tell you having just launched and indexed 2 new sites in the last 30 days, that the answer to the begging question "Can I use the same page/template site wide" is Yes. I have done this with no penalties. If Google deem pages as duplicate the bot will pick one to index and dump the rest in supplemental.
Extremely slow page load time. This should be under 30 seconds at 28.8.
Missing or inaccurate Doc Type Declarations. This is W3's list and code of recommended and proper Doc Types.
Low code to text/content ratio. You can use this tool to determine the ratio score per page, which should be 35-45% for good indexing strength.
Odd or unrelated content trail links. For example you click on Joe's Hammer Shop in search results....on the main page you are greeted by menu choices like Hattie's Puppy Pedicures, and Fred's Wheelchair Sales. Looks spammy to the bot, and bingo you land in the supplemental.
Pages with too many errors, that may be impossible or too difficult to crawl. Check them here. Its great if they validate, not totally necessary.
Reciprocal links, or at least too many of them....consider reducing them. Get rid of site rings and link farm or groups, Google is looking for organic links.
Buying Links..........Just DON'T do it. If Google doesn't find out on its own someone will rat you out.

Basically any problem short of banning caliber will land you in the supplemental index. Like I said though, consider it a staging area, or repair shop. At least you know there is a problem, other engines like DMOZ just file it in the trash bin and never even let you know.

How to get out of the supplemental index. Just like when you originally built your page, there is a process...this one is much the same. We are going to cover a step by step proposed checklist to help you identify and correct the problem.

Go here and check the html, meta, and load time problems until your score is 8.5 or better.
Go here and check your code to text ratio depending on the type of site this can be challenging but try to raise it at least 10% with RELEVANT text. The goal is 35%.
Delete and unrelated links to outside pages.
Check your duplicate content issues with this tool. This can be rather complicated stuff, you may need tech help from your server. We have an article on preventing duplicate content, so we won't cover the details here again.
Work on your organic and trusted links, these will be your "references".
Get some backlinks to internal pages, this is more easily accomplished with landing pages if you don't have them already.

I am quite sure there are other techniques, and perhaps even more to add. We try to keep this as straightforward here to accommodate all levels of webmaster skill, so even new webmasters can come here for help and DIY. If you have anything to add, feel free to comment.

Peace and SEO

Baby

SEOCog.com