Get Google To Deep Crawl Your Site To Rank Well

If you want to rank well with Google you must be "deep crawled" by their spiders. Below is a checklist of the most important ways to make sure your site is crawlable. And that's good news for you, because if you fully understand the art of creating a search engine friendly site then you've got a huge advantage over your competition. Get all these components right and you'll be able to blow away your comptetion before they even see you coming. Crawlability is just a fancy way of describing how easy it is for search engine spiders to download your pages, easily access your content, and find the rest of your pages based on how you've linked your site together. Google has a multi-tiered crawling strategy which allows only the most worthy pages for the main index, barring everything else to the supplemental index which is really ignored and forgotten. If your site has a spider speed bump and you might see your page dangle out of the rankings. You want spiders to eagerly gobble up your pages and return often. You want them vigorously crawling all the corners of your site to make sure your pages get indexed and, just as important, stay indexed and well-ranked in the engines.

Here's the checklist:

• You want search engines to index your content so place it in simple HTML text format. Placing your content in Flash, AJAX, JavaScript, frames, or other technologies will make it harder for search engines to index your content.

• If you want search engines to follow your links, place them in simple HTML format. Just like your content, don't hide those links in Flash or JavaScript.

• Employ both a regular HTML sitemap and an XML sitemap.

• Use your robots.txt file to prevent search engines from wasting time indexing unnecessary files.

• Use Xenu Link Sleuth and Google Webmaster Central to make sure your site doesn't have any broken internal links that search engines won't be able to crawl.

• Spiders don't fill out forms or click submit buttons, so don't require them in order to reach pages you want indexed. If you have content behind forms that needs to be indexed, make sure you provide an alternative link for the spiders to crawl.

• Don't make spiders use your site search engine to find your site's content either.

• Be careful with session IDs, or other technologies that generate large numbers of useless pages. They can also depress the overall PageRank equity of your site by spreading it out over vast numbers of purposeless pages.

• From both a search engine and a user perspective, short and descriptive URLs are ideal. A user should be able to tell what your page is about by reading the URL the search engine has displayed for your page's search engine listing.

• In general, subdirectories are better than subdomains.

• Avoid having excessive duplicate content on each page.

These are the three things you need to do with every site to rank well in Google or any other search enginge.

1) Creating content that search engines can easily crawl and index.

2) Linking your site together so that PageRank flows correctly and is channeled to your most important pages.

3) Placing your keywords in the proper locations on your page for optimal ranking effect.