If you are interested, Google has an FAQ on Googlebot, Google’s Web Crawler, in the Information for Webmasters section on their site.

Find out everything you ever wanted to know about Googlebot including:

• What is Feedfetcher?
• How often will Googlebot access my web pages?
• How do I request that Google not crawl parts or all of my site?
• Why isn’t Googlebot obeying my robots.txt file?
• How do I tell Googlebot not to crawl a single outgoing link on a page?
• How do I add my feed to the search results for Google’s personalized homepage and Google Reader?

Related Posts:

• Announcing SEO Site and Blog, SEO Profile!
• Google Approves of IP Delivery
• Title Tag and Meta Tags
• Wordpress.com Stats Plugin
• Getting Out of Supplemental Results

This entry was posted on Wednesday, February 22nd, 2006 at 4:58 am and is filed under Google. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.