The means by which a search engine moves through the web is called spidering the web, also known as web crawling. In other words, it crawls through the world wide web like a real spider crawls over a spider web - one thread at a time. This is an automatic process the search engines continuously engage in.
When a search engine comes to your site, it begins on the page of entry (may not necessarily be the home page -- depends where the spider came from). As it examines your page, if it comes across another link, it will follow that link to where it leads (i.e. follow another thread in the web). That link could be internal to your site, in which case the search engine continues to examine your site. It may also lead outside your site, in which case it has gone elsewhere.
When you request that a search engine index your site, you are asking for the search engine to crawl through your site, examine all pages, or spider your site completely. Be sure to limit your requests to no more than once every 2 months. Over requesting indexing is a surefire way to make the search engines ignore you.
When the search engine spiders your site, it examines your meta tags and content to determine how your site should be indexed in the search engine. Your content is given the greatest weight, but your meta tags are also very important for gaining valuable indexing.
More information:
What Does a Spider See When It Visits Your Site?
Just What Do Spiders Look For?
What is a Webcrawler on Wikipedia
Subscribe to:
Post Comments (Atom)
0 comments:
Post a Comment