Web Crawlers

[dougspaulding]

What in the heck is a Slurp Spider, and what is its purpose? I see it crawling around the site every evening late. Makes me feel like getting some insecticide or something.

Forgive me - computer illiterate.

[Zombie-F]

A Spider is a search engine's tool for indexing web pages. The spider "crawls" to each page on the forums and adds it to the search engine. Then, when someone does a search with certain keywords found in the page that was indexed by the spider, it comes back to that person's search results.

There's also a Google Spider... watch out for him. He has really large fangs and poisonous venom.

[dougspaulding]

I've heard of those Googlebots - shiver!

Thanks for the explanation.

[Dr Morbius]

What in the heck is a Slurp Spider, and what is its purpose? I see it crawling around the site every evening late. Makes me feel like getting some insecticide or something.

Forgive me - computer illiterate.

What do you mean, you "see it?"

[dougspaulding]

I see it here:

http://www.unpleasantstreet.com/forums/online.php

Either that, or I'm seeing spiders in my dreams - not a pleasant thought!