Archive of UserLand's first discussion group, started October 5, 1998.

Re: Inktomi's Crawler and EditThisPage.Com Performance

Author:Brent Simmons
Posted:9/18/2000; 6:03:27 PM
Topic:Inktomi's Crawler and EditThisPage.Com Performance
Msg #:21489 (In response to 21486)
Prev/Next:21488 / 21490

Are you sure it's a crawler and not some sort of WAP gateway?

The User-Agent specifies that it's Slurp:

http://www.inktomi.com/slurp.html

The page on Slurp says it's a robot. (It also says it doesn't do some of the things it's been doing. Which isn't to say it's always badly behaved.)




This page was archived on 6/13/2001; 4:56:45 PM.

© Copyright 1998-2001 UserLand Software, Inc.