Seen this spider before?

Hi there...

Has anyone ever seen this before?

When: Tue Aug 3 10:27:42 2004 Who: Mozilla/3.0 NAVIO (dtv0) Where: /(Details Removed)/(keyword)-(keyword)-chateau.html From: 192.220.74.15

It's been visiting one of my sites a HELL of a lot...

Constantly trying to visit certian pages over and over...

Not too sure whether to ban it.

Also, has anyone seen much activity from Slurp! Recently? It's been going CRAZY on my site.

DEEP crawls. Lots of pages have been indexed within the last 3 days, pretty much non-stop.

Anyone else seen anything similiar recently?

 

Can't recall that one, but there are so many now that you will need a spider identity site to figure them out, I know that I had bookmarked a few of those computer robotic sites.

Shawn or someone else could give you a list of these robots or a website or two that will help you identify them.

Just do a search on "robot spiders" and you will find the websites.

I would not ban any of them without knowing who they are, a lot of universities have their own spiders, even a lot of large companies do their own crawling now.

In your case SE, since you have so much money maybe Richard Branson is keeping an eye on you for a possible buy out

 

hehe - That would be nice