How do you know when your site has been deep crawled?

It is easy to tell when spiders and bots visit but how (as in where in my stats etc.) should I look to find out more about how deep they crawled?

About how often do the major bots do so?

Thanks!

 

I have no idea, plz say something about me also.

 

you will have to check your server logs. Or you can try to derive this info from "number of pages crawled per day" in your webmaster tools.

 

Just add your site to Google Webmaster Tools and get your crawl stats from there ;-) SY

 

Thank u so much for information

 

i use the google website, know the average of crawled, the high and the low!

 

To get the intimation as Google or Yahoo or MSN crawlers crwal your webpage, you can write a PHP script which checks the User Agent as well as the confirms it's credibility through reverse and forward DNS lookup.

Check this article on www.expertsguide.info/2010/08/send-mail-when-googlebot-crawls-a-webpage/ , for already written code in PHP.


Thanks!

 

You could also just do a "site:" search over your domain in each of the search engines to see how many pages you have indexed.

 

I think from the Google web master tools you can get the details from your crawled page.Just try it.