1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

How Do I find if google bot is visiting my site?

Discussion in 'Traffic Analysis' started by sudarshannus, Oct 16, 2008.

  1. #1
    How do I find when google bot is visiting my site? Is there any way to check that out?
    SEMrush
     
    sudarshannus, Oct 16, 2008 IP
    SEMrush
  2. bacanze

    bacanze Peon

    Messages:
    2,419
    Likes Received:
    127
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Your CPanel web logs should do.
     
    bacanze, Oct 16, 2008 IP
  3. fri3ndly

    fri3ndly Active Member

    Messages:
    111
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    60
    #3
    Thats if you have Cpanel.

    If you do simply go to the 'Latest Visitors' tab or Webalizer and that will tell you :)
     
    fri3ndly, Oct 16, 2008 IP
  4. sooribabu

    sooribabu Active Member

    Messages:
    65
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    53
    #4
    if your site is developed in php add this/customise this code to check google and yahoo bots visit

    <?php
    $email = "youremail@youremail.com";

    if(eregi("yahoo! Slurp",$_SERVER['HTTP_USER_AGENT']))

    {

    mail($email, "The Yahoo bot came to call yoursite",

    "Yahoo has visited: ".$_SERVER['REQUEST_URI']);

    }

    if(eregi("googlebot",$_SERVER['HTTP_USER_AGENT']))

    {
    mail($email, "The Googlebot came to call your site",

    "Google has visited: ".$_SERVER['REQUEST_URI']);
    }
    ?>
     
    sooribabu, Oct 16, 2008 IP
    hans likes this.
  5. justinlorder

    justinlorder Peon

    Messages:
    4,163
    Likes Received:
    61
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Google webmaster tools tell you everything. Didn't you know?
    Webmaster tools even let you know how many byte did google download from your site.
    Every details there, you can't miss that .
    another way is sooribabu 's php code.
     
    justinlorder, Oct 16, 2008 IP
  6. abercrombie

    abercrombie Peon

    Messages:
    654
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #6
    i usually use cpanel > awstats and there's a section to indicate last visits and how many times your robots.txt file has been hit.
     
    abercrombie, Oct 16, 2008 IP
  7. andaman

    andaman Peon

    Messages:
    93
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    put stat counter into your website and check it
     
    andaman, Oct 16, 2008 IP
  8. angilina

    angilina Notable Member

    Messages:
    7,824
    Likes Received:
    186
    Best Answers:
    0
    Trophy Points:
    260
    #8
    You can use google webmaster tools to see when google bot visited your site. I saw somewhere in this forum that we can also see google bot via google analytics. I suggest you search for some details, may be you will find the way to use google analytics to see google bot.
     
    angilina, Oct 16, 2008 IP
  9. Nonny

    Nonny Notable Member

    Messages:
    2,093
    Likes Received:
    120
    Best Answers:
    0
    Trophy Points:
    210
    #9
    You should get a Google Webmaster Tools account, if you haven't already.
    http://www.google.com/webmasters/tools/

    That will give you stats on Google's crawls of your web site, plus lots of other information.

    I don't think Google Analytics has that data, since the bots don't execute the javascript used for tracking.
     
    Nonny, Oct 16, 2008 IP
  10. gitamurtibhandar

    gitamurtibhandar Peon

    Messages:
    95
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #10
    this is check your cpanal
     
    gitamurtibhandar, Oct 16, 2008 IP
  11. hans

    hans Well-Known Member

    Messages:
    2,924
    Likes Received:
    126
    Best Answers:
    1
    Trophy Points:
    173
    #11
    my preferred method in addition to correct below methods is to download ( and save for 1+ yrs ) all access_log files for offline-processing by suitable software.

    any decent available offline log analyzer tool gives you stats for bogs.
    the offline method has the advantage to have all access data on hand in case of security problems. for that reason I also keep access_log files for up to a year or longer. to have back in time data for in depth research if needed. I needed - you'll need as well sooner or later!
     
    hans, Oct 16, 2008 IP
  12. SEOpaw

    SEOpaw Peon

    Messages:
    437
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #12
    1. Cpanel web logs
    2. Statcounter
    3. analytics
    4. web master tool
     
    SEOpaw, Oct 16, 2008 IP
  13. genuineseo

    genuineseo Active Member

    Messages:
    783
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    58
    #13
    you can try through Cpanel web logs or analytics.
     
    genuineseo, Oct 17, 2008 IP
  14. Mr Brightside

    Mr Brightside Peon

    Messages:
    66
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #14
    Google Webmaster Tools is a good start and as the previous poster mentioned Analytics can also give a good indication if the search bots are crawling your pages as it can be displayed in the 'Content' tab of most popular pages (if its gaining visitors its a good indication that the pages are indexed)
     
    Mr Brightside, Oct 17, 2008 IP
  15. kmy02

    kmy02 Peon

    Messages:
    3
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #15
    If your website is integerated with Google Analytics, you can check it at Visitors > Browser Capabilities > Browsers

    You can also check it through Google Webmaster Tool.

    Cheers..
     
    kmy02, Oct 17, 2008 IP
  16. ciplseo

    ciplseo Peon

    Messages:
    338
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #16
    ciplseo, Oct 17, 2008 IP
  17. jiggymehta

    jiggymehta Peon

    Messages:
    446
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #17
    I use google webmaster tools and thats pretty much it..
     
    jiggymehta, Oct 17, 2008 IP
  18. kidsnscience

    kidsnscience Banned

    Messages:
    411
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #18
    just check google last cache date of crawl page
     
    kidsnscience, Oct 17, 2008 IP
  19. bytuncay

    bytuncay Peon

    Messages:
    1
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #19
    www.googlebotlog.com

    googlebot crawling activity logging

    Add your site and your site crawling activities of the Save All Spiders Bot, Online Check
     
    bytuncay, Oct 16, 2009 IP
  20. seolion

    seolion Active Member

    Messages:
    1,495
    Likes Received:
    97
    Best Answers:
    0
    Trophy Points:
    90
    #20
    Lol! Thats the easiest way to fill your inbox if you include it in all pages and there is a lot of crawler activity.

    As many have already mentioned, Web server logs are the best (may not be easiest) way to see the crawler activity.

    If you can get a day's log or something simply search for Googlebot in it.
     
    seolion, Oct 24, 2009 IP