Weird Robot Crawling My Site

Discussion in 'Traffic Analysis' started by djromanof, May 5, 2010.

  1. #1
    Hello everyone!

    My site gets about 20-25 visitors per day and mostly through search engines. Today I logged into Statcounter and saw that I had almost 600 pageloads and it turns out that about 550 of them came from one specific IP address that loaded most of the pages my site has in about 7 minutes!

    The IP address is 95.211.113.138 and it is from the Netherlands... Does anyone what this is? It is obviously a robot but I cannot figure out exactly whats happening. Also many of my rankings fell more than 100 spots after that. I'm only using white and grey hat techniques to promote my site so I haven't done anything wrong.

    Any advice would be greatly appreciated!

    Thanks,
    DJ
     
    djromanof, May 5, 2010 IP
  2. nfzgrld

    nfzgrld Peon

    Messages:
    524
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Did you get the user-agent of that bot? Do you have a robots.txt file? Unless that's a legit bot, which it may not be, you need to ban it.
     
    nfzgrld, May 5, 2010 IP
  3. djromanof

    djromanof Member

    Messages:
    199
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    33
    #3
    Im afraid I dont know what exactly you are talking about... My site is a wordpress blog so the only thing I do is mess with wordpress stuff... How do I go about doing what you mentioned?
     
    djromanof, May 5, 2010 IP
  4. 50plus

    50plus Guest

    Messages:
    234
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #4
    You had a visit from a well known site scraper source. To prevent future occurrences from the same and the many other scrapers running wild on the web you should make yourself familiar with the following, just use your favourite search engine for info and tutorials :

    .htaccess (for Apache server)
    ISAPI-REWRITE (for Windows IIS server)
    robots.txt

    these will help you to prevent unwanted visitors to have access to your site and stop scrapers using your content for their own gain, which is the likely cause that your site tanked in the results.
     
    50plus, May 6, 2010 IP