Google webcache banned by robots.txt

Discussion in 'robots.txt' started by n00bl3t, Nov 25, 2010.

0
  1. #1
    Hello,

    How do I prevent this? Is this something under my control? I keep getting this every week. A few URLS from my site getting banned or is telling GOOGLE WEBCACHE not to crawl through ROBOTS.TXT
     
    n00bl3t, Nov 25, 2010 IP
  2. shadowking

    shadowking Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I am not getting the exact problem.
    Have you made a disallow statement in the robot txt file?
     
    shadowking, Nov 27, 2010 IP
  3. BrandDoctor

    BrandDoctor Well-Known Member

    Messages:
    1,136
    Likes Received:
    14
    Best Answers:
    0
    Trophy Points:
    190
    #3
    They are not getting banned.its intentional blocking to prevent duplicate content.and you not need to worry as they are not errors but just warning but no need to worry.
     
    BrandDoctor, Feb 4, 2011 IP
  4. gmendoza_0101

    gmendoza_0101 Peon

    Messages:
    39
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    to prevent duplicate content to your site google bot block some url of your site
     
    gmendoza_0101, Feb 7, 2011 IP