will User-agent: * Disallow: Allow: / result in duplicate content issues?

Discussion in 'robots.txt' started by alvinchua91, Oct 5, 2012.

0
  1. #1
    Hi I am curious, because recently google webmaster told me that A LOT of my URLS are suddenly blocked recently, and by robots.txt file though I didn't do anything.

    Thus I tried to get rid of the robots.txt files by making my blog

    originally it was
    User-agent: *
    Disallow: /search
    Allow: /

    NOW it is:
    User-agent: *
    Disallow:
    Allow: /

    Is that correct?


    Will my current one, result in duplicate content issues? Thanks so much for your help!
     
    alvinchua91, Oct 5, 2012 IP
  2. alvinchua91

    alvinchua91 Well-Known Member

    Messages:
    287
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    121
    #2
    hi.. can anyone help?
     
    alvinchua91, Oct 5, 2012 IP
  3. stataqw

    stataqw Peon

    Messages:
    37
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    User-agent: *
    Disallow:
    Allow: /
    This code is correct for robot file.
     
    stataqw, Oct 5, 2012 IP
  4. alvinchua91

    alvinchua91 Well-Known Member

    Messages:
    287
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    121
    #4
    so its fine?
     
    alvinchua91, Oct 6, 2012 IP
  5. ntmedia

    ntmedia Active Member

    Messages:
    118
    Likes Received:
    11
    Best Answers:
    7
    Trophy Points:
    90
    #5
    wait what's the point of that code is everything is allowed and nothing is disallowed?

    Simple delete all the code you've mentioned, it will have the same effect. Other than that it's "fine".
     
    ntmedia, Oct 7, 2012 IP
  6. Traditione

    Traditione Well-Known Member

    Messages:
    731
    Likes Received:
    23
    Best Answers:
    1
    Trophy Points:
    120
    #6
    hahahaha ^ this exactly

    "I'm going to make a door, and always leave it open." ....o...ok....

    OP, did you try just removing everything first? Maybe your site's just going through a little dance. Get a sitemap and robots validator to make sure nothing got changed. Might as well check .htaccess for any changes recently too
     
    Traditione, Oct 7, 2012 IP