Can robot.txt effect your earning?

Discussion in 'AdSense' started by checkmeeveryday, Feb 4, 2009.

  1. #1
    Can robot.txt effect your earning?

    If yes, how to optimise it?
     
    checkmeeveryday, Feb 4, 2009 IP
  2. Johnny 3 Tears

    Johnny 3 Tears Banned

    Messages:
    91
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    can anyone tell me what is that :D ?
     
    Johnny 3 Tears, Feb 4, 2009 IP
  3. Adpubster

    Adpubster Peon

    Messages:
    4,017
    Likes Received:
    153
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Adpubster, Feb 4, 2009 IP
  4. Johnny 3 Tears

    Johnny 3 Tears Banned

    Messages:
    91
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #4

    thanks ! will read it
     
    Johnny 3 Tears, Feb 4, 2009 IP
  5. checkmeeveryday

    checkmeeveryday Banned

    Messages:
    464
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I read it, but dont understand at all. Can somebody tell me in English :p
     
    checkmeeveryday, Feb 4, 2009 IP
  6. Adpubster

    Adpubster Peon

    Messages:
    4,017
    Likes Received:
    153
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Tell you what in English?
     
    Adpubster, Feb 4, 2009 IP
  7. joe3082

    joe3082 Peon

    Messages:
    158
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I don't think there is a way to optimize your robots.txt

    Just get one already made

    (google search)
     
    joe3082, Feb 4, 2009 IP
  8. Adpubster

    Adpubster Peon

    Messages:
    4,017
    Likes Received:
    153
    Best Answers:
    0
    Trophy Points:
    0
    #8
    The only real thing to be concerned about is to make sure it validates and that you don't exclude legit crawlers...Google's webmaster tools will help you with the former.
     
    Adpubster, Feb 4, 2009 IP
  9. PyramidSeo

    PyramidSeo Peon

    Messages:
    84
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Check the info in google.com as key word robot.txt
     
    PyramidSeo, Feb 4, 2009 IP
  10. checkmeeveryday

    checkmeeveryday Banned

    Messages:
    464
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    0
    #10
    Thanks you really help me :p
     
    checkmeeveryday, Feb 4, 2009 IP
  11. proxywhereabouts

    proxywhereabouts Notable Member

    Messages:
    4,027
    Likes Received:
    110
    Best Answers:
    0
    Trophy Points:
    200
    #11
    I could not understand.

    Take massmirror as an example.
    Look at their robots.txt
    They block all robots but then they got authority in google.

    So, meaning even if I block robots, I can still be indexed normally by google? No penalty will be given right?
     
    proxywhereabouts, Feb 4, 2009 IP
  12. ToddAddictive

    ToddAddictive Banned

    Messages:
    34
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #12
    of course it would, best keeping robots.txt
     
    ToddAddictive, Feb 4, 2009 IP
  13. Adpubster

    Adpubster Peon

    Messages:
    4,017
    Likes Received:
    153
    Best Answers:
    0
    Trophy Points:
    0
    #13
    What specifically in robots.txt would affect your earnings (I mean, other than what I said about having it disallowing robots, including the adsense crawler which may actually be a different beast)
     
    Adpubster, Feb 4, 2009 IP