1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Robots.txt file can't be seen but SEOQuake says it exists?

Discussion in 'robots.txt' started by Goran Nikolov, Jul 11, 2013.

0
  1. #1
    Hey there, i've searched the internet and asked on bunch of forums about this for a while and nobody seems to respond. This is the last place so i hope there will be generous people and answer me this question.

    Why when i type the robots.txt on this website (http://imustdateher.com/) i can't see the file, it redirects me to the home page but in the URL section stays what i've typed? Here it is you can see for your self: http://imustdateher.com/robots.txt

    Thanks in advance for this

    Cheers
     
    Goran Nikolov, Jul 11, 2013 IP
  2. sarahk

    sarahk iTamer Staff

    Messages:
    28,500
    Likes Received:
    4,460
    Best Answers:
    123
    Trophy Points:
    665
    #2
    Its a standard wordpress site so the file doesn't actually exist. Go into the admin settings and look at the bottom of the reading settings

    [​IMG]
    that box should be unchecked.

    Why you don't see the normal file though is unclear
     
    sarahk, Jul 11, 2013 IP
  3. Goran Nikolov

    Goran Nikolov Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    1
    #3
    Hey sarahk thanks for your reply. I know that that box is if you want to be indexed or not. The website i provided above is not mine i just wanted to know why and how is that done. The SEOquake firefox plug in says that there is a robots.txt file.

    Thanks
     
    Goran Nikolov, Jul 11, 2013 IP
  4. sarahk

    sarahk iTamer Staff

    Messages:
    28,500
    Likes Received:
    4,460
    Best Answers:
    123
    Trophy Points:
    665
    #4
    Aaaah, I don't know seoquake but they might be doing something funky like hiding it from human visitors and showing it to the bots. Try doing a test with curl to see what you get
     
    sarahk, Jul 11, 2013 IP
  5. Goran Nikolov

    Goran Nikolov Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    1
    #5
    Yep they might, but how do you mean to test it with curl, sorry but i am not so advanced in those stuff.. :(
     
    Goran Nikolov, Jul 11, 2013 IP
  6. sarahk

    sarahk iTamer Staff

    Messages:
    28,500
    Likes Received:
    4,460
    Best Answers:
    123
    Trophy Points:
    665
    #6
    I'd worry about something else personally. I see little advantage in this "trick".
     
    sarahk, Jul 11, 2013 IP
  7. Goran Nikolov

    Goran Nikolov Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    1
    #7
    Well i can't see what the advantage is but definitely there is something.. :) + the same website has a pretty nice structure so google ranks the same very high.

    I see you are a web developer, can i have you on skype?
     
    Goran Nikolov, Jul 11, 2013 IP
  8. MokricaIrina

    MokricaIrina Member

    Messages:
    96
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    48
    #8
    Your http://imustdateher.com/robots.txt page serves 200 server response, that's why all plugins say that the file exists. If it would return 404, the plugin would work correctly.
     
    MokricaIrina, Jul 17, 2013 IP