Is this robots txt oky

Discussion in 'Google Sitemaps' started by Saiduly, Nov 29, 2009.

  1. #1
    Hello
    I thought might something wrong in my robots txt
    its looks
    User-agent: *
    Disallow:

    Sitemap: http://www.domainname.com/sitemap.xml.gz

    do you guys think is this oky?
    but when I want to see the sitemap the zip file pop up for download:confused:
     
    Saiduly, Nov 29, 2009 IP
  2. MaxPowers

    MaxPowers Well-Known Member

    Messages:
    264
    Likes Received:
    5
    Best Answers:
    1
    Trophy Points:
    120
    #2
    This looks good.

    This is a compressed file (shown by the .gz ending) so it will appear as a download that you must uncompress with GZip on Linux (or WinRAR on windows).

    Google can fully manage doing this, so your sitemap and robots.txt are ok. If you want to read it on the web, you will need to use an uncompressed file which does not use the .gz ending.

    Simply removing the .gz from the filename won't work. You will need to uncompress it first.
     
    MaxPowers, Nov 30, 2009 IP
  3. donluttrull

    donluttrull Peon

    Messages:
    359
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Pretty much you have it setup to allow everything (assuming the text you posted here IS what is in the robots file). It depends on what you are trying to accomplish. Many people make sure to have the site redirect either to "domain.com" or "www.domain.com" no matter which a person enters, another one people put in there are any bots that mess with the site or constant roving in their files. Another is for various directories to NOT index on their storage.

    Research robot files and you will see a million combinations, another to look at is "seo robots txt file" to see what others suggest as best for optimization and indexing purposes.
     
    donluttrull, Nov 30, 2009 IP
  4. nobel85

    nobel85 Peon

    Messages:
    256
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #4
    i think better you change your robot text
    make it new is very easy

    because dont waste your time with uncle google
    than you lose thousand dollars, better make sure you have fix it
     
    nobel85, Nov 30, 2009 IP