Updating Robots.txt and Sitemap.sml files

Discussion in 'HTML & Website Design' started by MarkStrobel3367, Sep 20, 2012.

0
  1. #1
    This might seem like a stupid question, but do I need to update my website's Robots.txt and/or Sitemap.sml whenever I add pages to my site? Thanks.
     
    MarkStrobel3367, Sep 20, 2012 IP
  2. GMF

    GMF Well-Known Member

    Messages:
    855
    Likes Received:
    113
    Best Answers:
    19
    Trophy Points:
    145
    #2
    robots.txt - depends on what's in it

    Sitemap.xml - yes, needs to be updated

    (except you have some automated software that does this)
     
    GMF, Sep 21, 2012 IP
  3. devikap

    devikap Peon

    Messages:
    5
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Right we need to update sitemap.xml.
    its good that you have asked this question as sometimes an important page is added we have to update sitemap.xml
     
    devikap, Sep 21, 2012 IP
  4. MarkStrobel3367

    MarkStrobel3367 Well-Known Member

    Messages:
    116
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    103
    #4
    Thank you so much for your response.
     
    MarkStrobel3367, Sep 21, 2012 IP
  5. Kousha

    Kousha Peon

    Messages:
    34
    Likes Received:
    0
    Best Answers:
    1
    Trophy Points:
    0
    #5
    If you are using any CMS or a dynamic website, it may update sitemap automatically.
     
    Kousha, Sep 21, 2012 IP
  6. deathshadow

    deathshadow Acclaimed Member

    Messages:
    9,732
    Likes Received:
    1,999
    Best Answers:
    253
    Trophy Points:
    515
    #6
    99% of normal websites, wasting time or effort on either of those, is, well... a waste.

    robots.txt should only be needed to tell search engines NOT to index things; as a rule of thumb, WHY WOULD YOU WANT TO DO THAT?!? I mean, sure, there are special cases like example directories where you don't want to steal a clients search mojo, but apart from that, just what are you doing that would require a robots.txt apart from a empty one to avoid 404's? Remember, there is no such thing as 'follow' or 'index', the valid values are 'nofollow' and 'noindex'.

    Sitemaps are one of those bits of noodle-doodle hoodoo-voodoo that only makes sense if your on-site navigation gets down on it's knees in front of the proverbial equine. If all the pages in your site are linked to by other pages in the site so search engines can spider it properly, a sitemap.xml is pointless, redundant, and again, a waste of time and effort.

    Decent article on the topic:
    http://www.dailyblogtips.com/qa-what-is-a-sitemap-do-i-need-it/

    No matter what the SEO-tards who don't know enough about how this stuff actually works to open their mouths on the subject or the whackjobs who want us to throw extra layers of XML at EVERYTHING tries to tell you. Basically if you "need" a sitemap, IMHO there's something horrifically wrong with how the site is built.

    Though admittedly with most people sleazing out HTML 3.2, then slapping 4 tranny or 5 lip service on it, "designing" some goof-assed inaccessible PSD then shoe-horning content into it, ignoring all of the progress STRICT and all the advantages of separation of presentation from content -- it's entirely possible that there is indeed something horrifically wrong.
     
    deathshadow, Sep 21, 2012 IP
  7. MarkStrobel3367

    MarkStrobel3367 Well-Known Member

    Messages:
    116
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    103
    #7
    Thank you so much for your replies.
     
    MarkStrobel3367, Sep 21, 2012 IP
  8. alexkboorman

    alexkboorman Peon

    Messages:
    70
    Likes Received:
    0
    Best Answers:
    1
    Trophy Points:
    0
    #8
    I feel like maybe my presence is no longer needed on these forums.
     
    alexkboorman, Sep 22, 2012 IP