1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Help needed - robts.txt and sitemap

Discussion in 'robots.txt' started by jack_sparrow, Aug 3, 2007.

0
Page 5 of 5
  1. halo2and34u1

    halo2and34u1 Peon

    Messages:
    85
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #81
    visit vision fez's sites they explain it in a good fashion
     
    halo2and34u1, May 1, 2009 IP
  2. Nancy1234

    Nancy1234 Peon

    Messages:
    93
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #82
    Thanks for this answers i think this will also save your Your Bandwidth.
     
    Nancy1234, May 4, 2009 IP
  3. e-marketing guru

    e-marketing guru Active Member

    Messages:
    228
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #83
    It is important
     
    e-marketing guru, May 21, 2009 IP
  4. budz

    budz Active Member

    Messages:
    154
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #84
    robots.txt
    just see and learn the example of others site robots.txt

    which part u are not allowed to index in search engine so put in disallow path
    that is simply easy
     
    budz, Jun 5, 2009 IP
  5. affiliatesense99

    affiliatesense99 Banned

    Messages:
    110
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #85
    I think, if you don't want to control spiders then there is no need to use robots.txt, for sitemaps xml is must and useful.
     
    affiliatesense99, Jun 19, 2009 IP
  6. alphamonk

    alphamonk Peon

    Messages:
    116
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #86
    Does disallowing access to certain folders make those folder more susceptible to hack attacks from bad guys.
     
    alphamonk, Jun 27, 2009 IP
  7. AutoTerminal

    AutoTerminal Active Member

    Messages:
    184
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    51
    #87
    If i want every page of my site to be visited by Google and Yahoo, does it need to have robot.txt?
     
    AutoTerminal, Nov 16, 2009 IP
  8. raquo

    raquo Peon

    Messages:
    108
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #88
    you can find complete references about robot.txt in google :)
     
    raquo, Apr 8, 2010 IP
  9. Damager

    Damager Peon

    Messages:
    1
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #89
    Forget robots.txt file. It is nothing important.

    Learn more on sitemap specially if you site have thousands of pages. make more than one sitemaps if needed.

    I guess I am bit late to reply here.
     
    Damager, May 7, 2010 IP
  10. kevoh99

    kevoh99 Peon

    Messages:
    175
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #90
    Robots.txt is cool to specify which parts of your site should be indexed and which ones should not be
     
    kevoh99, May 25, 2010 IP
Page 5 of 5