1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

The function of robots.txt

Discussion in 'Google Sitemaps' started by kie, Nov 20, 2007.

  1. #1
    Frens I have read lot of articles about robots in DP
    So I just want to ask for a conclusion
    Is the robot function only for:
    - allowing and dissalowing site for being crawl by spider
    - for letting spider know our sitemap, ex: for google spider to know our sitemap
    Is it only for that? :) if there is still another fuction please let me know, thanks :)
    SEMrush
     
    kie, Nov 20, 2007 IP
    SEMrush
  2. astup1didiot

    astup1didiot Notable Member

    Messages:
    5,927
    Likes Received:
    269
    Best Answers:
    0
    Trophy Points:
    280
  3. kie

    kie Peon

    Messages:
    330
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #3
    I have reviewed them, and I just want to know the conclusion of it :) so I ask frens bout it again..coz I am still a newbie :p
     
    kie, Nov 21, 2007 IP
  4. astup1didiot

    astup1didiot Notable Member

    Messages:
    5,927
    Likes Received:
    269
    Best Answers:
    0
    Trophy Points:
    280
    #4
    What excatly do you want to know about it?
     
    astup1didiot, Nov 21, 2007 IP
  5. kie

    kie Peon

    Messages:
    330
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I mean the function of robots.txt besides, these:
    - allowing and dissalowing site for being crawl by spider
    - for letting spider know our sitemap, ex: for google spider to know our sitemap
    Thanks for helping ssandecki :)
     
    kie, Nov 21, 2007 IP
  6. chandubhai

    chandubhai Banned

    Messages:
    556
    Likes Received:
    27
    Best Answers:
    0
    Trophy Points:
    0
    #6
    As far as i know it is for... Come on... the name itself suggests what it is for.
     
    chandubhai, Nov 22, 2007 IP
  7. kie

    kie Peon

    Messages:
    330
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Man..I am a newbie so you can not just give me answer like that..if you dun want to help its fine but dun give such answer, thanks :)
     
    kie, Nov 22, 2007 IP
  8. astup1didiot

    astup1didiot Notable Member

    Messages:
    5,927
    Likes Received:
    269
    Best Answers:
    0
    Trophy Points:
    280
    #8
    Don't worry, I dropped him\her some -REP for that negative response which was uncalled for.

    http://en.wikipedia.org/wiki/Robots.txt

    The wiki entry is great for what your asking, it will explain a lot better than the last URL I gave you, I believe this should answer your basic question on what robots.txt can be used for and its real use on the internet.
     
    astup1didiot, Nov 22, 2007 IP
  9. chandubhai

    chandubhai Banned

    Messages:
    556
    Likes Received:
    27
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Man... I did not mean to offend you.
     
    chandubhai, Nov 22, 2007 IP
  10. kie

    kie Peon

    Messages:
    330
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #10
    Thanks ssandecki, you are a real helper :) now I know more already thanks again n God Bless :)
     
    kie, Nov 22, 2007 IP