1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

yes it is important to keep robots.txt

Discussion in 'robots.txt' started by varsha reddy, Feb 1, 2012.

  1. #1
    The structure of a robots.txt is pretty simple (and barely flexible) – it is an endless list of user agents and disallowed files and directories. Basically, the syntax is as follows:
    User-agent:
    Disallow:
    “User-agent” are search engines' crawlers and disallow: lists the files and directories to be excluded from indexing. In addition to “user-agent:” and “disallow:” entries, you can include comment lines – just put the # sign at the beginning of the line:
    # All user agents are disallowed to see the /temp directory.
    User-agent: *
    Disallow: /temp/
     
    varsha reddy, Feb 1, 2012 IP
  2. tiffanywilliams12i2

    tiffanywilliams12i2 Peon

    Messages:
    164
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    yes, robots file is very imporant always improtant to keep it going
     
    tiffanywilliams12i2, Apr 10, 2012 IP