How to use Robots.txt to get only front page indexed?

Discussion in 'robots.txt' started by thundercow, Dec 12, 2008.

  1. #1
    I can't figure out how to do this. I want to use a robots.txt file to let the search engines index ONLY the front page, and no other pages.

    I built a directory for my client, a geographic directory of self storage businesses. There are like 8000 categories, and no content yet. I want to get the front page indexed without indexing 7999 empty pages.
     
    thundercow, Dec 12, 2008 IP
  2. qszheng

    qszheng Peon

    Messages:
    7
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    You may find answer here: google.com/support/webmasters/bin/answer.py?hl=en&answer=40360
     
    qszheng, Dec 14, 2008 IP
  3. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #3
    You can use this code:

    User-agent: *
    Disallow: /
    Allow: /index file
     
    manish.chauhan, Jan 20, 2009 IP
  4. suvendug

    suvendug Member

    Messages:
    382
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    28
    #4
    To avoid index you need to add this meta tag <meta name="googlebot" content="noindex, nofollow, noarchive" />

    Please understand you need to add this tag in those pages, which you don't want to index :)
     
    suvendug, Jul 21, 2012 IP
  5. huseyinyilmaz

    huseyinyilmaz Greenhorn

    Messages:
    50
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    6
    #5
    thanks everbody good information
     
    huseyinyilmaz, Jul 23, 2012 IP
  6. mr.siiin

    mr.siiin Peon

    Messages:
    12
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    how can i ceat robots for adsense
     
    mr.siiin, Jul 30, 2012 IP
  7. Jason0078

    Jason0078 Greenhorn

    Messages:
    22
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    11
    #7
    Use plugin KB robot.txt
     
    Jason0078, Jul 30, 2012 IP