1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Disallow: / ????

Discussion in 'robots.txt' started by Ridhdhi, Jul 3, 2008.

0
  1. #1
    Hi all,

    I have placed below code robot.txt of my site:

    User-agent: *
    Disallow: /

    Will this stop crawling of my home page supposed name as www.hfg.com/ ???

    Please give me a quick ans??

    I am in great need of this.
     
    Ridhdhi, Jul 3, 2008 IP
  2. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #2
    yes..it'll block your whole website....For more information visit Robots.txt
     
    manish.chauhan, Jul 4, 2008 IP
  3. Ridhdhi

    Ridhdhi Banned

    Messages:
    184
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Ok then what should i write to just make my home page crawl and make other pages disallow?
     
    Ridhdhi, Jul 4, 2008 IP
  4. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #4
    Use these instructions in your Robots.txt:
    User-agent: *
    Disallow: /
    Allow: /index.shtml

    This'll only index your Domain and restrict other internal pages and subfolders....:)
     
    manish.chauhan, Jul 4, 2008 IP
    Ridhdhi likes this.
  5. Ridhdhi

    Ridhdhi Banned

    Messages:
    184
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Hey Manish,
    Thanks for quick reply...reputation added ;)
    can u please tell me what does this shtml stand for???

     
    Ridhdhi, Jul 4, 2008 IP
  6. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #6
    Your welcome....:)
    SHTML stands for Server-parsed HTML. It is a file extension for Web pages that contain server side includes (SSIs)...
    I hope you got your answer...:)
     
    manish.chauhan, Jul 4, 2008 IP
  7. Ridhdhi

    Ridhdhi Banned

    Messages:
    184
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Ridhdhi, Jul 4, 2008 IP
  8. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #8
    manish.chauhan, Jul 4, 2008 IP
  9. Ridhdhi

    Ridhdhi Banned

    Messages:
    184
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Hi Manish,

    That domain name was just for example.I am having an other domain.
    Sorry for miss conception :(


     
    Ridhdhi, Jul 4, 2008 IP
  10. seoorganizers

    seoorganizers Peon

    Messages:
    111
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    all your pages are stopped by you, bots cannot crawls youuuuuu site, remove it firstt
     
    seoorganizers, Aug 13, 2008 IP
  11. khepri

    khepri Peon

    Messages:
    84
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #11
    What best robots.txt config for backlinks.com on wordpress weblog?
     
    khepri, Aug 13, 2008 IP