I need anyone expert who can create robots.txt file for myBB forum

Discussion in 'robots.txt' started by kartarsingh, Mar 17, 2012.

0
  1. #1
    Hello,

    I want an expert who can create mybb forum robots.txt file which allows only forums and posts to be displayed in search engines. disallowing other pages like memberlist.php etc.
    gtalk :- kartarsingh015[@]gmail[.]com

    thank you.
     
    kartarsingh, Mar 17, 2012 IP
  2. Albert_w

    Albert_w Peon

    Messages:
    9
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Robots.txt file has default setting below. This setting allows your website to crawl by all search engines.

    user-agent:*
    Disallow:

    If you want to restrict some pages not to crawl by search engines then put the directory name like below example.

    user-agent:*
    Disallow: /cgi-bin
    Disallow: /sign up

    These are simple settings of robots.txt file. thats all.
     
    Albert_w, Mar 19, 2012 IP
  3. tiffanywilliams12i2

    tiffanywilliams12i2 Peon

    Messages:
    164
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    you can always generate them with generators. or your host can generate them for you liek mine does
     
    tiffanywilliams12i2, Apr 10, 2012 IP
  4. Pepdeal

    Pepdeal Peon

    Messages:
    149
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Check it.Note depending on your domain root,change "forum"


    User-Agent: * Disallow: /forum/captcha.php Disallow: /forum/editpost.php Disallow: /forum/misc.php Disallow: /forum/modcp.php Disallow: /forum/moderation.php Disallow: /forum/newreply.php Disallow: /forum/newthread.php Disallow: /forum/online.php Disallow: /forum/printthread.php Disallow: /forum/private.php Disallow: /forum/ratethread.php Disallow: /forum/report.php Disallow: /forum/reputation.php Disallow: /forum/search.php Disallow: /forum/sendthread.php Disallow: /forum/task.php Disallow: /forum/usercp.php Disallow: /forum/usercp2.php Disallow: /forum/calendar.php Disallow: /forum/*action=emailuser* Disallow: /forum/*action=nextnewest* Disallow: /forum/*action=nextoldest* Disallow: /forum/*year=* Disallow: /forum/*action=weekview* Disallow: /forum/*action=nextnewest* Disallow: /forum/*action=nextoldest* Disallow: /forum/*sort=* Disallow: /forum/*order=* Disallow: /forum/*mode=* Disallow: /forum/*datecut=* Disallow: /forum/fbcore/fbconnect.png Disallow: /forum/member.php?action=login Disallow: /forum/member.php?action=register Allow: /
     
    Pepdeal, Jul 28, 2012 IP