Dear Friends, I am having a forum website and using SMF software. Recently, I came to know that each reply of each thread is indexed in google seperatly which may be considered as a spamming by google. Also, I read that if you will create a robot.txt file for your forum then google will update the new response of each thread under one search result and that will not be considered as spamming. Please tell me and if possible , give me the complete code for robot.txt file that will help me to solve the above problem. My Forum Website is : http://forum.WeDost.Com Thank You.
Thank you for this. One more question, How to stop same thread response being displayed twice in google search engine? Thanks.
thread /google/how-to-get-indexed-in-google-search-engine-quickly/ replys /google/how-to-get-indexed-in-google-search-engine-quickly/msg3/#msg3 /google/how-to-get-indexed-in-google-search-engine-quickly/msg4/#msg4 try add this to robots.txt? Disallow: /*/msg or u can set rel=nofollow to reply link?
Thank you .. I have created my robot.txt file n upload to my site. have a look and tell me If I need any addition or removal? http://forum.wedost.com/robots.txt Thank you.
A little change: User-agent: * Disallow: /index.php?action=search Disallow: /index.php?action=calendar Disallow: /index.php?action=login Disallow: /index.php?action=register Disallow: /index.php?action=profile Disallow: /index.php?action=stats Disallow: /index.php?action=arcade Disallow: /index.php?action=printpage Disallow: /index.php?PHPSESSID= Disallow: /index.php?*rss Disallow: /index.php?*wap Disallow: /index.php?*wap2 Disallow: /index.php?*imode Disallow: /index.php?action=search2 Disallow: /index.php?action=pm Disallow: /index.php?action=activate Disallow: /index.php?action=reminder Disallow: /*/?action=unread Disallow: /*/?action=sendtopic Disallow: /*/msg Disallow: /Themes/ Disallow: /Sources/ Disallow: /Smileys/ Disallow: /Packages/ Disallow: /header/ Disallow: /images/ Disallow: /avatars/ Disallow: /attachments/ Disallow: /gallery/*jpeg Disallow: /?action=printpage Disallow: /?prev_next=prev Disallow: /?prev_next=next Disallow: /*sort Disallow: /?action=unread Code (markup):
What is the difference between this and my robotx.txt . I think you have just removed * at the last. My question: Is there any effect of * at the end? Thanks for your help.
if you want hide all jpeg images from image folder then you right Disallow: /image/*jpeg so google not index any of ur images from that folder