Is it ok to copy contents of robot.txt from another website?

HI,

I am not sure how to create a robot.txt for my new forum website. Is it ok to copy contents of robot.txt from another website?

Thanks

 

yeah if its a gud and reputed website u can do that but just check the platform, dunno about forums but for blog it can be like if that person is using wordpress and u r using joomla so that might not be a gud idea...just my thought,,,,

 

u can get the robots.txt file from your google webmaster itself. you dnt need to copy from any other site...

 

nope.. every website have different link structure, so it might not work properly.. i suggest to just use some online robots.txt creator: http://www.google.com/search?hl=&q=...3GGLL_enPH411PH411&ie=UTF-8&meta=cr=countryUS

 

For websites with multiple subdomains, each subdomain must have its own robots.txt file. If example.com had a robots.txt file but a.example.com did not, the rules that would apply for example.com would not apply to a.example.com

To create a robot.txt for your site,follow the given steps below

Step1. Open Notepad or equivalent (do NOT use a word document program as these add hidden characters that can do funny things).

Copy the contents of robots-txt.txt to your Notepad.

If you don’t want certain folders or pages to be indexed you can disallow them by following the above format:

* Disallow: /name of folder to disallow/
* or
* Disallow: /NameOfPageToDisallow.htm

Make sure you don’t leave a blank Disallow action:

* Disallow:

Including a blank Disallow: will prevent your entire site from being indexed. That would be bad!

Step2
Save your document as robots.txt.

Step3
Upload your robots.txt to your site’s root file (where you site’s index.php is located).

 

I would suggest using your robots.txt file quite intelligently. Don't copy someone else roboots.txt, create it by yourself as per your needs

 

But its has /member.php and /global.php all all other weird stuff. How do you make sure that you are entering a correct thing if you don't have much knowledge of what you are doing? I was thinking to copy the robot.txt from the site with similar platform. Like if we both use vB or IPBoard. Then after I get a lil familiar with how things work, I can edit or remove as necessary Right now my priority to have it up there. I did some research too, but doesn't really understand it fully. I hope you guys understands what I mean .

 

if both your site directory structures are same, then ofcourse you can copy the robots.txt. there is no harm in that.

 

If it's the same type of site and you sure the structure is the same (let's say they both are simple wordpress blogs), then yes, it surely is ok.