Giv me an intro to robots.txt

I am new to this topic
can you please give a short description
on the use of robots.txt

 

Read this page: http://en.wikipedia.org/wiki/Robots.txt

 

How do we create this
simply make a file robots.txt?

 

yes use notepad to create one and upload it to your root folder of your website

 

I need google to crawl all content in my forum
And i dont have a robots.txt

 

For that, you can make a sitemap and submit to Google.
Read this page: http://en.wikipedia.org/wiki/Sitemaps
Also you can use a tool to create your sitemap automatically.

 

which tool?

 

Google provides "The Google Sitemap Generator" is a Python script (for advanced users).

Also you can search the web for other online tools like this: http://www.xml-sitemaps.com/

 

Since i am new to website programming
For me it was a great help .....
thanks ajsa52

 

great info

 

I suggest that you read this http://www.robotstxt.org/wc/robots.html

 

hmmmm. use this sitemap generator.

 

Before posting, you should check the question atleast one time. The question is about robots.txt