How can I use robots.txt?

I want to restrict certain files on my site,How can I do?

 

Create a plain text file (ASCII/ANSI) as follows:

User-agent: *
Disallow: filename.htm
Disallow: /directoryname/
Disallow: /directoryname/filename.htm

Code (markup):

The first line disallows (instruction to NOT spider) a single file.
The second line disallows an entire directory.
The third line disallows a single file in a directory.

Upload this file to the root directory on your domain.