Is this correct? User-agent: scooter Disallow: / User-agent: lycos Disallow: / User-agent: slurp Disallow: / User-agent: googlebot Disallow: /contact.htm Disallow: /terms.htm Disallow: /print.php This is what I WANT it to do: User-agent: scooter Disallow: / #DISALLOW ALTAVISTA FROM THE WHOLE SITE User-agent: lycos Disallow: / #DISALLOW LYCOS FROM THE WHOLE SITE User-agent: slurp Disallow: / #DISALLOW YAHOO FROM THE WHOLE SITE User-agent: googlebot Disallow: /contact.htm #DISALLOW GOOGLE FROM WWW.MYSITE.COM/CONTACT.HTM Disallow: /terms.htm #DISALLOW GOOGLE FROM WWW.MYSITE.COM/TERMS.HTM Disallow: /print.php #DISALLOW GOOGLE FROM WWW.MYSITE.COM/PRINT.PHP Is that what it does?
Its fine but you can compact it to: User-agent: scooter User-agent: lycos User-agent: slurp Disallow: / User-agent: googlebot Disallow: /contact.htm Disallow: /terms.htm Disallow: /print.php