Ok, I'm paranoid about using robots because I'm afraid of blocking google from indexing my site. I know, very lame. I want to load my ads via iframe but injected using javascript. I then want to prevent google from indexing the iframe url as another page. Here is what I have: User-agent: * Disallow: /includes/ads.php Disallow: /includes/adLeft.php Disallow: /includes/adRight.php Code (markup): Here is what I use to call the ads: <script type="text/javascript" language="JavaScript"> <!-- document.write("<iframe src='/includes/ads.php' quality=high WIDTH=100% HEIGHT=435 frameborder=0 scrolling=no></iframe>"); //--> </script> Code (markup): Is this correct? Or am I doing something wrong here?
Sure, your code it's okay but bear in mind that many bad bots will not obey your robots.txt so it is a good practice put an .htaccess file inside your includes directory with this simple rule order deny,allow deny from all Code (markup): This will stop others from reading or linking to your includes directory but will not block yourself including those files in your iframes.
If Your iframe ads are placed in /includes directory then it would be better to block whole include directory than individual ads url.. you can use: User-agent: * Disallow: /includes/