Hello i read more about robots then i made robots text and i want to know your opinion in it User-agent: * # Disallow all files in these directories Disallow: /xmlrpc.php Disallow: /cgi-bin/ Disallow: /wp-trackback.php Disallow: /wp-settings.php Disallow: /wp-rss2.php Disallow: /wp-rss.php Disallow: /wp-register.php Disallow: /wp-rdf.php Disallow: /wp-pass.php Disallow: /wp-mail.php Disallow: /wp-login.php Disallow: /wp-load.php Disallow: /wp-links-opml.php Disallow: /wp-feed.php Disallow: /wp-cron.php Disallow: /wp-config-sample.php Disallow: /wp-config.php Disallow: /wp-commentsrss2.php Disallow: /wp-comments-post.php Disallow: /wp-blog-header.php Disallow: /wp-atom.php Disallow: /wp-app.php Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /themes/ Disallow: /wp-content/plugins/ Disallow: /wp-content/cache/ Disallow: /wp-content/themes/ Disallow: /wp-content/upgrade/ Disallow: /wp-content/woo_custom/ Disallow: /wp-content/woo_uploads/ Allow: /wp-content/uploads User-agent: Googlebot Disallow: /feed Disallow: /contact-us Disallow: /feed/$ Disallow: /trackback Disallow: /comments Disallow: /search Disallow: /tag Disallow: /author Disallow: /feed/ Disallow: /category/*/* disallow: */trackback disallow: */*/trackback disallow: */feed Disallow: /*?* Disallow: /*? Disallow: /z/ Disallow: /search/*/feed Disallow: /search/*/* Disallow: /*feed= Disallow: /?feed= Disallow: /*tag= Disallow: /*?tag= Disallow: /*m= Disallow: /*&cp= Disallow: /*.html$ Disallow: /*.doc$ Disallow: /*.flv$ Disallow: /*.swf$ Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.cgi$ Disallow: /*/feed/$ Disallow: /*/feed/rss/$ Disallow: /*/trackback/$ Disallow: /*/*/feed/$ Disallow: /*/*/feed/rss/$ Disallow: /*/*/trackback/$ Disallow: /*/*/*/feed/$ Disallow: /*/*/*/feed/rss/$ Disallow: /*/*/*/trackback/$ Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.gz$ Disallow: /*.cgi$ Disallow: /*.wmv$ Disallow: /*.png$ Disallow: /*.gif$ Disallow: /*.jpg$ Disallow: /*.cgi$ Disallow: /*.xhtml$ Allow: /wp-content/uploads Allow: /*?$ User-agent: Googlebot-Image Allow: / # digg mirror User-agent: duggmirror Disallow: / # allow adsense bot on entire site User-agent: Mediapartners-Google Disallow: /*.php$ Disallow: /*.html$ Disallow: /*.doc$ Disallow: /*.flv$ Disallow: /*.swf$ Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Allow: /index.php # MSN User-agent: msnbot Disallow: /feed Disallow: /contact-us Disallow: /feed/$ Disallow: /trackback Disallow: /comments Disallow: /search Disallow: /tag Disallow: /author Disallow: /feed/ Disallow: /category/*/* disallow: */trackback disallow: */*/trackback disallow: */feed Disallow: /*?* Disallow: /*? Disallow: /z/ Disallow: /search/*/feed Disallow: /search/*/* Disallow: /*feed= Disallow: /?feed= Disallow: /*tag= Disallow: /*?tag= Disallow: /*m= Disallow: /*&cp= Disallow: /*.html$ Disallow: /*.doc$ Disallow: /*.flv$ Disallow: /*.swf$ Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.cgi$ Disallow: /*/feed/$ Disallow: /*/feed/rss/$ Disallow: /*/trackback/$ Disallow: /*/*/feed/$ Disallow: /*/*/feed/rss/$ Disallow: /*/*/trackback/$ Disallow: /*/*/*/feed/$ Disallow: /*/*/*/feed/rss/$ Disallow: /*/*/*/trackback/$ Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.gz$ Disallow: /*.cgi$ Disallow: /*.wmv$ Disallow: /*.png$ Disallow: /*.gif$ Disallow: /*.jpg$ Disallow: /*.cgi$ Disallow: /*.xhtml$ Allow: /wp-content/uploads Allow: /*?$ # Yahoo! User-agent: Slurp Disallow: /feed Disallow: /contact-us Disallow: /feed/$ Disallow: /trackback Disallow: /comments Disallow: /search Disallow: /tag Disallow: /author Disallow: /feed/ Disallow: /category/*/* disallow: */trackback disallow: */*/trackback disallow: */feed Disallow: /*?* Disallow: /*? Disallow: /z/ Disallow: /search/*/feed Disallow: /search/*/* Disallow: /*feed= Disallow: /?feed= Disallow: /*tag= Disallow: /*?tag= Disallow: /*m= Disallow: /*&cp= Disallow: /*.html$ Disallow: /*.doc$ Disallow: /*.flv$ Disallow: /*.swf$ Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.cgi$ Disallow: /*/feed/$ Disallow: /*/feed/rss/$ Disallow: /*/trackback/$ Disallow: /*/*/feed/$ Disallow: /*/*/feed/rss/$ Disallow: /*/*/trackback/$ Disallow: /*/*/*/feed/$ Disallow: /*/*/*/feed/rss/$ Disallow: /*/*/*/trackback/$ Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.gz$ Disallow: /*.cgi$ Disallow: /*.wmv$ Disallow: /*.png$ Disallow: /*.gif$ Disallow: /*.jpg$ Disallow: /*.cgi$ Disallow: /*.xhtml$ Allow: /wp-content/uploads Allow: /*?$ # Ask User-agent: Teoma Disallow: /feed Disallow: /contact-us Disallow: /feed/$ Disallow: /trackback Disallow: /comments Disallow: /search Disallow: /tag Disallow: /author Disallow: /feed/ Disallow: /category/*/* disallow: */trackback disallow: */*/trackback disallow: */feed Disallow: /*?* Disallow: /*? Disallow: /z/ Disallow: /search/*/feed Disallow: /search/*/* Disallow: /*feed= Disallow: /?feed= Disallow: /*tag= Disallow: /*?tag= Disallow: /*m= Disallow: /*&cp= Disallow: /*.html$ Disallow: /*.doc$ Disallow: /*.flv$ Disallow: /*.swf$ Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.cgi$ Disallow: /*/feed/$ Disallow: /*/feed/rss/$ Disallow: /*/trackback/$ Disallow: /*/*/feed/$ Disallow: /*/*/feed/rss/$ Disallow: /*/*/trackback/$ Disallow: /*/*/*/feed/$ Disallow: /*/*/*/feed/rss/$ Disallow: /*/*/*/trackback/$ Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.gz$ Disallow: /*.cgi$ Disallow: /*.wmv$ Disallow: /*.png$ Disallow: /*.gif$ Disallow: /*.jpg$ Disallow: /*.cgi$ Disallow: /*.xhtml$ Allow: /wp-content/uploads Allow: /*?$ # disallow archiving site User-agent: ia_archiver Disallow: / Sitemap: http://www.sitename.com/sitemap.xml PHP:
######################## # Add By Sara Tancredi # ######################## User-agent: * # Disallow all files in these directories Disallow: /xmlrpc.php Disallow: /cgi-bin/ Disallow: /wp-trackback.php Disallow: /wp-settings.php Disallow: /wp-rss2.php Disallow: /wp-rss.php Disallow: /wp-register.php Disallow: /wp-rdf.php Disallow: /wp-pass.php Disallow: /wp-mail.php Disallow: /wp-login.php Disallow: /wp-load.php Disallow: /wp-links-opml.php Disallow: /wp-feed.php Disallow: /wp-cron.php Disallow: /wp-config-sample.php Disallow: /wp-config.php Disallow: /wp-commentsrss2.php Disallow: /wp-comments-post.php Disallow: /wp-blog-header.php Disallow: /wp-atom.php Disallow: /wp-app.php Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /themes/ Disallow: /wp-content/plugins/ Disallow: /wp-content/cache/ Disallow: /wp-content/themes/ Disallow: /wp-content/upgrade/ Disallow: /wp-content/woo_custom/ Disallow: /wp-content/woo_uploads/ Allow: /wp-content/uploads User-agent: * Disallow: /feed Disallow: /contact-us Disallow: /feed/$ Disallow: /trackback Disallow: /comments Disallow: /search Disallow: /tag Disallow: /author Disallow: /feed/ Disallow: /category/*/* disallow: */trackback disallow: */*/trackback disallow: */feed Disallow: /*?* Disallow: /*? Disallow: /z/ Disallow: /search/*/feed Disallow: /search/*/* Disallow: /*feed= Disallow: /?feed= Disallow: /*tag= Disallow: /*?tag= Disallow: /*m= Disallow: /*&cp= Disallow: /*.html$ Disallow: /*.doc$ Disallow: /*.flv$ Disallow: /*.swf$ Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.cgi$ Disallow: /*/feed/$ Disallow: /*/feed/rss/$ Disallow: /*/trackback/$ Disallow: /*/*/feed/$ Disallow: /*/*/feed/rss/$ Disallow: /*/*/trackback/$ Disallow: /*/*/*/feed/$ Disallow: /*/*/*/feed/rss/$ Disallow: /*/*/*/trackback/$ Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.gz$ Disallow: /*.cgi$ Disallow: /*.wmv$ Disallow: /*.png$ Disallow: /*.gif$ Disallow: /*.jpg$ Disallow: /*.cgi$ Disallow: /*.xhtml$ Allow: /wp-content/uploads Allow: /*?$ Sitemap: http://www.sitename.com/sitemap.xml PHP: These settings are sufficient There is no need to add anything new to them Thank you for this topic Sara Tancredi
I would not suggest anyone to just copy others robots.txt file and paste the same on his own blog. I would recommend you to study about robots.txt and then prepare it as per requirement of your blog.
Hello... I need a good robot.txt I have start a new Blog a few days ago. You people show me many robot.txt but no Domains where I can see traffic or PR Can you post some domains who use this robot.txt so I can see... if the site going up with the ranking And If you have a good ranking. Maybe I will use your Robot.txt and promote it in my Blog by your Names ^^ Ok... here is mine
robots is one of the best plugins in wordpress bcz i installed it in my blogs but i dont knw deep knw abt it .