Hello, I was wondering if someone could take a look at my magazines (www.tunermaneuvermag.com) and tell what I can fix/do to improve SEO.
File in question: http://tunermaneuvermag.com/robots.txt This bit is telling all bots who pay attention to ignore your site: You may want to remove it.
heres my current robots.txt what do I need to change? Sitemap: http://tunermaneuvermag.com/sitemap.xml User-agent: * # disallow all files in these directories Disallow: /cgi-bin/ Disallow: /z/j/ Disallow: /z/c/ Disallow: /stats/ Disallow: /dh_ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /contact/ Disallow: /tag/ Disallow: /wp-content/b Disallow: /wp-content/p Disallow: /wp-content/themes/askapache/4 Disallow: /wp-content/themes/askapache/c Disallow: /wp-content/themes/askapache/d Disallow: /wp-content/themes/askapache/f Disallow: /wp-content/themes/askapache/h Disallow: /wp-content/themes/askapache/in Disallow: /wp-content/themes/askapache/p Disallow: /wp-content/themes/askapache/s Disallow: /trackback/ Disallow: /*?* Disallow: */trackback/ User-agent: Googlebot # disallow all files ending with these extensions Disallow: /*.php$ Disallow: /*.js$ Disallow: /*.inc$ Disallow: /*.css$ Disallow: /*.gz$ Disallow: /*.cgi$ Disallow: /*.wmv$ Disallow: /*.png$ Disallow: /*.gif$ Disallow: /*.jpg$ Disallow: /*.cgi$ Disallow: /*.xhtml$ Disallow: /*.php* Disallow: */trackback* Disallow: /*?* Disallow: /z/ Disallow: /wp-* Allow: /wp-content/uploads/ # allow google image bot to search all images User-agent: Googlebot-Image Allow: /* # allow adsense bot on entire site User-agent: Mediapartners-Google* Disallow: /*?* Allow: /z/ Allow: /about/ Allow: /contact/ Allow: /wp-content/ Allow: /tag/ Allow: /manual/* Allow: /docs/* Allow: /*.php$ Allow: /*.js$ Allow: /*.inc$ Allow: /*.css$ Allow: /*.gz$ Allow: /*.cgi$ Allow: /*.wmv$ Allow: /*.cgi$ Allow: /*.xhtml$ Allow: /*.php* Allow: /*.gif$ Allow: /*.jpg$ Allow: /*.png$ # disallow archiving site User-agent: ia_archiver Disallow: / # disable duggmirror User-agent: duggmirror Disallow: / PHP:
That's not your current robots.txt file. I linked to it. This is the content of your current robots.txt file: Where specifically are you getting what you quoted?
Yup, I see it's been changed. Have to ask though. Why do you have all of that? Is it needed? Do you really want to block archive.org? Why do you have directories disallowed that don't exist on your site? This is a good start: Put that and then we can build on that. What specifically do you need to disallow from the search engine spiders?
I have no idea even what should be in a robots.txt lol I copied it from a site that said that was the best for SEO when it comes to wordpress. So what do I need to put?
Hows this? I am not sure what I need to disallow I know I use cateorgies for my posts. User-agent: * Disallow: /cgi-bin Disallow: /wp-admin Disallow: /wp-includes Allow: /tag/ Disallow: /author Disallow: /wget/ Disallow: /httpd/ Disallow: /i/ Disallow: /f/ Disallow: /t/ Disallow: /c/ Disallow: /j/ Allow: /about/ Allow: /contact/ Allow: /wp-content/ User-agent: Mediapartners-Google Allow: / User-agent: Adsbot-Google Allow: / User-agent: Googlebot-Image Allow: / User-agent: Googlebot-Mobile Allow: / User-agent: ia_archiver-web.archive.org Disallow: / Sitemap: http://www.askapache.com/sitemap.xml PHP: