My robot.txt file

My robot.txt file says this:

User-agent: *
Disallow:


does this harm my seo efforts in anyway?

 

Well... No. It won't harm.

NULL value of DISALLOW: is guiding the crawler to crawl the entire site, and not blocking anything. But if you do use CMS or any Login/Admin section, then you should put the in DISALLOW: list.

 

I tried creating one on Google and it gave me:

User-agent: *
Allow:

is that bad?

my current one can be found under my domain wegotthiscovered. com but it uses disallow.

 

A blank Disallow means that you dont want search engines to block indexing any part of your site. Does not harm your seo efforts in any way.

@Matt Jo: it is not bad if you dont want to restrict search engines to index any part of your website.

 

so which should I use

User-agent: *
Allow:

or

User-agent: *
Disallow:

 

Hey Matt both are same. You can use either.

btw, I see that your site is an authority site. Your categories are well indexed in Google. Are you simply trying to market you site here at DP forum ...

 

they're the same? but one says allow and one says disallow?? lol

and I have no idea what an authority site means but no I'm not trying to market my site, I'm trying to learn more about SEO

 

Correct, one says allow and the other disallow. But you're not mentioning any directory or path for that allow/disallow. See if you're using wordpress, and if you dont want search engines to index your "wp-admin" folder then you should write

Well an authority site is a site for which google search will display the list of categories. You can read more at the google webmaster blog.

 

should I tell it to disallow my wp-admin?

 

yes you should. Please go ahead and copy my robots.txt and modify it to suit your needs.

 

I already have it set to noindex in my Wordpress SEO settings, do I need to do it again? Although I find it odd that in Wordpress SEO it says it's no index yet it doesn't mention it in the robots.txt

 

I dont know what wordpress seo setting you're talking about.

 

it's a plugin by Yoast

 

See Joost de Valk of yoast is a renowned SEO expert. I don't think his plugins will be buggy. Or if you still think that it is a bug, then go ahead and contact him.

 

No, it won't harm unless you don't want one of your page to be crawled.

 

I would add one more line in there so it reads:

User-agent: *
Sitemap: [Full url of sitemap.xml file]
Allow:

Search engine crawlers other than Google often look here for your sitemap location so it keeps you covered (I'm assuming you have a sitemap.xml file - if not, get one!)