Robots.txt index issue.

Anytime I see a new post idexed, it reads:

Title / URL>
A description for this result is not available because of this site's robots.txt

I dont get it, there is nothing in the robots.txt that
should be causing posts to appear like this in google...

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

Any ideas?

 

It would not be easy to get real answer without telling us the URL. Based on my limited experience, it could be caused by multiple (or even single) request blocked by 'robots.txt'.
To make it simple:

1. The site such as example.com is indexed properly but URL of 'example.com/post-one.html' appear blocked by 'robots.txt'
2. The URL 'example.com/post-one.html' has one or multiple request to any area which didn't allow Search Engine Bots.
   It appear blocked because the request is important to render the page properly. CSS, JS, or any important stuff ought to be accessible by SE Bots. In my case, a plugin make a call inside WP-Admin directory and thus make SE bots couldn't crawl and render the page properly.

To debug your problem, I suggest to do these things:

1. Go to Search Console (Google Webmaster Tools / GWT)
   Any problem related with Google Search Result usually available in GWT.
2. Tap on "Google Index" and Choose "Blocked Resources". If there are any blocked resources, try to click the URL and use the suggestion to fix it (It has many useful tips to solve the problems).
3. In GWT, tap on "Crawl" and select "robots.txt Tester"
   Sometimes Bots fetch wrong 'robots.txt' due to server errors or wrong configuration on the site.
   Error and Warning related with fetching the site are available in this page.
4. Last, use menu "Fetch as Google" to find out how Google Bots render the page.
   Enter the URL and click "Fetch and Render". any warning and errors related with rendering the site will be shown immediately. Errors related with 'robots.txt' will be displayed too.

After trying few step by steps above, I hope you could identify the problem and fix it immediately.

Arick

 

There are several urls under blocked resources:

/wp-admin/css
/wp-content/themes/
/wp-content/plugins
/wp-includes/css

There is nothing in robots.txt that would block any of these urls...
however, there could have been, I have changed robots.txt several
times since last month, but I cannot remember if anything in robots
(before) that was actually blocking main portions of the site....

I have fetched the main site, plus posts urls that appeared to
be affected; so i guess it is just wait and see at this point.

the last reported block of urls was on: 9/13/2016

 

If these conditions met:

1. Your 'robots.txt' (on the website) is similar with 'robots.txt' listed in Google Search Console (GSC / GWT) available in GWT (GSC) => "Crawl" => "robots.txt Tester" and,
2. No blocked resources on any Index-able URL

I think the problems would no longer exist once Google flushing cache on Search Result Page.