Problem mobile friendly test

Discussion in 'robots.txt' started by irfansyah, Jun 23, 2016.

0
  1. #1
    hello all citizens dp. I have a question that is precisely why my website when I checked in google mobile friendly test do not show the entirety of my website to see it because I block some sources robots txt. How can I fix this without making changes to the robots.txt. From several sources I read that now do not have to block as wp-admin and wp-include.
     
    irfansyah, Jun 23, 2016 IP
  2. sarahk

    sarahk iTamer Staff

    Messages:
    28,807
    Likes Received:
    4,534
    Best Answers:
    123
    Trophy Points:
    665
    #2
    It won't be robots.txt, when you view the same pages from your phone do you see the content?
     
    sarahk, Jun 23, 2016 IP
  3. irfansyah

    irfansyah Well-Known Member

    Messages:
    327
    Likes Received:
    14
    Best Answers:
    0
    Trophy Points:
    185
    #3
    irfansyah, Jun 23, 2016 IP
  4. sarahk

    sarahk iTamer Staff

    Messages:
    28,807
    Likes Received:
    4,534
    Best Answers:
    123
    Trophy Points:
    665
    #4
    I use the default robots.txt on my sites
    User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
    Code (markup):
    If you have a plugin that is messing with the file then you may need to review the settings or consider ditching the plugin altogether.

    It's logical that if you prevent non-human traffic to your site and then as a non-human tool to tell you what your site looks like that the non-human output might not be what you expect.

    Care to share your robots.txt file?
     
    sarahk, Jun 23, 2016 IP
  5. irfansyah

    irfansyah Well-Known Member

    Messages:
    327
    Likes Received:
    14
    Best Answers:
    0
    Trophy Points:
    185
    #5
    This my robot txt 
    
sitemap: http://desainavirss.com/sitemap.xml
User-agent: *
# disallow all files in these directories
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /cgi-bin/
Disallow: /wp-content/
Disallow: /archives/
Disallow: /*?*
Disallow: *?replytocom
Disallow: /author/
Disallow: /search/
Disallow: /comments/feed/
Disallow: */trackback/
Disallow: /wp-*
Disallow: /*?*
User-agent:Mediapartners-Google*
Allow:/
User-agent:Googlebot-Image
Allow:/wp-content/uploads/
User-agent:Adsbot-Google
Allow:/
User-agent:Googlebot-Mobile
Allow:/
    Code (markup):
     
    irfansyah, Jun 23, 2016 IP
  6. sarahk

    sarahk iTamer Staff

    Messages:
    28,807
    Likes Received:
    4,534
    Best Answers:
    123
    Trophy Points:
    665
    #6
    And here's your problem

    Disallow: /wp-content/

    What tool created that file?
    What options does it give you?
    I'd actually stop being paranoid and just go back to the default robots.txt and play it safe.
     
    sarahk, Jun 24, 2016 IP
  7. irfansyah

    irfansyah Well-Known Member

    Messages:
    327
    Likes Received:
    14
    Best Answers:
    0
    Trophy Points:
    185
    #7
    Please give me default for the best robot txt and please describe what it is can work?
     
    irfansyah, Jun 24, 2016 IP
  8. sarahk

    sarahk iTamer Staff

    Messages:
    28,807
    Likes Received:
    4,534
    Best Answers:
    123
    Trophy Points:
    665
    #8
    "best" is very subjective but I've given you the default for wordpress
    it basically shuts down access to the admin folder while still giving access to the ajax content.
     
    sarahk, Jun 24, 2016 IP
  9. Host Little

    Host Little Greenhorn

    Messages:
    8
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    21
    #9
    You are disallowing google from reading wp-content, thats all your images and media! you want them to crawl that.
     
    Host Little, Jul 14, 2016 IP