1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

How do I unblock my robots.txt

Discussion in 'robots.txt' started by scott2342, Nov 24, 2014.

  1. #1
    Hey,
    SEMrush
    My site is www.slscomputing.co.uk and in my webmaster tools it says that my robot.txt is being blocked on the homepage.

    I've tried to fix it via yoast plugin and looked online for different methods but it still shows the error and also shows no meta in search results.

    What do i do?

    Thanks
     
    scott2342, Nov 24, 2014 IP
    SEMrush
  2. Sagar More

    Sagar More Greenhorn

    Messages:
    1
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    21
    #2
    Your robots file is good but your sitemap.xml file is missing, you add that and also add description to your website page. where as the issue what you are getting for that check if you have made any changes in the Google Webmaster and also try using different plugin.

    If you want I can help you out with error. You can PM me for the same.
     
    Sagar More, Jan 16, 2015 IP
  3. siddhu156

    siddhu156 Greenhorn

    Messages:
    7
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    21
    #3
    The first image shows no problem with robots.txt .
    The second image shows there was a timeout problem looking up your domain's DNS. BUt you don't show the detail which would have said on what date that problem was.
    The third image shows that on Oct 15 (6 days ago) Googlebot could not access the site - maybe that was due to the DNS lookup timeout.
    It doesn't mean the same error persists today.
    Your robots.txt file is not quite right.
    Get rid of the attracta sitemap and all the comments related to it.
    Your robots.txt should be this:
    User-agent:*Disallow:/feed
    Disallow: /*/feed
    Disallow: /xmlrpcDisallow:/wp-
    Disallow: /?p=Disallow:/*trackback
    Allow: /wp-content/uploads/Add a robots "noindex,follow" meta tag to tag, category, archive, author pages to prevent content duplication. The Yoast plugin you are using allows that.


    Get rid of Adsense ads or move them completely out of direct line of view (bottom of the page or far right).


    Fix the title of your homepage to be more useful. Right now it only echoes the domain url.

    Coming to the content: is your content original and unique? Extremely doubtful.
     
    siddhu156, May 27, 2015 IP
  4. meethi jain

    meethi jain Active Member

    Messages:
    169
    Likes Received:
    0
    Best Answers:
    1
    Trophy Points:
    56
    #4
    try to make a robots.txt file with online availbal tool, may be you are doing mistake some where.
     
    meethi jain, Jun 3, 2015 IP