1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Robots.txt is blocking my labels

Discussion in 'robots.txt' started by Mrinal_Roy, Feb 5, 2015.

  1. #1
    In my adsense account in "revenue optimization" i have crawl errors then when i click "fix crawl errors" then this..

    Blocked Urls Error
    http://www.rechargeoverload.in/search/label Robot Denied

    My robots.txt:

    User-agent: Mediapartners-Google

    User-agent: *
    Disallow: /search
    Allow: /

    Sitemap: http://www.rechargeoverload.in/atom.xml?redirect=false&start-index=1&max-results=500
    Sitemap: http://www.rechargeoverload.in/atom.xml?redirect=false&start-index=501&max-results=500
    Mrinal_Roy, Feb 5, 2015 IP
  2. vgopalsujith

    vgopalsujith Active Member

    Likes Received:
    Best Answers:
    Trophy Points:
    Go to Blogger Settings > Search preferences > Custom robots.txt > edit > Remove "Disallow: /search" > Save.
    This should help the crawlers crawl complete site including labels. Hope this helps...
    vgopalsujith, Feb 5, 2015 IP