1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.
  2. Better Analytics for WordPress Get It Free

URLs restricted by robots.txt for blogger

Discussion in 'Google Sitemaps' started by fbg, Aug 8, 2006.

  1. #1
    My blog is at blogger.

    It is showing

    URLs restricted by robots.txt

    and in the list of pages that is blocked it shows the main site URL

    Sitemaps are verified. Google stopped indexing my blog.

    Please suggest.
     
    fbg, Aug 8, 2006 IP
  2. h4nh4n

    h4nh4n Peon

    Messages:
    932
    Likes Received:
    16
    Best Answers:
    0
    Trophy Points:
    0
    #2
    h4nh4n, Aug 8, 2006 IP
  3. fbg

    fbg Peon

    Messages:
    9
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    can anyone tell me when was my blog crawled lastly by google?

    my site adres is itly.blogspot.com

    is google crawling my blog?
     
    fbg, Aug 9, 2006 IP
  4. Jean-Luc

    Jean-Luc Peon

    Messages:
    601
    Likes Received:
    30
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Jean-Luc, Aug 9, 2006 IP
  5. fbg

    fbg Peon

    Messages:
    9
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    when i visit

    itly.blogspot.com/robots.txt

    it shows

    Does that mean robots will not visit my blog? How to make my blogs accessable to robots?
     
    fbg, Aug 9, 2006 IP
  6. Jean-Luc

    Jean-Luc Peon

    Messages:
    601
    Likes Received:
    30
    Best Answers:
    0
    Trophy Points:
    0
    #6
    It was not there yesterday. You just created it ?

    Anyway, as it is now, it disallows all robots. No "polite" robot will visit you blog.

    To make your blog accessible to robots, just remove the robots.txt file or replace it by an empty file.

    Jean-Luc
     
    Jean-Luc, Aug 9, 2006 IP
  7. fbg

    fbg Peon

    Messages:
    9
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I added this in the source code of my blog

    <meta name="robots" content="FOLLOW,INDEX" />

    How to remove / replace the robots.txt. Please note that my blog is at blogspot so I can access only the source code of my blog.
     
    fbg, Aug 9, 2006 IP
  8. abanalex

    abanalex Peon

    Messages:
    41
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Hi,

    I Have Same Problem Here

    I was receiving good rankings in google for some keywords but I recently changed my xml template to another new xml template of blogger and ever since I am not being indexed at the google at all, except for some pages all other and New pages are not showing in google at all, even if typed that url or important keywords from that page. Can someone please tell me what wrong I could have done which has resulted in this disaster for my blog. I need someone’s help in this regard urgently. By the way, i use blogger (blogspot) sub domain as my blog site. Thank you.

    These lines is appearing there in webmaster tool robot.text file.

    User-agent: Mediapartners-Google
    Disallow:

    User-agent: *
    Disallow: /search

    Sitemap: http://domain name/feeds/posts/default?orderby=updated



    How can i set this problem. ???? plz help me. how can i get my position back in google search engine.

    How can i Solve This Problem . Plz Share Complete Method To Resolve This Problem
     
    abanalex, Aug 28, 2011 IP
  9. wordpressfanboy

    wordpressfanboy Peon

    Messages:
    63
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #9
    If you are using labels for you posts, Blogger by default will restrict those label links. http.../search/label/Out of Place

    This helps eliminate duplicate content. Unless if there are other links aside from label URLs appearing on the uncrawled list, your blog should be fine. Sometimes it just takes quite a while before Google re-indexes your new content.

    Happy blogging! :D
     
    wordpressfanboy, Nov 12, 2011 IP
  10. jforjaxi

    jforjaxi Peon

    Messages:
    2
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    Please tell me why my google webmasters tool is saying me that

    "URL restricted by robots.txt"

    hundreds of pages of my site are having such problems according to this google webmaster tool.
    please help me out in solving it.

    and if possible then please send me email on

    thanks a lot..
     
    jforjaxi, Mar 2, 2012 IP
  11. jforjaxi

    jforjaxi Peon

    Messages:
    2
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #11
    please tell me if your problem of

    URL restricted by robots.txt

    is solved then how can i too solve this problem
     
    jforjaxi, Mar 2, 2012 IP
  12. same2cool

    same2cool Greenhorn

    Messages:
    34
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #12
    for blogger dont use robots.txt file it is useless for it. you need to use meta robots tag for it, it has amazing execution for it.
     
    same2cool, Mar 6, 2012 IP
  13. pkpaul33

    pkpaul33 Peon

    Messages:
    5
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #13
    I have a same problem too.
     
    pkpaul33, Jun 30, 2012 IP