Social Buttons
  1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Urls block by webmaster...

Discussion in 'Google Sitemaps' started by azadbhaibhai, Oct 13, 2012.

0
  1. #1
    As you can see in image that 2377 sitehealth.jpg robot.jpg urls are blocked by robots.txt in webmastertool. So how can i fid those blocked url and unblock them . Plz help me regarding this ..

    My visitors dropped to 400 from 1000....my high impression page that is http://www.fansofapk.com/search/label/Android themes is bocked and its always showing in webmaster tools in health section...So i want every thing should get index and plz tell me which robots.txt file will be correct for my site...
    Here is my robots.txt http://www.fansofapk.com/robots.txt

    robot.jpg sitehealth.jpg
    SEMrush
     
    azadbhaibhai, Oct 13, 2012 IP
    SEMrush
  2. websitetools

    websitetools Well-Known Member

    Messages:
    1,510
    Likes Received:
    25
    Best Answers:
    4
    Trophy Points:
    170
    #2
    websitetools, Oct 14, 2012 IP
  3. Saif Rizvi

    Saif Rizvi Member

    Messages:
    143
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    46
    #3
    Hello

    Regarding robots.txt there is a simple code of just two lines -

    User agent:
    Disallow:

    There is no such any other code which relates to robots. Make sure there is no allow term in robots.txt file.
    User agent self allows all crawler to crawl your website and disallow term disallow the particular page, folder and particular webpage which you doesnt want to crawl .
    Suppose a particular webpage is under construction and you doesn't want to crawl that particular page then you put that page in disallow term.
     
    Saif Rizvi, Nov 1, 2012 IP