1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

I have a doubt in robots.txt - need help

Discussion in 'robots.txt' started by mikstroa, Aug 30, 2010.

  1. #1
    I have uploaded robot.txt, which conatained

    User-agent: *
    Disallow: /


    so after i did , for the past 2 days , i have a problem in my website adsense ads are not displaying properly.

    sometimes it shows sometimes not.

    when i checked in google webmaster tools.
    it says that my robot blocked robot visit on my page.

    Now i removed my robots.txt

    Now i want to know
    will the site show adsense ads normally.
    will the google index my web pages.

    Thanks
     
    mikstroa, Aug 30, 2010 IP
  2. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #2
    Since you have removed the robots.txt code, now your webpage would be indexed and your adsense ads will also be visible. The reason your ads were not visible was the fact that you had blocked the all crawlers to access the website and it also disabled the ad sense crawlers to vist your webiste, hence your ads were stopped being visible there. In case you want not to index your website and wants that ad sense ads continue to be visible then use following code:

    User-agent: *
    Disallow: /

    User-agent: Mediapartners-Google*
    Disallow:
     
    manish.chauhan, Aug 30, 2010 IP
  3. DoDo Me

    DoDo Me Peon

    Messages:
    2,257
    Likes Received:
    27
    Best Answers:
    0
    Trophy Points:
    0
    #3
    User-agent: *
    Disallow: /

    User-agent: Mediapartners-Google
    Disallow:

    (no star after google)
     
    DoDo Me, Aug 30, 2010 IP
  4. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #4
    * in the code has been added because there are several media bots that Google uses to crawl the webpages and put ads on the web pages. So * will cover all those bots and enable them to access the webpages. However, I believe that without using * would do the same job. But if you use the *, it will give you clear picture of the process.
     
    manish.chauhan, Aug 30, 2010 IP
  5. mikstroa

    mikstroa Active Member

    Messages:
    110
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    71
    #5
    Thank you very much.

    What you said is 100% correct.

    Now adsense ads starts appears on my web pages. Google crawls my site now. Now i am waiting for the google index.
     
    mikstroa, Aug 30, 2010 IP