1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Robots.txt help

Discussion in 'Search Engine Optimization' started by anandsapple, Aug 21, 2013.

0
  1. #1
    Block Google on all pages except homepage via robots.txt

    ....
    User-agent: googlebot
    Disallow: /*

    Will above mentioned code work?
     
    anandsapple, Aug 21, 2013 IP
  2. Helge Sverre

    Helge Sverre Prominent Member Affiliate Manager

    Messages:
    840
    Likes Received:
    99
    Best Answers:
    2
    Trophy Points:
    305
    Digital Goods:
    2
    #2
    Helge Sverre, Aug 22, 2013 IP
  3. anandsapple

    anandsapple Greenhorn

    Messages:
    64
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #3
    That will never work because i want to block only Google not all the robots.
     
    anandsapple, Aug 22, 2013 IP
  4. Helge Sverre

    Helge Sverre Prominent Member Affiliate Manager

    Messages:
    840
    Likes Received:
    99
    Best Answers:
    2
    Trophy Points:
    305
    Digital Goods:
    2
    #4
    Do 
    User-Agent: googlebot
Allow: /$
Disallow: /
    Code (markup):
    then...
     
    Helge Sverre, Aug 22, 2013 IP
  5. anandsapple

    anandsapple Greenhorn

    Messages:
    64
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #5
    Allow: /$

    what does middle line denotes? I think $ denotes dynamic pages ...
     
    anandsapple, Aug 22, 2013 IP
  6. Helge Sverre

    Helge Sverre Prominent Member Affiliate Manager

    Messages:
    840
    Likes Received:
    99
    Best Answers:
    2
    Trophy Points:
    305
    Digital Goods:
    2
    #6
    Helge Sverre, Aug 22, 2013 IP
  7. Roger Pelt

    Roger Pelt Active Member

    Messages:
    121
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    53
    #7
    Yes, I do agree with Helge,

    $ designates the end of the URL so when you follow below code then you may block your entire website for Google bot except your home page.

    User-agent: googlebot
Allow: /$
Disallow: /*
    Code (markup):
    If you have configured your website in Google webmaster then follow this step to test your robots.txt file live

    • Sign in your Google Webmaster account
    • Click on Crawl from the left side
    • Select Blocked URLs
    • Here you can check your robots.txt file live
    I have tasted above code from Google webmaster tool and it's work
     
    Roger Pelt, Aug 22, 2013 IP
    Helge Sverre likes this.
  8. ManiQuadraincorp

    ManiQuadraincorp Active Member

    Messages:
    119
    Likes Received:
    0
    Best Answers:
    1
    Trophy Points:
    56
    #8
    User-agent: * Disallow: /
    *allow :/ method only follow this one
     
    ManiQuadraincorp, Sep 16, 2013 IP