1020 links Restricted by robots.txt and just 9 index in Google webmaster why ?

Discussion in 'Google' started by fxlock, Feb 2, 2011.

  1. #1
    1020 links Restricted by robots.txt and just 9 index in Google webmaster why ?

    this really not good , what should i do ?

    url:

    Forex News
     
    fxlock, Feb 2, 2011 IP
  2. imperialDirectory

    imperialDirectory Peon

    Messages:
    395
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #2
    you have 1410 pages indexed by Google

    Input in Google's search box:
    
    site:http://www.forexsq.com/
    
    Code (markup):
    Edited:
    Also, here's your robots.txt file
    
    User-agent: *
    Crawl-delay: 5
    Disallow: 
    Disallow: /3rdparty/
    Disallow: /antispam.txt
    Disallow: /admin/
    Disallow: /admin/admin_index.php
    Disallow: /backup/
    Disallow: /cache/templates_c/
    Disallow: /cache/admin_c/
    Disallow: /install/
    Disallow: /internal/
    Disallow: /languages/
    Disallow: /libs/
    Disallow: /LICENSE.txt
    Disallow: /modules/
    Disallow: /plugins/
    Disallow: /readme.html
    Disallow: /search.php
    Disallow: /search/
    Disallow: /searchurl/
    Disallow: /spamlog.log
    Disallow: /tag/
    Disallow: /templates/
    Disallow: /user/view/shaken/
    Disallow: /user/view/history/
    Disallow: /user/view/published/
    Disallow: /user/view/commented/
    Disallow: /user/view/voted/
    Disallow: /recent/
    Disallow: /yesterday/
    Disallow: /today/
    Disallow: /week/
    Disallow: /month/
    Disallow: /year/
    Disallow: /alltime/
    Disallow: /upcoming/recent/
    Disallow: /upcoming/yesterday/
    Disallow: /upcoming/today/
    Disallow: /upcoming/week/
    Disallow: /upcoming/month/
    Disallow: /upcoming/year/
    Disallow: /upcoming/alltime/
    
    Code (markup):
    If you think any of the subfolder suppose to be allowed by Googlebot, remove the line in Robots.txt in your root folder
     
    imperialDirectory, Feb 2, 2011 IP
  3. fxlock

    fxlock Well-Known Member

    Messages:
    777
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    138
    #3
    thank you for advice , but i know that more than 1400 page index , you mean if i remove for example this line from robot.txt : Disallow: /upcoming/week/ the restricted links reduce ?

    i also have another question that once a page index by google means its ok with google ? i mean google do not index copy page ? or it index also copy page unless we use NoIndex code
     
    fxlock, Feb 3, 2011 IP