Need robots.txt help - pages not getting indexed! $30 if you figure it out!

Discussion in 'Programming' started by kianah, Dec 22, 2008.

  1. #1
    Hi, I will paypal you $30 if you can tell me why my pages are not getting indexed and it proves to be successful. If you need more info I can PM you the URL.

    Basically I am trying to get pages with the ALLOW indexed, but so far i can only get the homepage. I think it has to do with the fact that the allowed page is also the default name of the folders? Thanks in advance!

    User-agent: Googlebot
    # Crawl-delay: 45

    Disallow: /about-us/
    Disallow: /accounts-receivables/
    Allow: /accounts-receivables/default.asp/
    Disallow: /adobeconnect/
    Disallow: /amc/
    Disallow: /api/
    Disallow: /appliance/
    Disallow: /applications/
    Disallow: /avon/
    Disallow: /bizcard/
    Disallow: /chairmansclub/
    Disallow: /channels/
    Disallow: /conferencing/
    Disallow: /conferencing-web-audio/
    Allow: /conferencing-web-audio/default.asp/
    Disallow: /contact-us/
    Disallow: /cos/
    Disallow: /css/
    Disallow: /customers/
    Disallow: /customer-service/
    Disallow: /cybersource/
    Disallow: /demos/
    Disallow: /desktop-fax/
    Disallow: /desktop-fax-online/
    Allow: /desktop-fax-online/default.asp/
    Disallow: /docs/
    Disallow: /document-delivery/
    Disallow: /document-delivery-paperless-statements/
    Allow: /document-delivery-paperless-statements/default.asp/
    Disallow: /downloads/
    Disallow: /dst/
    Disallow: /ecard2007/
    Disallow: /emarketing/
    Disallow: /emarketing-direct/
    Allow: /emarketing-direct/default.asp/
    Disallow: /error/
    Disallow: /fax2mail/
    Disallow: /footers/
    Disallow: /functions/
    Disallow: /headers/
    Disallow: /home/
    Disallow: /home-css/
    Disallow: /home-images/
    Disallow: /home-includes/
    Disallow: /hostcontrols/
    Disallow: /ilwc/
    Disallow: /ilwcs-demo/
    Disallow: /images/
    Disallow: /img/
    Disallow: /inc/
    Disallow: /includes/
    Disallow: /industry/
    Disallow: /investors/
    Disallow: /js/
    Disallow: /knowledge-center/
    Disallow: /kw/
    Disallow: /legal-notices/
    Disallow: /marketingautomation/
    Disallow: /marketing-automation/
    Disallow: /monthlyplans/
    Disallow: /my-account/
    Disallow: /navigation/
    Disallow: /notifications/
    Disallow: /notifications-reminders-alerts/
    Allow: /notifications-reminders-alerts/default.asp/
    Disallow: /partnerapps/
    Disallow: /partner-programs/
    Disallow: /platinumclub/
    Disallow: /premiereconnect/
    Disallow: /press-room/
    Disallow: /privacy-policy/
    Disallow: /ProductFeedback/
    Disallow: /prudential-ct-realty/
    Disallow: /reach/
    Disallow: /referral-ap/
    Disallow: /sametime/
    Disallow: /sap/
    Disallow: /scripts/
    Disallow: /sitedocs/
    Disallow: /site-map/
    Disallow: /smartview/
    Disallow: /solution/
    Disallow: /swf/
    Disallow: /template/
    Disallow: /terms-conditions/
    Disallow: /trials/
    Disallow: /try-it-now/
    Disallow: /us/
    Disallow: /us-invoice/
    Disallow: /xml/
     
    kianah, Dec 22, 2008 IP
  2. new

    new Peon

    Messages:
    1,433
    Likes Received:
    45
    Best Answers:
    0
    Trophy Points:
    0
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #2
    hey remove these trailing slashes from default.asp/ and you should be set
     
    new, Dec 22, 2008 IP
  3. rochow

    rochow Notable Member

    Messages:
    3,991
    Likes Received:
    245
    Best Answers:
    0
    Trophy Points:
    240
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #3
    You will be more successfully only telling it what not to index.

    Don't worry about using "allow", search engines will find all your pages by following your site.
     
    rochow, Dec 22, 2008 IP
  4. AbhishekS

    AbhishekS Peon

    Messages:
    309
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #4
    Have not recieved payment for this job so far.
     
    AbhishekS, Jan 7, 2009 IP
  5. cyblog

    cyblog Active Member

    Messages:
    172
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    As Seller:
    100% - 0
    As Buyer:
    100% - 0
    #5
    you need strong do follow backlink for that pages, from high pr and updated dayli site or try link that pages from your homepage :)
     
    cyblog, Jan 29, 2009 IP