Problems with indexing

Discussion in 'Search Engine Optimization' started by vad111, Nov 1, 2007.

  1. #1
    Hello there,
    Can any one tell why Google and Yahoo indexes only my first article of my blog?

    It seems crawls only the title of the main domain and the title of the first article!? The each title is optimized.
    The other articles just don't appear in the Google's index!
    May be my robots.txt file is built this way?
    Can any one check my robots,txt file please:

    User-agent: *
    Disallow: /cgi-bin
    Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes
    Disallow: /trackback
    Disallow: /page
    Disallow: /page/*
    Disallow: */page/*
    Disallow: /category/*/*
    Disallow: */trackback
    Disallow: */*/trackback
    Disallow: /*?*
    Disallow: /*?


    # Google Image
    User-agent: Googlebot-Image
    Disallow:
    Allow: /*


    # Google AdSense
    User-agent: Mediapartners-Google*
    Disallow:
    Allow: /*


    # Internet Archiver Wayback Machine
    User-agent: ia_archiver
    Disallow: /


    # digg mirror
    User-agent: duggmirror
    Disallow: /
    ----------------------------------------------------------

    I just don't fully understand this file.

    Very much appreciate your help!

    Vad
     
    vad111, Nov 1, 2007 IP
  2. seoz87

    seoz87 Peon

    Messages:
    262
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Remove this

    Disallow: /page
    Disallow: /category/*/*
    Disallow: /page/*
    Disallow: */page/*
     
    seoz87, Nov 1, 2007 IP
  3. ForgottenCreature

    ForgottenCreature Notable Member

    Messages:
    7,473
    Likes Received:
    173
    Best Answers:
    0
    Trophy Points:
    260
    #3
    That's your problem source right there. They won't index anything else until you remove those 4 lines.
     
    ForgottenCreature, Nov 1, 2007 IP
  4. vad111

    vad111 Active Member

    Messages:
    140
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #4
    I was told that those lines are to prevent duplicate content.
    If I remove those lines the will crawl the duplicate content, isn't it???
    My blog: www(dot)plentyofluck(dot)com
    Check it out please how it is built, you'll see that all articles are on the main page
    Vad
     
    vad111, Nov 1, 2007 IP
  5. silveraden

    silveraden Banned

    Messages:
    3,833
    Likes Received:
    109
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I have the same problem..where can I find these details?

    Thanks
     
    silveraden, Nov 1, 2007 IP
  6. vad111

    vad111 Active Member

    Messages:
    140
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #6
    I found out that those lines are not a problem, the problem was that I didn't have any link to those articles and Google just don't considered it important to crawl!!!
    Vad
     
    vad111, Nov 3, 2007 IP