Hello there, Can any one tell why Google and Yahoo indexes only my first article of my blog? It seems crawls only the title of the main domain and the title of the first article!? The each title is optimized. The other articles just don't appear in the Google's index! May be my robots.txt file is built this way? Can any one check my robots,txt file please: User-agent: * Disallow: /cgi-bin Disallow: /wp-admin Disallow: /wp-includes Disallow: /wp-content/plugins Disallow: /wp-content/cache Disallow: /wp-content/themes Disallow: /trackback Disallow: /page Disallow: /page/* Disallow: */page/* Disallow: /category/*/* Disallow: */trackback Disallow: */*/trackback Disallow: /*?* Disallow: /*? # Google Image User-agent: Googlebot-Image Disallow: Allow: /* # Google AdSense User-agent: Mediapartners-Google* Disallow: Allow: /* # Internet Archiver Wayback Machine User-agent: ia_archiver Disallow: / # digg mirror User-agent: duggmirror Disallow: / ---------------------------------------------------------- I just don't fully understand this file. Very much appreciate your help! Vad
That's your problem source right there. They won't index anything else until you remove those 4 lines.
I was told that those lines are to prevent duplicate content. If I remove those lines the will crawl the duplicate content, isn't it??? My blog: www(dot)plentyofluck(dot)com Check it out please how it is built, you'll see that all articles are on the main page Vad
I found out that those lines are not a problem, the problem was that I didn't have any link to those articles and Google just don't considered it important to crawl!!! Vad