1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Google crawling but not adding to its index. Why?

Discussion in 'SEO' started by fluid, Aug 3, 2006.

  1. #1
    Since my bad experience with Google sitemaps where all my indexed pages were dropped, GoogleBot has been behaving weird on my site. When i removed the sitemap, the indexed pages went up to 67. I've been adding lots of pages everyday (more than 500 now after 3 months) and from my logs i can see that GoogleBot has been crawling all the newly added pages. But the problem is that it is not adding them to its index! I dont understand why? Can anybody shed some light please?
    SEMrush
     
    fluid, Aug 3, 2006 IP
    khasmoth likes this.
    SEMrush
  2. Bondat

    Bondat Peon

    Messages:
    2,398
    Likes Received:
    217
    Best Answers:
    0
    Trophy Points:
    0
    #2
    What is your site's URL?
     
    Bondat, Aug 3, 2006 IP
  3. PinoyIto

    PinoyIto Notable Member

    Messages:
    5,863
    Likes Received:
    170
    Best Answers:
    0
    Trophy Points:
    260
    #3
    Same here... I have 4 months site where google bot crawling every day but don't even get a single page listed in google.
     
    PinoyIto, Aug 3, 2006 IP
    khasmoth likes this.
  4. terryuk

    terryuk Notable Member

    Messages:
    3,963
    Likes Received:
    320
    Best Answers:
    0
    Trophy Points:
    255
    #4
    Umm wierd, i managed to get my new site indexed within a couple of days... you done the normal, and submitted to directories etc? I can't think why it wouldn't index your site unless it doesn't like you :p
     
    terryuk, Aug 3, 2006 IP
    khasmoth likes this.
  5. MarkB

    MarkB Peon

    Messages:
    33
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Does it have any no-index META tags?
     
    MarkB, Aug 3, 2006 IP
    khasmoth likes this.
  6. fluid

    fluid Active Member

    Messages:
    679
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    70
    #6
    There's just one no-index, no-follow meta tag on a password protected area of the site. The site URL is http://www.fancyacar.co.uk/

    Logs from today shows GoogleBot crawling the new pages again but no sign of them indexed in Google :(
     
    fluid, Aug 3, 2006 IP
  7. frisby

    frisby Well-Known Member

    Messages:
    1,378
    Likes Received:
    39
    Best Answers:
    0
    Trophy Points:
    140
    #7
    I see 66 pages indexed.
     
    frisby, Aug 3, 2006 IP
    khasmoth likes this.
  8. fluid

    fluid Active Member

    Messages:
    679
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    70
    #8
    3 months back i had 66 pages indexed. And today it's still the same although i've added more than 500+ new pages which GoogleBot has been crawling but not added to its index. And it's not like i've added the new pages yesterday, there've actually been added long time back! So im confused why it's not updating its index.
     
    fluid, Aug 3, 2006 IP
  9. NetMidWest

    NetMidWest Peon

    Messages:
    1,677
    Likes Received:
    151
    Best Answers:
    0
    Trophy Points:
    0
    #9
    NetMidWest, Aug 3, 2006 IP
  10. CrankyDave

    CrankyDave Peon

    Messages:
    280
    Likes Received:
    23
    Best Answers:
    0
    Trophy Points:
    0
    #10
    Well, if you put any credence into what Matt Cutts has to say, whether or not your website is deemed "worthy enough" to index entirely is dependant upon the number and types of links pointing to your site.

    You have no links! Google shows 3 and when I checked one at random that Yahoo shows it's not indexed in Google...

    http://www.google.com/search?source...ww.abifind.com/cat/Shopping/Autos/Classifieds

    Working on adding content is a good thing but if you're not working on solid links pointing to your content it's not going to get indexed.

    Dave
     
    CrankyDave, Aug 3, 2006 IP
  11. fluid

    fluid Active Member

    Messages:
    679
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    70
    #11
    To NetMidWest

    What do you mean by:

    The articles that i have on my website are freely available to republish. So they must be on lots of other automative sites as well. Do you think that may be the problem with the indexing then? If you are positive about it, i'll get them removed. Please let me know.
     
    fluid, Aug 3, 2006 IP
  12. fluid

    fluid Active Member

    Messages:
    679
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    70
    #12
    fluid, Aug 3, 2006 IP
  13. CrankyDave

    CrankyDave Peon

    Messages:
    280
    Likes Received:
    23
    Best Answers:
    0
    Trophy Points:
    0
    #13
    You need to have anchor text links pointing to your site. The anchor text should be key phrases you wish to rank on. They should point to the most appropriate page of your site for that key phrase. They should be placed on directories and other related sites that get indexed.

    Dave
     
    CrankyDave, Aug 3, 2006 IP
  14. fluid

    fluid Active Member

    Messages:
    679
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    70
    #14
    Thanks for the tip CrankyDave.

    Surprising though, on MSN and yahoo i have about 20 links pointing to my site but on Google just two? How come?
     
    fluid, Aug 3, 2006 IP
  15. chachacallis

    chachacallis Well-Known Member

    Messages:
    1,087
    Likes Received:
    14
    Best Answers:
    0
    Trophy Points:
    128
    #15
    i dont think it was the sitemap that got you de-indexed i had the same problem a month ago an now all my webpages are now indexed. you should added the sitemap back and wait and see what happens.
     
    chachacallis, Aug 3, 2006 IP
  16. CrankyDave

    CrankyDave Peon

    Messages:
    280
    Likes Received:
    23
    Best Answers:
    0
    Trophy Points:
    0
    #16
    Do a Google search for the URL of the links you see on MSN and Yahoo. I linked to one of those URL's that I searched above.

    Dave
     
    CrankyDave, Aug 3, 2006 IP
  17. fluid

    fluid Active Member

    Messages:
    679
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    70
    #17
    Right. I did a search on yahoo and the directory 4ppl is linking to my site. Did a search on Google for this link and it was in the search results. So how come Google doesnt show this link as a valid incoming link?

    Here's the URL of the link:
    http://www.4ppl.com/directory/Shopping/Automotive/

    PS: There are other valid links showing on MSN and Yahoo which Google is not showing.
     
    fluid, Aug 3, 2006 IP
  18. CrankyDave

    CrankyDave Peon

    Messages:
    280
    Likes Received:
    23
    Best Answers:
    0
    Trophy Points:
    0
    #18
    Google only shows a sample of what they know, never all of them. If Google has the page your link is on indexed, then they either already know or will know about the link when the page is recrawled.

    Dave
     
    CrankyDave, Aug 3, 2006 IP