google not indexing after site update

Discussion in 'Google' started by Digitalguy, Mar 30, 2010.

  1. #1
    recently i did a complete layout change + change of script.. earlier i had 100+ pages indexed in google but after switching to wordpress none of my pages were updated... google webmaster tools shows no known errors also i see frequent bot visits.. what can be the problem ? usually how long does it take to index after a site/script upgrade ? its a little over a week now ;)

    update:

    i just noticed a yourdomain.com/robots.txt saying

    but no such file in my cpanel root folder :p

    ps: wordpress privacy is set to public
     
    Last edited: Mar 30, 2010
    Digitalguy, Mar 30, 2010 IP
  2. Sxperm

    Sxperm Notable Member

    Messages:
    4,386
    Likes Received:
    142
    Best Answers:
    0
    Trophy Points:
    225
    #2
    I would recommend you to not addicted too much on the way Google cache your updated page for now. Sometime your new added page got cached within a few seconds after the update, sometime it does take a day before got cached. This could happen even you have done a lot of link building or submit your content to social bookmark already. This could happen sometime even your submiited contetn has being brought up to front page in major social bookmark such as Digg. Googlebots does acting weird lately because some major update is going on. Wait and see :)
     
    Sxperm, Mar 30, 2010 IP
  3. rostpad

    rostpad Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    I also have this problem. What do I do not know ...
     
    rostpad, Mar 31, 2010 IP
  4. Digitalguy

    Digitalguy Active Member

    Messages:
    545
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    60
    #4
    i updated my robots.txt to allow all search engines.. it might take some time to get updated.. btw google webmaster tool is still showing my old robots.txt (last downloaded 7 hours ago)
     
    Digitalguy, Mar 31, 2010 IP
  5. China_girl

    China_girl Well-Known Member

    Messages:
    4,095
    Likes Received:
    72
    Best Answers:
    0
    Trophy Points:
    195
    #5
    Atleast my google webmasters tool is showing the correct update to me. I dont really check it regularly.
     
    China_girl, Mar 31, 2010 IP
  6. WishBone

    WishBone Peon

    Messages:
    2,566
    Likes Received:
    15
    Best Answers:
    0
    Trophy Points:
    0
    #6
    What happened to my site before was, I was getting the result from a Data Center that is not yet updated with my site's content because when I ask a friend from other country to look at the cache of my site, he see the latest one. This could happen to your site too...
     
    WishBone, Mar 31, 2010 IP
  7. harsha22

    harsha22 Guest

    Messages:
    200
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Keep updating the content.. Google would index
     
    harsha22, Mar 31, 2010 IP
  8. Digitalguy

    Digitalguy Active Member

    Messages:
    545
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    60
    #8
    finally got indexed now... damn! im not sure how that weird robots.txt got there without my knowledge.. must be something to do with wordpress ;)
     
    Digitalguy, Mar 31, 2010 IP
  9. zonexx

    zonexx Peon

    Messages:
    80
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    removing robots file could be one way out. Coz that way you allow everything.
    anyways, it seems to be solved now.
     
    zonexx, Mar 31, 2010 IP
  10. Brandon Sheley

    Brandon Sheley Illustrious Member

    Messages:
    9,721
    Likes Received:
    612
    Best Answers:
    2
    Trophy Points:
    420
    #10
    is that a real answer, or a "spun" answer
    It sort of hurts my head to even read it...

    to the OP, did the url structure change?
    build more links, you'll be okay
    and you should modify your robots.txt file, you don't want the bots scanning everything, tell them what to focus on. If you're unsure what to place in it, do a google search for "wordpress robot.txt file" and you'll find several suggestions.
     
    Brandon Sheley, Mar 31, 2010 IP
  11. jaypuri

    jaypuri Peon

    Messages:
    79
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #11
    add the following meta tag in your site and resubmit the sitemap in Google Webmasters Tools

    <meta content='all, index, follow' name='googlebot'/>
    <meta content='all, index, follow' name='robots'/>
     
    jaypuri, Mar 31, 2010 IP
  12. Flashnest

    Flashnest Peon

    Messages:
    369
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #12
    Just be patient. I did a similar thing at the begining of the year... My new urls began to get indexed again after about a month or two but they still don't rank well yet like they used to... I made the mistake of not redirecting my old urls to my new ones...
     
    Flashnest, Mar 31, 2010 IP
  13. Digitalguy

    Digitalguy Active Member

    Messages:
    545
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    60
    #13
    Yes.. entire url structure was changed.. google is still trying to crawl my old pages, rss etc bot never gives up lol

    changed robots.txt to

    i think im kinda lucky.. now im starting to get traffic from google just couple of hours after allowing everything in robots.txt.
     
    Digitalguy, Mar 31, 2010 IP
  14. Michael smith

    Michael smith Peon

    Messages:
    313
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #14
    i only can say that just wait.. there is no any problem..
     
    Michael smith, Apr 1, 2010 IP
  15. China_girl

    China_girl Well-Known Member

    Messages:
    4,095
    Likes Received:
    72
    Best Answers:
    0
    Trophy Points:
    195
    #15
    So you are saying that it depends on the ip address and the country you are watching the results ?
     
    China_girl, Apr 1, 2010 IP
  16. jaypuri

    jaypuri Peon

    Messages:
    79
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #16
    Robots.txt file should be like this

    User-agent: *
    Allow: /

    Your file say "disallow". You will need to change it.
     
    Last edited: Apr 1, 2010
    jaypuri, Apr 1, 2010 IP
  17. wmvvincent

    wmvvincent Peon

    Messages:
    15
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #17
    Yes, I agree with you. I have been checking by blog recent posts are indexed but some old post still not showing.
    Thanks for sharing.
     
    wmvvincent, Apr 1, 2010 IP
  18. expertofexperts

    expertofexperts Active Member

    Messages:
    1,040
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    80
    #18
    Is there any problem with the wordpress theme? Find out that
     
    expertofexperts, Apr 1, 2010 IP