Indexing question with wordpress

Discussion in 'WordPress' started by wskmm, Feb 16, 2009.

  1. #1
    With a self-hosted Wordpress blog, if you set your home page to have say 3 posts, when you make a new post is it normal for a new post to be indexed initially both on the home page and also indexed simultaneously as a permalink and both to show up in google hopefully at same time? Or does the home page get indexed initially and later also the permalink?
     
    wskmm, Feb 16, 2009 IP
  2. mrhrk01

    mrhrk01 Well-Known Member

    Messages:
    664
    Likes Received:
    28
    Best Answers:
    0
    Trophy Points:
    145
    #2
    That depends on whether you have any restrictions set up in your robots.txt so that whether the crawler/bots will index/follow your home page or the other post/page i believe.
     
    mrhrk01, Feb 18, 2009 IP
  3. PowerZ

    PowerZ Peon

    Messages:
    74
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Chec kyour robots.txt file, and then the bots will follow your homepage to other pages, i think anyway
     
    PowerZ, Feb 18, 2009 IP
  4. LinksAndTraffic

    LinksAndTraffic Peon

    Messages:
    187
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    0
    #4
    I have seen strange activity from google on this count. You would think that they would crawl the index page first and then crawl the internal pages. But my observations of their activity - by checking the archived copy of pages in their index - is much different.

    I noted on several occasions that while they are working with an archive of the index that is weeks old, the internal pages of the blog get crawled usually within 24-48 hours of posting.

    I had been annoyed at the fact that they were not regularly pulling the index page, because I wanted my internal pages crawled. It worked out that they are crawling the internal pages when I want them to do so, but I am still somewhat confused that they don't pick up the index page as frequently as I believe that they would.
     
    LinksAndTraffic, Feb 18, 2009 IP
  5. wskmm

    wskmm Peon

    Messages:
    112
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Does the robots.txt file have to have something specific in it so that both the index as well as individual posts as permalinks get crawled? Don't understand the robots.txt too well. Thanks
     
    wskmm, Feb 23, 2009 IP
  6. ghassan1988

    ghassan1988 Peon

    Messages:
    12
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    On of my blogs got indexed with the title of a topic under the description, i find that strange
     
    ghassan1988, Feb 23, 2009 IP