Subdirectory not being crawled OR indexed!

Discussion in 'Websites' started by tateote, Aug 3, 2005.

0
  1. #1
    I have a site www.pursefanatic.com which is being indexed and crawled on a regular basis by all the search engines.

    However, my articles directory HAS NOT been crawled once.

    WHY??? What do I need to do to get it crawled?

    help!

    This is the link I want indexed:

    http://www.pursefanatic.com/articles/
     
    tateote, Aug 3, 2005 IP
  2. soapbath

    soapbath Peon

    Messages:
    909
    Likes Received:
    45
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Is the site linked to from other places? Do you have a robots.txt in the folder? How long has the folder been there? Are all the articles Dups?
     
    soapbath, Aug 3, 2005 IP
    Crazy_Rob likes this.
  3. daed

    daed Peon

    Messages:
    93
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Yeah I have the same problem.. I watch the bots crawl the first page and just sit there all retarded like... it's ridiculous. I don't have a robots.txt file, and I am paying DP members and other people to have links on PR3-4-5 sites.. and still the bots hesitate and crawl random pages only. I almost wonder if the bots do it specifically to toy with us so we can't figure them out :p
     
    daed, Aug 3, 2005 IP