I've been trying to get pages indexed but I notice only the pages immediately linked from the index page get spidered at all. I've tried various spider simulators with the same results...however when I use a tool to create a google sitemap it spiders all the pages I want fine and I've submitted this sitemap. But currently only about 5 pages get spiderd, when on the second page I'm linking to a whole directory filled with unique highly related content. I've experimented with different links on my index page, and whatever page I link to get spidered but then nothing more...can anyone offer some help about what I can do?
i have solution for u.. you have index page ok.. so make one another page ie. seo.html and put that page's link on index page so, that page will index quickly. now do one thing put all page's link on seo.html page using anchor text of your choice. now when your seo.html page crawled all pages whose link are there on that seo.html page. i hope this will work fine for you.. just give update here so other members can know about it. all the best.