Dynamic site Crawling and Ranking Issues? Need Help?

Hi All,

Currently I am optimizing a dynamic website and facing certain crawling and ranking issues. I hope will get proper solution here.

Actually my site is a fully dynamic website using DB. There is one index.php page and on root there is one folder called /pages/. In pages folders we have maintained other dynamic files which will includes together to make one page.

These pages are include files e.g. If i need to called the footer portion than I include the footer file into the main index.php file. Pages will generate on click. When somebody clicks on the a link they will go to index.php and the query will be like www.mysite.com/index.php?view=file. This will show the content of file.php file which is there in the /pages/ folder. And again I am using .htaccess file for mod rewriting. So the same URL given above will be look like http://www.mysite.com/category/file/index.html.

Each single file will appear with / and index.html. Please tell me is that ok? Can I make all pages as /index.html?

One more thing as the site is dynamic. Number of pages a day give 404 OR are delted from its location? so in that case I am redirecting that particular page to its upper level category page. I am using 301 for this redirection. Is this a proper thing to use 301 redirection for all the not found page?

OR Its better to have a custom 404 page for in case of high volume of not found problems, especially for a dynamic website.

I am also using a canonical tag for each URLs as well as a base tag. As its a dynamic website I need to use them to tell GoogleBot that for this page you have to crawl and index the Canonical tag URLs only. because sometings Bot considered the Sessions ID URls and create duplications.

Actually its a very vast website. I am doing lot of efforts for this site but dont know why it takes too much time for next crawl. Also let me know if somebody have any formula for a dynamic site for indexing and crawling specially.

Thanks for any help in advance.

 

The / followed by index.html is fine as far as SE's are concerned, just make sure you optimize the rest of the url for each page.

As for the 404, you should make a useful 404 page and direct all page not found errors to it, and on the page give something useful and send the user to your homepage (or somewhere relevant to the page they were looking for). Also add some code to your 404 to send you an email or log the details of the 404, then you can check them to make sure they are genuine 404s and not your site doing something it shouldnt

Crawling speed will depend on lots of things, if the site is new or a lot of the content remains static (from a SE's point of view) then the crawl rate will be slower.

 

i agree.. create a proper 404 page that will take your visitor back to the homepage