Hi Guys, This is my first post. I have been getting into web development recently. Here is my second site: http://theultimatelinksharingsite.com/ Everything is going well except for one thing... Google doesn't seem to crawl my database. E.g. It picks up the links on index, contact us etc pages but does not go into all the link pages that have been posted. Here is a example: If I type '2000 - 2006 Nissan Almera Tino Service Repair Manual' into google it will return my home page that the link is on but it wont return the correct page, which is: http://theultimatelinksharingsite.com/Service%20Manuals/Nissan/2000%20-%202006%20Nissan%20Almera%20Tino%20Service%20Repair%20Manual As you can imagine this would have a negative effect on driving traffic to my site. Thanks for reading, any help appreciated. Brian
Well I wouldn't say there is any... I am going to put some work into back links but didn't think that would effect Google crawling my site. Does it? Can you explain a briefly how it would? Thanks, Brian
Google definitely does not crawls your database. It crawls you pages. I do see that google crawls your pages, not only yhr ones from your index. Just go to google and write: site:http://theultimatelinksharingsite.com It will shows all indexed pages, including ones generated from your DB. Since your website is new google will not visit/index it frequiently. If your website's content will change frequently, with the time google will mark your website and will visit/index more often. What you can do now, is make sure all your new pages are linked, sitemap updated. Try to obtain back links to your new pages (from known websites). As example, use web directories or classified websites. Such websites indexed almost every hour. Yoiur link from these pages will be picked-up and indexed faster. Use secial networks. Like Twitter. Post tweet about your page and put link to your new page.
Oh I didn't realise you can do that site: search. So this shows Google does index the database but it doesn't list the pages in the search. For example I copied one of the page title that is in the database that shows when I search for site:http://theultimatelinksharingsite.com. I copied "Ducati 888 Service Manual | Service Manuals | FREE DOWNLOADBLE" (aware of the spelling mistake, corrected now but has not been indexed since) and even put it in quotes and Google did not return it even though I know it is there as I copied it through the site: search. Does this mean the only way I will get Google to list all the pages is to work on back links etc?
you have to do strong promotion for that keyword and use that keyword in the meta tags(title, description, keywords) of that page.
If you look at the following code you can see it already promotes the relevant keywords: <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <title>Ducati 888 Service Manual | Service Manuals | FREE DOWNLOADABLE</title> <meta name='description' content='Ducati 888 Service Manual | Service Manuals | FREE DOWNLOADABLE' /><meta name='keywords' content='Service Manuals, Ducati,Ducati 888 Service Manual' /><link rel="stylesheet" type="text/css" href="/style.css" /> <style type="text/css"> When I use the "" for the Google search and search the title exactly it doesn't even pick it up, it says no results found.
It will take time till all your pages will be indexed. To speedup the process you can create back links from websites which are indexed regularly (hourly). Lets say you can put free classified ad on craigslist with link to your page. This ad will be indexed by google within 1 hour and your link may be picked up from there.
Gotcha will do... Have to say though still very surprised google does not return even when it has nothing alternative to return!! It just says results without quotes etc... Thanks for the help. I've started building links so hopefully it will help.
I would say keep bookmarking in some top sites with high pagerank, simultaneously start gathering backlinks for your site. Contact people from your same niche and ask them to comment on your posts. Sooner or later you will get some free backlinks too from them...
Do you have any specific reason to have those space separated URL? Replace the spaced with some other easily readable character like - or _. May be that'd help. You can also generate a sitemap and submit it to Google.
Yea I was looking at the URL's before and was wondering about them. I think different browsers use % for a space. I will have a look and see if I can change it to somehting more suitable. As for the site map, there is already a up to date xml one. The RSS is not working properly yet. I'll have a look at getting that sorted and see what happens.