An article I just wrote. Also note the little adsense tip. Feel free to add yours. Or anything you know of google and products and would like to share. ------------------ Is google just a search engine? Is google just a search engine? Well no, it is also a webmasters site info center... Notice I said webmaster, people having websites like you and me! Today I will show you: How to use google and find broken URL's on your website, How to know where google ranks your website without scanning page after page in results. A huge time saver. How to know which keywords people type on google when your website is displayed in results. Very useful to fine tune your website and target higher searched terms. How to know if google's having troubles indexing your website, and fix them. How to make a ROBOTS.TXT file and instantly test it for errors. All this using just your GMAIL email ID! So let's start, shall we? Log on to http://google.com/webmasters/sitemaps In the login form, enter your gmail ID and password and click login. Once logged in, you will see a box which says "add a site". Enter the URL of your website in this box and click "OK". For example I enter: http://www.businessbazee.com Now if google is able to connect to this site, meaning there are no DNS or server troubles, it will show you a confirmation message and add your site to your account. If there are DNS troubles, you need to contact your host, because right now google is not able to connect to your website which means it won't be able to index your pages either. Now that you have added your website, return back to the previous page. Here you will see the URL of your website next to a check box. (If you used the back button, you will have to refresh the page). You will also see some other information like Sitemap, and Verify. These are what we are interested in. By clicking sitemap, you will be able to add a sitemap which is basically a list of URL's under your domain. You can simply type each URL in a text file(notepad) and submit it to google. Or you can use a sitemap generator. There are many free ones available. Search google for "online free sitemap generator". Using these you will be able to specify additional information like when you change your pages and so on in your sitemap. The verify link. Once you have verified that you are the website owner of the website you added, google will show you lot of information about your website. Don't tell this to anyone, but I used this to boost my adsense income... To verify, click the verify link and google will ask you to upload a blank html file on your website. Now if you are the owner of the website you added, you have the ability to upload pages to it. This is what google wants to confirm. If they find that file on your server, the site's verified and it's safe to show you information about it! Once verified, in your account click the URL of your website you added. On the page that follows in the side menu, click "STATISTICS". If google is listing your website in search results, it will show you all the terms which people are searching for when your website comes before them.( Top search queries). How is this useful for you? In a minute about that... It also shows you the location of your site in google results. (Average top position). Then it also will show you if the person searching clicked the link of your website or did they ignore it. (Top search query clicks) Here's how to use this information to benefit you. Recently google started listing 1 of my free resources website in it's results and at a prominant position. This particular website was listed for many "Top search queries". Example html templates, free templates etc. (just examples) However the people seeing my website were not visiting the website as google was not showing these pages in "Top search query clicks". What did I do? I simply changed the title tags of these pages a little. Example my previous title displayed in search results: Softwares, scripts, templates, downloads. New title: Lots of free html templates,softwares,scripts,free download! What happenned? People started clicking my title in google results and now I have good amount of traffic coming on those websites! I know that google shows my site for templates, not for softwares and people looking at my site are also interested in just templates. So I told them that there are many free templates here. Between you and me, I also placed google adsense on that website. People are clicking those too... Moving on to the next section: Crawl Errors. Here google lists any URL's of your website which their index bots were unable to find or index because of the following reasons: HTTP errors... This is when your page won't load when it's URL is typed in the browser. You need to check if all links on your website are pointing to the exact page. May be there's a typo in the navigation? Example I named a page "products.html" while making it but accidentally when addit in navigation, I typed "product.html". Obviously, both google and all my visitors on my website are getting a 404 page not found error. Time to fix it! Note google also lists URL's linked from external sites. You won't be able to control it in case you don't control your link management completely. (Advertisements you place on other websites, reciprocal link building etc.) Then there are listings for pages which google was unable to index for other reasons like, Network errors, follow errors and similar ones. I would like to mention follow errors here. Same example of product.html as above: I renamed the page as "products.php" and instead of updating all navigation, contacting everyone linking to this page, I used a redirect method. If someone types. mydomain.com/products.html in browser, they will be redirected to mydomain.com/products.php now. Google and other search engines will not be able to follow all types of redirects for various reasons, resulting in eventually dropping this page from search results. If this page was responsible for plenty of traffic from search engines, then it's worth following google's webmaster guidelines for this purpose. Finally let's look at the ROBOTS.TXT tool that google gives me. Robots.txt is a simple text file placed in the root of your domain to guide search engines for better and restricted indexing. If you want all your pages indexed by all search engines, simply open your notepad and type the following: User-agent: * Save the blank document by the name "robots.txt". Now upload this to your root directory, where you have your index.html placed. Some search engines like MSN for example, need a robots.txt file in order to start indexing your website. If they don't find this file, they won't index your website. Google however uses this file just to list the restricted URL's which you don't want search engines to index. Before adding any URL in your robots.txt, you can see if search engines like google will actually mark them as restricted or not. This is how: Click "robots.txt" in your google account. In the first large box (textarea), type all URL's you want to block. Don't worry, this is just a test and these pages will not actually be blocked while checking. Example I want my CGI-BIN directory to be blocked by all search engines. So now the code in my file looks something like below. User-agent: * Disallow: /cgi-bin/ Second is a drop down box which you can change in case you want a specific google bot to be restricted. Leave it as is for just the crawler google bot. Click "check". Google now shows you if this particular URL you just checked will be blocked or not. Once you have checked all URL's you want to block, then modify your actual robots.txt and upload it to your server. I strongly suggest you use this tool as google is capable of indexing even deeper than you think it could. That's another topic... Enjoy! --------------- Credits: Jeet runs websites likewhere you canCompletely automate your online business and prevent download theft. Start your online business with not 1, not 2, not 3 but lots of products and get them for pennies Best wishes jeet