Google - Latest word on dynamic vs. static URLs

[minstrel]

Dynamic URLs vs. static URLs
Written by Juliane Stiller and Kaspar Szymanski,
Google Search Quality Team
Monday, September 22, 2008

Chatting with webmasters often reveals widespread beliefs that might have been accurate in the past, but are not necessarily up-to-date any more. This was the case when we recently talked to a couple of friends about the structure of a URL. One friend was concerned about using dynamic URLs, since (as she told us) "search engines can't cope with these." Another friend thought that dynamic URLs weren't a problem at all for search engines and that these issues were a thing of the past. One even admitted that he never understood the fuss about dynamic URLs in comparison to static URLs. For us, that was the moment we decided to read up on the topic of dynamic and static URLs. First, let's clarify what we're talking about:

What is a static URL?
A static URL is one that does not change, so it typically does not contain any url parameters. It can look like this: http://www.example.com/archive/january.htm. You can search for static URLs on Google by typing filetype:htm in the search field. Updating these kinds of pages can be time consuming, especially if the amount of information grows quickly, since every single page has to be hard-coded. This is why webmasters who deal with large, frequently updated sites like online shops, forum communities, blogs or content management systems may use dynamic URLs.

What is a dynamic URL?
If the content of a site is stored in a database and pulled for display on pages on demand, dynamic URLs maybe used. In that case the site serves basically as a template for the content. Usually, a dynamic URL would look something like this: http://code.google.com/p/google-chec...s/detail?id=31. You can spot dynamic URLs by looking for characters like: ? = &. Dynamic URLs have the disadvantage that different URLs can have the same content. So different users might link to URLs with different parameters which have the same content. That's one reason why webmasters sometimes want to rewrite their URLs to static ones.

Should I try to make my dynamic URLs look static?
Following are some key points you should keep in mind while dealing with dynamic URLs:

1. It's quite hard to correctly create and maintain rewrites that change dynamic URLs to static-looking URLs.
2. It's much safer to serve us the original dynamic URL and let us handle the problem of detecting and avoiding problematic parameters.
3. If you want to rewrite your URL, please remove unnecessary parameters while maintaining a dynamic-looking URL.
4. If you want to serve a static URL instead of a dynamic URL you should create a static equivalent of your content.

Which can Googlebot read better, static or dynamic URLs?
We've come across many webmasters who, like our friend, believed that static or static-looking URLs were an advantage for indexing and ranking their sites. This is based on the presumption that search engines have issues with crawling and analyzing URLs that include session IDs or source trackers. However, as a matter of fact, we at Google have made some progress in both areas. While static URLs might have a slight advantage in terms of clickthrough rates because users can easily read the urls, the decision to use database-driven websites does not imply a significant disadvantage in terms of indexing and ranking. Providing search engines with dynamic URLs should be favored over hiding parameters to make them look static.

Let's now look at some of the widespread beliefs concerning dynamic URLs and correct some of the assumptions which spook webmasters.

Myth: "Dynamic URLs cannot be crawled."
Fact: We can crawl dynamic URLs and interpret the different parameters. We might have problems crawling and ranking your dynamic URLs if you try to make your urls look static and in the process hide parameters which offer the Googlebot valuable information. One recommendation is to avoid reformatting a dynamic URL to make it look static. It's always advisable to use static content with static URLs as much as possible, but in cases where you decide to use dynamic content, you should give us the possibility to analyze your URL structure and not remove information by hiding parameters and making them look static.

Myth: "Dynamic URLs are okay if you use fewer than three parameters."
Fact: There is no limit on the number of parameters, but a good rule of thumb would be to keep your URLs short (this applies to all URLs, whether static or dynamic). You may be able to remove some parameters which aren't essential for Googlebot and offer your users a nice looking dynamic URL. If you are not able to figure out which parameters to remove, we'd advise you to serve us all the parameters in your dynamic URL and our system will figure out which ones do not matter. Hiding your parameters keeps us from analyzing your URLs properly and we won't be able to recognize the parameters as such, which could cause a loss of valuable information.

Following are some questions we thought you might have at this point.

Does that mean I should avoid rewriting dynamic URLs at all?
That's our recommendation, unless your rewrites are limited to removing unnecessary parameters, or you are very diligent in removing all parameters that could cause problems. If you transform your dynamic URL to make it look static you should be aware that we might not be able to interpret the information correctly in all cases. If you want to serve a static equivalent of your site, you might want to consider transforming the underlying content by serving a replacement which is truly static. One example would be to generate files for all the paths and make them accessible somewhere on your site. However, if you're using URL rewriting (rather than making a copy of the content) to produce static-looking URLs from a dynamic site, you could be doing harm rather than good. Feel free to serve us your standard dynamic URL and we will automatically find the parameters which are unnecessary.

Can you give me an example?
If you have a dynamic URL which is in the standard format like foo?key1=value&key2=value2 we recommend that you leave the url unchanged, and Google will determine which parameters can be removed; or you could remove uncessary parameters for your users. Be careful that you only remove parameters which do not matter. Here's an example of a URL with a couple of parameters:

http://www.example.com/article/bin/a...8906&query=URL

language=en - indicates the language of the article
answer=3 - the article has the number 3
sid=8971298178906 - the session ID number is 8971298178906
query=URL - the query with which the article was found is [URL]

Not all of these parameters offer additional information. So rewriting the URL to www.example.com/article/bin/answer.foo?language=en&answer=3 probably would not cause any problems as all irrelevant parameters are removed.

The following are some examples of static-looking URLs which may cause more crawling problems than serving the dynamic URL without rewriting:

www.example.com/article/bin/answer.foo/en/3/98971298178906/URL
www.example.com/article/bin/answer.foo/language=en/answer=3/
sid=98971298178906/query=URL
www.example.com/article/bin/answer.foo/language/en/answer/3/
sid/98971298178906/query/URL
www.example.com/article/bin/answer.foo/en,3,98971298178906,URL

Rewriting your dynamic URL to one of these examples could cause us to crawl the same piece of content needlessly via many different URLs with varying values for session IDs (sid) and query. These forms make it difficult for us to understand that URL and 98971298178906 have nothing to do with the actual content which is returned via this URL. However, here's an example of a rewrite where all irrelevant parameters have been removed:

www.example.com/article/bin/answer.foo/en/3

Although we are able to process this URL correctly, we would still discourage you from using this rewrite as it is hard to maintain and needs to be updated as soon as a new parameter is added to the original dynamic URL. Failure to do this would again result in a static looking URL which is hiding parameters. So the best solution is often to keep your dynamic URLs as they are. Or, if you remove irrelevant parameters, bear in mind to leave the URL dynamic as the above example of a rewritten URL shows:

www.example.com/article/bin/answer.foo?language=en&answer=3

We hope this article is helpful to you and our friends to shed some light on the various assumptions around dynamic URLs. Please feel free to join our discussion group if you have any further questions.

[offshore web development]

What i can say is "Excellent article"
Great job mate. Thanks for sharing this fantastic article.

[minstrel]

It should finally silence the vBSEO mod_rewrite crowd, though it probably won't.

[freewarecollection]

This article makes me shock.

[markowe]

Yeah, I was just reading this on the Google blog and was pretty stunned! Certainly a blow for conventional wisdom!

I would still be in favour of rewriting my URLs though - it seems Google is trying to discourage people from clumsily rewriting them and potentially preventing the GoogleBot from properly crawling content. If you have a consistent url structure, I don't see why that should cause a problem. Maybe the GoogleBot tries to navigate sites according to the apparent url structure, like http://mysite.com/item/fruit/banana -> http://mysite.com/item/fruit, even though there is no actual "fruit" page...

I would still use "friendly" urls though because:

- they look nicer
- they are friendlier to human beings
- there may still be a SEO benefit in the placing full titles etc. in the URL - surely item/fruit/banana is more indicative of the content than item.php?foodstuff=fruit&delicacy=banana..!

If Google wants to crawl my site, they can look at the sitemap, or follow the links for goodness' sakes!

[IEmailer.com]

Interesting find minstrel, thank you for sharing

[gspowart]

Thats a good read- thanks for sharing this.

I often wondered why dynamic URLs were such an (apparent) issue since they're generally unique and easy to read so no real issues.

I think it's like most things - don't go nuts with your dynamic URLs (like having hundreds of parameters) but don't be afraid to use them!

Having said that - I still like simple URLs!

[carlg]

So, does this mean instead of creating keyword friendly urls, I should create keyword friendly parameters?

In the past, I created keyword friendly urls such as:

www.example.com/blue-example.html

So does this now mean that

www.example.com/product.html?type=blueexmaple

is a better url than
www.example.com/product.html?type=12

[markowe]

Originally Posted by carlg

So, does this mean instead of creating keyword friendly urls, I should create keyword friendly parameters?

Good question! Again, I am sticking with "SEF" URLs...

[minstrel]

Originally Posted by carlg

So, does this mean instead of creating keyword friendly urls, I should create keyword friendly parameters?

In the past, I created keyword friendly urls such as:

www.example.com/blue-example.html

So does this now mean that

www.example.com/product.html?type=blueexmaple

is a better url than
www.example.com/product.html?type=12

No. It means youi don't need to and probably shouldn't do anything to the URL.

More information:

New study: which web page elements lead to high Google rankings?
22 May 2007

The German company Sistrix (paper in German) analyzed the web page elements of top ranked pages in Google to find out which elements lead to high Google rankings. They analyzed 10,000 random keywords, and for every keyword, they analyzed the top 100 Google search results.

Which web page elements lead to high Google rankings?
Sistrix analyzed the influence of the following web page elements: web page title, web page body, headline tags, bold and strong tags, image file names, images alt text, domain name, path, parameters, file size, inbound links and PageRank.

• Keywords in the title tag seem to be important for high rankings on Google. It is also important that the targeted keywords are mentioned in the body tag, although the title tag seems to be more important.
  
• Keywords in H2-H6 headline tags seem to have an influence on the rankings while keywords in H1 headline tags don't seem to have an effect.
  
• Using keywords in bold or strong tags seems to have a slight effect on the top rankings. Web pages that used the keywords in image file names often had higher rankings. The same seems to be true for keywords in image alt attributes.
  
• Websites that use the targeted keyword in the domain name often had high rankings. It might be that these sites get many inbound links with the domain name as the link text.
  
• Keywords in the file path don't seem to have a positive effect on the Google rankings of the analyzed web sites. Web pages that use very few parameters in the URL (?id=123, etc.) or no parameters at all tend to get higher rankings than URLs that contain many parameters.
  
• The file size doesn't seem to influence the ranking of a web page on Google although smaller sites tend to have slightly higher rankings.
  
• It's no surprise that the number of inbound links and the PageRank had a large influence on the page rankings on Google. The top result on Google has usually about four times as many links as result number 11.

[falsealarm]

OK, I get the idea they are trying to communicate but I still said "Oh my GOD" when I read it. What do they think we are idiots? Did they look at the example they were providing at the end? Who would rewrite their URLs in that fashion anyway?

I still say that we should rewrite dynamic URLs whenever possible as the article clearly states there is an advantage even though minimal. Yet, we need to ensure that the rewrites are done without causing any issues like the ones stated above. This article does not say "do not rewrite your dynamic URLs" as some people are taking it.

[kianah]

So Google is telling you now not make SEO friendly URLs. Pretty soon they'll be telling webmasters how to wipe their @$$es.

[minstrel]

Originally Posted by falsealarm

OK, I get the idea they are trying to communicate but I still said "Oh my GOD" when I read it. What do they think we are idiots? Did they look at the example they were providing at the end? Who would rewrite their URLs in that fashion anyway?

I still say that we should rewrite dynamic URLs whenever possible as the article clearly states there is an advantage even though minimal. Yet, we need to ensure that the rewrites are done without causing any issues like the ones stated above. This article does not say "do not rewrite your dynamic URLs" as some people are taking it.

Perhaps you missed this part:

Does that mean I should avoid rewriting dynamic URLs at all?
That's our recommendation,

and this:

www.example.com/article/bin/answer.foo/en/3

Although we are able to process this URL correctly, we would still discourage you from using this rewrite as it is hard to maintain and needs to be updated as soon as a new parameter is added to the original dynamic URL. Failure to do this would again result in a static looking URL which is hiding parameters. So the best solution is often to keep your dynamic URLs as they are.

[minstrel]

Major Google SEO Change: Google Prefers You Don't Use URL Rewrites
September 23, 2008
SEO RoundTable

The Google blog has officially said that they prefer that in most cases, you should not use rewrites to change your dynamic URLs to static looking URLs. In my opinion, this is a major 180° on basic SEO practices. SEO 101 is to make sure your dynamic URLs are search engine friendly. But now, that has all changed with this blog post and even well before this blog post.

Back in October 2006, we told you, Dynamic URLs? Google Is Officially 'OK' With Them. But even then, Google still recommended, "rewriting dynamic URLs into user-friendly versions" as good practice. Recently, in the past few months, we have covered many dynamic topics that all implied issues with using static URLs in place of dynamic URLs. Here are those blog posts:

Google Webmaster Tools Warns Of Spikes in Bandwidth Fees
Google: Duplicate Content Penalty Does Not Exist
Google Sitemaps "Too Many Redirects" Warning

Why the change of heart from Google? For one reason, they can now better understand the purpose of the page from the URL structure. JohnMu explained:

One of the reasons for that is that we can use the information provided through the parameters to better understand what your site is doing with those parameters. For instance,the URL http://www.mysite.com/search.php?q=keyword can give us information about what is happening, it could even allow us to recognize that this is a search form and perhaps let us attempt other keywords that might lead us to content that we haven't seen for your site. On the other hand, a URL like http://www.mysite.com/search/keyword does not give us any information at all about what the "file name" is used for.

You see, Google can learn more information about the page and what it is suppose to deliver based on the URL structure.

See also http://cre8pc.com/blog/archives/662

[JeddahBikers]

Really gr8 article , Thanks .

[dmi]

This surprised me. Great read!

[wilhb81]

WoW, a salute to the Big G, as he's always has some new ideas to torture the users lol

[umpump]

awesome article. I've heard both sides of the fence on this one. I currently use re-write for everything now, but it's good to know my full dynamic sites from the past arent really effected!

[varun8211]

Is this a good URL - http://trekshare.com/members/johnnyi...r-Winter-2000/

OR this one -

http://trekshare.com/member=johnnyic...ur-Winter-2000

[Francis Drake]

Old news?

Sure, it does not matter if you change an URL like this:
http://url.com/index.php?member=john&group=SEOs
to http://url.com/index/member/john/group/SEOs

but of course it still matters if you change an URL like
http://url.com/index.php?uid=2398&ugr=4
to http://url.com/index/member/john/group/SEOs