I use Joomla CMS and from my perspective it all comes down to compatibility. Some extensions work in PHP4 that don't work in PHP5 and viceversa.
There are a lot of differences. I think the most important was the OOP improvements, and the way objects are automatically passed as references (like they should be). PHP 5 has more functions and works faster. I don't think there is ANY excuse for using PHP4.
Many new functions were introduced in PHP5. Also, many old functions were edited. For eg strrcmp function. Also, as ThatKidBanks said some CMS dont work with PHP5. (JOOMLA 1.5)
In with the new out with the old. PHP depreciates several old PHP4 functions. It also add several new methods, functions...etc
It depends, what are you trying to do? If you are creating a brand new site, then use php5 as php 4 is no longer supported. If you are trying to migrate a site then you will need to look at many things like 'super-global' variables, short tags, deprecated functions and so forth. But I wouldn't even consider staying on php4, you might want to do the work now and move to php5 rather than later and be forced to take the plunge. FFMG