is it against the rules to open google ads in a new window. I have seen it done on other web sites, but I am not sure if it is against the rules.
It is against the rules to alter the code provided by G and in that code it doesn't open the page in a new window.
Only those google premier partners can open adsense ads in new windows. You can apply this if you have an established high traffic site.