Don't allow access to http://<ipAddress>/<domain>

I just updated my webserver to Debian Sarge with Apache 2. So far, everything works fine, except one problem I never ran into before and haven't found anything trying to google for a solution (probably just because I don't know what to look for).

My websites are saved under var/www/<domain> (e.g. var/www/example) and I reduced the default config in /etc/apache2/site-available/default to this:

Servername localhost

NameVirtualHost *:80
NameVirtualHost *:8080
NameVirtualHost *:443

ErrorLog /var/log/apache2/error.log
CustomLog /var/log/apache2/access.log combined
ServerAdmin webmaster@localhost

DocumentRoot /var/www/

<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>

<Directory /var/www/>
AllowOverride None
Order deny,allow
</Directory>

ServerSignature On

<VirtualHost *:80>
</VirtualHost>
<VirtualHost *:8080>
</VirtualHost>
<VirtualHost *:443>
</VirtualHost>

In another include (mysites) I place the VirtualHost directive:

<VirtualHost *:80>
ServerName example.com
DocumentRoot /var/www/example
</VirtualHost>

<Directory /var/www/balance>
php_flag engine on
php_flag register_globals on
</Directory>

Obviously, this works just fine accessing the site with www.example.com but at the same time I can access it with 12.34.567.890/example as well.

I can't figure out how to configure apache not to allow that. I didn't find anything in the old servers config that was set up by somebody else.

So, I'm not sure if this is a completely trivial and therefore rather newbie problem or if Apache 2 handles this differently.

In any case if somebody could at least point me in the right direction I would greatly appreciate it.

P