Simple! If you have cPanel, create a backup of your home directory and then go to phpmyadmin. Select your database, go to "Export", select all the tables, choose "Complete inserts", then choose "Save to file" with no compression.
...or you can use joomlacloner.com It's a commercial joomla component but works like a charm when you need to restore your backup.
you can also use the joomla backup components. these components allows you to backup your site from your joomla admin panel. there is a backup category on joomla extemsions sites
@trichnosis, there are lots of components in backup section over there. Can you name a one u used before ? the best one?.
I would suggest you use cPanel backup which can backup everything for you. That's what I am using at the moment.
Indeed, if you use cPanel (i have knowledge over that cPanel) you can easily backup your site in different ways, i.e. backup website files, backup database, or make a full backup of both together.
just login to shell ( ssh ( use putty if on windows ) and tar + gz all the content of the public_html dir. then download to local box from ftp or http. move it of course to the right dir so you can download later. you can backup the database daily with a little joomla plugin that will send the database to your email works for me