WordPress itself is free. Hosting is really what you are paying for. Most hosting plans will enable you to have a WordPress installation free of charge as long as you pay for the hosting space (disk space and bandwidth) for the month. You can often get a starter plan for $5 / month.
Hosting can start cheap, and become expensive. It really depends on your needs. Starting off you can probably get going with $5 / month hosting or for more managed hosting going up to $25 / month to start.
I would suggest a $5 or $10/month host until your website starts doing good then switch to something reliable. The $5/month hosting services that offer Unlimited everything aren't so reliable... http://wordpress.org/download/ - wordpress download (free). You'll need hosting that supports php & mysql.