I am new to seo so please bare with me. If i have a website is it a good idea to have a RSS feed on every page?
From a SEO perspective having a RSS feed on every page will not make a difference. You want to display your RSS feed in highly visible spots on your website so it is seen. If that means placing it on every page then that is your decision, but many sites only place it on their homepage or other index pages like a news index page.
Use http://www.bigbold.com/rssdigest/ for an easy to implement feed with PHP or javascript code. It has been a good (free) service and is getting better with the much anticipated update.