I think it can be done either way as far as SEO is concerned. But subdomain looks more professiona IMO.
I don't think the 2 actually have anything in common... mysite.com/forum only specifies a certain directory and the forum.mysite.com is just a subdomain which you can change it later... so you can set it so that the subdomain forum.mysite.com points to mysite.com/forum yes, i think that's it. if someone could clarify.