It all depends on the size of the proxy and possible estimate traffic. Try places such as GoDaddy. SAWHosting. Are you talking about an online proxy? or a downloadable one?
Really you wouldn't need much for minimum/no amounts of traffic, you just need to be with a host that can easily scale up to bigger packages without much hassle if you need it. If it's just for you to use say at a work place or something, and not much traffic would go through, you wouldn't need much at all.
I am talking about an online proxy. I am planning on letting others using it, and getting a lot of traffic. Would a VPS work? Or do I have to have a fully dedicated server.
A VPS would work, even shared hosting at first (before too much traffic). The only issue is if you do go with shared hosting, make sure the host can help with transfer to VPS or dedicated. You would also want to make sure the host allows proxies on their shared servers.
You need to buy dedicated server only, if you buy VPS then it will bycross resource usage then owner will send you warning mail, also always disable smtp because people are sending spam mails through proxy, if its then your IP and website will be blacklisted.
Even shared hosting would be fine to start... just make sure it's something scalable. Bear in mind the problem with proxy sites is the type of traffic they attract... you are bound to run into trouble...
Since you only started proxy,then shared hosting is the best choice. Once you get lot traffic,only switch to vps.