If the domain is domain.com and do I just point it to ns1.domain.com and ns2.domain.com? Or do I have to do something with A records? Thanks!
You need to setup ns1.domain.com and ns2.domain.com on your dedicated server using BIND/NSD, then register the IP addresses for ns1/ns2 with your domain registrar. After that's done, set your domain's nameservers to ns1/ns2.domain.com and you should be good to go.
If you don't want to deal with setting up your DNS properly (you need two or more servers), you can use something like dns.he.net or another free DNS host.
A records need to be set on the Hosting Account DNS Zone. Also you need to setup child nameserver at your domain registrar. You can also setup Redundant DNS so if you running server under cloud