I was going to include a few small php scripts within my html page and was wanting to know if I need to include a mysql_connect for each one of can I just do one in the header?
It's not unsafe. It's an alternative. mysql_connect() is often use in procedural programming, while mysqli in object-oriented.
Depending on your database, you might want to connect/use/close every time you need a database access. Too many open connections slow a database down, and crashing with an open connection could cause you to lose database access. (This is true for any database, not just MySQL, and for any connection method.) The actual overhead of opening and closing the connection is tiny compared to the time it takes to transfer the data to or from the user.