For anyone who didn' t know I just found out you can create a robots.txt file right in google web tools! Check it out: