RObots.txt help pleasee!!!!!!

Hi DP,

I want some with my Robots.txt file.

Okay, I want to edit that file in my Wordpress sites , but I'm not sure where to search for it.
I am hosted with Hostgator if that helps.

Thanks in advance!

 

If you can't find it, probably it doesn't exist?

 

u need to create that file and upload it to ur host

 

If you can't find one, then create a new one in your index, and name it as Robots.txt. And enter the following code in it.

User-agent: *
Disallow: 
Disallow: /cgi-bin/

Code (markup):

 

I just tried the

domain.com/robots.txt in the urlbar and it showed up

User-agent: *
Disallow:


I just wanted to check if it is right.
Now that it is, I don't need to edit it anymore.

However, I still couldn't find the file.

Thanks for all the help!

 

The will be located in 'public_html'.

 

you have to create Roboot.txt file and then upload it

 

Just like most folks have suggested here, you have to create the file then upload it via FTP. If you have wordpress installed in your root directory, the following might help

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/
Disallow: /wp-login.php
Disallow: /wp-register.php

User-agent: Mediapartners-Google
Allow: /

User-agent: Adsbot-Google
Allow: /

User-agent: Googlebot-Image
Allow: /

User-agent: Googlebot-Mobile
Allow: /

Cheers ;-)