Log in or Sign up

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 5 of 19

Sort by:: Title Start Date; Replies Views; Last Message ↑

This one is for robots.txt expertsL How does one pull his off...

astounding, Jul 18, 2007

Replies:

3

Views:

1,181

trichnosis

Jul 22, 2007
blocking url like /?option=com_content&task=emailform&id=1000&itemid =8

trichnosis, May 13, 2007

Replies:

3

Views:

2,142

trichnosis

Jul 22, 2007
www. & non-www. (fixing robots.text)

theultimatepublishing, Jul 30, 2007

Replies:

5

Views:

1,359

trichnosis

Aug 6, 2007
Robots.txt please help!!!

frio, Jul 30, 2007

Replies:

3

Views:

1,519

trichnosis

Aug 6, 2007
Can I use robots.txt to protect my blog feed?

autorave, Mar 28, 2007

Replies:

6

Views:

2,609

trichnosis

Aug 6, 2007
Picsearch/PSbot - what's the verdict, ban it?

markowe, May 20, 2007

Replies:

2

Views:

2,874

trichnosis

Aug 6, 2007
psbot and gigabot wont stop even when banned from robots.txt

SaN-DeeP, Feb 28, 2006

Replies:

7

Views:

3,907

trichnosis

Aug 6, 2007
robot.txt

wijasenna, Jul 13, 2007

Replies:

3

Views:

1,960

trichnosis

Aug 6, 2007
Please review my Robots.txt

coolcow, Jul 17, 2007

Replies:

1

Views:

1,176

trichnosis

Aug 6, 2007
I really need your help

othonas, Jul 19, 2007

Replies:

7

Views:

1,466

trichnosis

Aug 6, 2007
Disallow search bots beyond the root folder content

Brajeshwar, Jul 26, 2007

Replies:

4

Views:

2,134

Brajeshwar

Aug 6, 2007
how many sitemaps do I put in the robots.txt file

Audiomad, Jul 10, 2007

Replies:

2

Views:

1,663

trichnosis

Aug 6, 2007
Bad Bot Advice

ArtfulWebSites, Mar 23, 2007

Replies:

10

Views:

2,843

trichnosis

Aug 6, 2007
A default robots.txt for all my site.......... Please review

danieloffice, Jul 13, 2007

Replies:

1

Views:

1,468

trichnosis

Aug 7, 2007
Traffic bot

turboadds, May 7, 2007

Replies:

2

Views:

1,711

trichnosis

Aug 7, 2007
Can someone check my robots.txt file

st_jimi, Jul 2, 2007

Replies:

3

Views:

1,685

trichnosis

Aug 7, 2007
I dont know the very basic thing about robots.txt

cocaine, Feb 26, 2007

Replies:

10

Views:

2,289

twistedspikes

Aug 8, 2007
How to block images directory?

sufi, May 26, 2007

Replies:

3

Views:

2,913

trichnosis

Aug 27, 2007
Standard Disallowed Pages with Robots.txt

Diablos, Aug 23, 2007

Replies:

2

Views:

1,201

Directory_SEO

Aug 27, 2007
Help in encouraging Yahoo and MSN robots to obedience

ThomaSeidler, Jun 14, 2007

Replies:

12

Views:

2,830

evera

Aug 28, 2007
How can I block a particular section of my forum?

Abhik, Aug 29, 2007

Replies:

1

Views:

1,169

evera

Sep 10, 2007
should i change this and my robots.txt and how?

scuba5794, Sep 6, 2007

Replies:

4

Views:

1,069

seoworld

Sep 11, 2007
Name of yahoo, google and msn bot

blabla11, Aug 8, 2007

Replies:

9

Views:

20,120

seoworld

Sep 11, 2007
Only googlebot,slurp and msnbot should crawl my site.

birdsq, Sep 10, 2007

Replies:

5

Views:

2,860

harrysmith

Sep 28, 2007
Does robots.txt work for all search engines?

Markas, Sep 29, 2007

Replies:

5

Views:

3,046

harrysmith

Oct 2, 2007
How to use robots.txt in blogs?

frogman49th, Oct 6, 2007

Replies:

0

Views:

1,037

frogman49th

Oct 6, 2007
301 Redirect the robots.txt outside the root domain?

n0chance, Oct 9, 2007

Replies:

3

Views:

3,593

mcfox

Oct 9, 2007
Adding 301 redirect for more than 1000 pages

greatlogix, Oct 3, 2007

Replies:

1

Views:

1,661

Ladadadada

Oct 14, 2007
How to block archive.org bot with both robots.txt AND .htaccess

EducationLinks, Aug 9, 2007

Replies:

6

Views:

14,986

hans

Oct 21, 2007
Error pages on my site - robots.txt ?

bad_bob00, Sep 21, 2007

Replies:

7

Views:

1,650

Nucca_The_Warrior

Oct 22, 2007
robots.txt and delet the pages

surekhaweb, Oct 2, 2007

Replies:

1

Views:

1,023

Nucca_The_Warrior

Oct 22, 2007
http code 200 mystery

clickalot, Oct 13, 2007

Replies:

4

Views:

1,899

clickalot

Oct 23, 2007
Excluding subdmains

frio, Oct 30, 2007

Replies:

3

Views:

1,055

Kuldeep1952

Oct 30, 2007
Help with robots.txt file

compindustries, Jun 19, 2006

Replies:

13

Views:

3,064

muks

Nov 1, 2007
Robots.txt Guide

Boardwalk, Oct 16, 2007

Replies:

6

Views:

2,012

hans

Nov 1, 2007
block pages in robots.txt

login, Nov 2, 2007

Replies:

2

Views:

2,401

Monty

Nov 2, 2007
Do I Need A Robots.txt?

rayqsl, Oct 13, 2007

Replies:

11

Views:

1,757

Kuldeep1952

Nov 2, 2007
Limit bot access frequency

softarea51, Nov 1, 2007

Replies:

7

Views:

5,361

ajsa52

Nov 5, 2007
How to make robots.txt file for my site

Armaan143, Oct 24, 2007

Replies:

8

Views:

5,282

Armaan143

Nov 6, 2007
Can I tell spiders to only come at night?

sirisaacnewton2007, Oct 17, 2007

Replies:

8

Views:

1,352

harrysmith

Nov 7, 2007
Let index just the HP

||NeX||, Nov 3, 2007

Replies:

4

Views:

1,102

||NeX||

Nov 9, 2007
Robots .txt restricted - assistance needed

snooks, Nov 9, 2007

Replies:

2

Views:

1,737

snooks

Nov 9, 2007
robots.txt

schwab, Nov 12, 2007

Replies:

5

Views:

1,237

schwab

Nov 13, 2007
What should a robots.txt say?

Seobiznezz, Nov 7, 2007

Replies:

4

Views:

1,435

silence

Nov 17, 2007
How to block keywords via robots.txt ?

Pammer, Nov 19, 2007

Replies:

2

Views:

3,316

Pammer

Nov 19, 2007
Robots.txt for Virtual Subdomains?

Avalanche, Nov 14, 2007

Replies:

1

Views:

1,294

energy.fs

Nov 21, 2007
Sort of a catch-22 here.

acrimon, Nov 15, 2007

Replies:

4

Views:

1,370

Ladadadada

Nov 22, 2007
Follow signup.php but NOT the proceeding query string

MrThePlague, Nov 27, 2007

Replies:

1

Views:

1,576

Kuldeep1952

Nov 28, 2007
robots.txt to deny all spiders for proxy?

Mysmasken, Nov 29, 2007

Replies:

3

Views:

21,251

Mysmasken

Nov 30, 2007
What is a Robot???

eathosting.com, Dec 1, 2007

Replies:

7

Views:

1,632

gab1982

Dec 20, 2007

Showing threads 201 to 250 of 911

Thread Display Options

Loading...

(You must log in or sign up to post here.)

Page 5 of 19