robots.txt

Find help with setting up your robots.txt file on your web site.

Page 7 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. blankeyecue

    robot.txt help

    blankeyecue, Jul 27, 2010
    Replies:
    4
    Views:
    1,815
    The_Hatta
    Sep 20, 2010
  2. sponge.nika

    I need a robots.txt to allow converted urls

    sponge.nika, Sep 4, 2010
    Replies:
    1
    Views:
    1,315
    digitalpoint
    Sep 4, 2010
  3. .TIEU

    All except One - Robots.txt

    .TIEU, Aug 28, 2010
    Replies:
    8
    Views:
    5,213
    .TIEU
    Sep 2, 2010
  4. Fracisc

    NoFollow links from robots.txt

    Fracisc, Aug 15, 2010
    Replies:
    7
    Views:
    1,823
    ravishanker
    Sep 1, 2010
  5. yulsa

    Do search crawlers see robots.txt or redirects first?

    yulsa, Aug 28, 2010
    Replies:
    3
    Views:
    1,714
    yulsa
    Aug 31, 2010
  6. mikstroa

    I have a doubt in robots.txt - need help

    mikstroa, Aug 30, 2010
    Replies:
    4
    Views:
    1,355
    mikstroa
    Aug 30, 2010
  7. jitendraag

    robots.txt for recruiting SEOs

    jitendraag, Aug 25, 2010
    Replies:
    3
    Views:
    1,442
    manish.chauhan
    Aug 30, 2010
  8. eniva

    Best Robots.txt For Wordpress Installed Blogs

    eniva, Sep 23, 2008
    Replies:
    12
    Views:
    11,137
    seo.lover
    Aug 29, 2010
  9. solution9

    how i can force search engine crawler to crawl my site ?

    solution9, Aug 15, 2010
    Replies:
    8
    Views:
    3,828
    seo.lover
    Aug 29, 2010
  10. kevoh99

    Is robots.txt really safe?

    kevoh99, Jul 17, 2010
    Replies:
    12
    Views:
    3,085
    seo.lover
    Aug 29, 2010
  11. saadhome2

    What Is Use Of Robot Text File.

    saadhome2, Mar 24, 2010
    Replies:
    13
    Views:
    2,589
    seo-india
    Aug 27, 2010
  12. jitendraag

    Last.fm's robots.txt

    jitendraag, Jul 28, 2010
    Replies:
    3
    Views:
    2,397
    jitendraag
    Aug 17, 2010
  13. smartamit04

    Can we use robot for content of page?????

    smartamit04, Aug 4, 2010
    Replies:
    2
    Views:
    1,224
    Mikesloug
    Aug 10, 2010
  14. khemraj

    Robot.txt help

    khemraj, Jul 30, 2010
    Replies:
    3
    Views:
    1,257
    bieberfc.com
    Aug 9, 2010
  15. astrohope

    What to Allow and Disallow on robots.txt

    astrohope, Jul 5, 2010
    Replies:
    6
    Views:
    2,289
    joseph pop
    Aug 5, 2010
  16. PuneetJvw

    How bad is it to NOT have a robots.txt file

    PuneetJvw, Jan 16, 2010 ... 2
    Replies:
    22
    Views:
    6,089
    smartamit04
    Aug 4, 2010
  17. catilucky

    robots.txt

    catilucky, Aug 2, 2010
    Replies:
    1
    Views:
    1,088
    zacharooni
    Aug 3, 2010
  18. Talker

    Robots.txt for Vbulletin

    Talker, Oct 13, 2006
    Replies:
    11
    Views:
    6,944
    zinghana
    Jul 31, 2010
  19. life31

    Please help me in listing the useful bots or user agents.

    life31, Jun 15, 2009
    Replies:
    3
    Views:
    2,098
    zero_124
    Jul 27, 2010
  20. Neil UK

    Robots.txt and Google crawl error

    Neil UK, Jul 10, 2010
    Replies:
    6
    Views:
    2,247
    mattstevens40
    Jul 27, 2010
  21. internet money making

    is it necessay robots.txt

    internet money making, Apr 27, 2010
    Replies:
    3
    Views:
    1,329
    travelguides
    Jul 25, 2010
  22. mobixone

    Please Guid me

    mobixone, Apr 16, 2009
    Replies:
    9
    Views:
    2,969
    newlogo
    Jul 20, 2010
  23. Coldfaun

    Problem with robots.txt

    Coldfaun, Jul 12, 2010
    Replies:
    14
    Views:
    1,810
    Groovystar
    Jul 17, 2010
  24. cDc

    Blocking certain querystring parameters

    cDc, Jul 13, 2010
    Replies:
    3
    Views:
    1,240
    manish.chauhan
    Jul 15, 2010
  25. jnelson563

    Help with blocking affiliate URLs

    jnelson563, Jun 17, 2010 ... 2
    Replies:
    23
    Views:
    3,971
    manish.chauhan
    Jul 12, 2010
  26. fadetoblack22

    Robots.txt not working for wordpress.

    fadetoblack22, Jul 8, 2010
    Replies:
    6
    Views:
    2,407
    fadetoblack22
    Jul 9, 2010
  27. Chalkywhite

    Blog logs and robots.txt

    Chalkywhite, Jul 8, 2010
    Replies:
    3
    Views:
    1,157
    manish.chauhan
    Jul 9, 2010
  28. wan.ariffuddin

    How do i remove my history from INTERNET ARCHIVE WAYBACK MACHINE

    wan.ariffuddin, Jul 8, 2010
    Replies:
    1
    Views:
    7,844
    manish.chauhan
    Jul 8, 2010
  29. arcademan

    block root directory and all subdirectories?

    arcademan, Jun 29, 2010
    Replies:
    2
    Views:
    1,573
    manish.chauhan
    Jul 7, 2010
  30. durgesh871

    How to analyze robots.txt for errors ?

    durgesh871, Jun 30, 2010
    Replies:
    3
    Views:
    1,365
    alexeyd
    Jul 2, 2010
  31. arcademan

    block root directory and all subdirectories?

    arcademan, Jun 29, 2010
    Replies:
    1
    Views:
    1,070
    manish.chauhan
    Jun 30, 2010
  32. milan.mysterio

    what is robot.txt and how is it important for SEO

    milan.mysterio, Jun 8, 2010
    Replies:
    8
    Views:
    1,764
    edpatton
    Jun 29, 2010
  33. Groovystar

    Need help on robots.txt

    Groovystar, Jun 24, 2010
    Replies:
    2
    Views:
    1,131
    Groovystar
    Jun 25, 2010
  34. Rakhi Chowdhary

    My Site Shows many pages which I don't want search engine to crawl

    Rakhi Chowdhary, Jun 22, 2010
    Replies:
    3
    Views:
    1,353
    manish.chauhan
    Jun 24, 2010
  35. Charlee22

    Pattern Matching in Robots.txt !

    Charlee22, Jun 22, 2010
    Replies:
    1
    Views:
    1,335
    manish.chauhan
    Jun 23, 2010
  36. Tanha Karim

    Is robots.txt file decrease our Adsense Revenue ?

    Tanha Karim, Jun 3, 2010
    Replies:
    4
    Views:
    1,515
    manish.chauhan
    Jun 22, 2010
  37. zinghana

    Robots.txt advice needed

    zinghana, Jun 18, 2010
    Replies:
    1
    Views:
    1,011
    manish.chauhan
    Jun 20, 2010
  38. SearchBliss

    Google's Robots.txt Tip

    SearchBliss, May 30, 2010
    Replies:
    5
    Views:
    1,609
    manish.chauhan
    Jun 13, 2010
  39. cahyoaja

    robot.txt for blogspot

    cahyoaja, May 8, 2010
    Replies:
    1
    Views:
    1,101
    manish.chauhan
    Jun 11, 2010
  40. Ralarash

    Robots txt file permission

    Ralarash, Apr 30, 2010
    Replies:
    9
    Views:
    22,540
    manish.chauhan
    Jun 11, 2010
  41. thosecars82

    robots.txt how prevent pages under website's root folder from being listed by Google

    thosecars82, Jun 3, 2010
    Replies:
    1
    Views:
    1,178
    manish.chauhan
    Jun 11, 2010
  42. No_name

    URL restriction tips

    No_name, Jun 2, 2010
    Replies:
    1
    Views:
    1,234
    manish.chauhan
    Jun 11, 2010
  43. 1rapid

    help make a robots.txt

    1rapid, Jun 6, 2010
    Replies:
    3
    Views:
    1,125
    manish.chauhan
    Jun 11, 2010
  44. ysribharath

    am i correct ?

    ysribharath, Jun 10, 2010
    Replies:
    1
    Views:
    963
    manish.chauhan
    Jun 10, 2010
  45. internet money making

    robot.txt is necessary ?

    internet money making, Apr 26, 2010
    Replies:
    10
    Views:
    1,666
    aeroz1
    Jun 7, 2010
  46. Imozeb

    Is this robots.txt file correct?

    Imozeb, May 5, 2010
    Replies:
    10
    Views:
    1,708
    luckzz1000
    May 30, 2010
  47. internet money making

    Robot tag

    internet money making, Apr 29, 2010
    Replies:
    4
    Views:
    1,154
    lonew
    May 29, 2010
  48. mithu.singh
    Locked

    What is robot.txt files.

    mithu.singh, May 22, 2010
    Replies:
    13
    Views:
    1,406
    goldenminds
    May 25, 2010
  49. jack_sparrow

    Help needed - robts.txt and sitemap

    jack_sparrow, Aug 3, 2007 ... 3 4 5
    Replies:
    89
    Views:
    24,250
    kevoh99
    May 25, 2010
  50. weathor

    Too many googlebot hits

    weathor, Apr 6, 2010 ... 2
    Replies:
    25
    Views:
    6,948
    kevoh99
    May 25, 2010
Showing threads 301 to 350 of 911

Thread Display Options

Loading...
Page 7 of 19