robots.txt

Find help with setting up your robots.txt file on your web site.

Page 7 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. blankeyecue

    robot.txt help

    blankeyecue, Jul 27, 2010
    Replies:
    4
    Views:
    1,772
    The_Hatta
    Sep 20, 2010
  2. sponge.nika

    I need a robots.txt to allow converted urls

    sponge.nika, Sep 4, 2010
    Replies:
    1
    Views:
    1,276
    digitalpoint
    Sep 4, 2010
  3. .TIEU

    All except One - Robots.txt

    .TIEU, Aug 28, 2010
    Replies:
    8
    Views:
    5,165
    .TIEU
    Sep 2, 2010
  4. Fracisc

    NoFollow links from robots.txt

    Fracisc, Aug 15, 2010
    Replies:
    7
    Views:
    1,762
    ravishanker
    Sep 1, 2010
  5. yulsa

    Do search crawlers see robots.txt or redirects first?

    yulsa, Aug 28, 2010
    Replies:
    3
    Views:
    1,650
    yulsa
    Aug 31, 2010
  6. mikstroa

    I have a doubt in robots.txt - need help

    mikstroa, Aug 30, 2010
    Replies:
    4
    Views:
    1,297
    mikstroa
    Aug 30, 2010
  7. jitendraag

    robots.txt for recruiting SEOs

    jitendraag, Aug 25, 2010
    Replies:
    3
    Views:
    1,389
    manish.chauhan
    Aug 30, 2010
  8. eniva

    Best Robots.txt For Wordpress Installed Blogs

    eniva, Sep 23, 2008
    Replies:
    12
    Views:
    11,048
    seo.lover
    Aug 29, 2010
  9. solution9

    how i can force search engine crawler to crawl my site ?

    solution9, Aug 15, 2010
    Replies:
    8
    Views:
    3,750
    seo.lover
    Aug 29, 2010
  10. kevoh99

    Is robots.txt really safe?

    kevoh99, Jul 17, 2010
    Replies:
    12
    Views:
    2,933
    seo.lover
    Aug 29, 2010
  11. saadhome2

    What Is Use Of Robot Text File.

    saadhome2, Mar 24, 2010
    Replies:
    13
    Views:
    2,478
    seo-india
    Aug 27, 2010
  12. jitendraag

    Last.fm's robots.txt

    jitendraag, Jul 28, 2010
    Replies:
    3
    Views:
    2,339
    jitendraag
    Aug 17, 2010
  13. smartamit04

    Can we use robot for content of page?????

    smartamit04, Aug 4, 2010
    Replies:
    2
    Views:
    1,198
    Mikesloug
    Aug 10, 2010
  14. khemraj

    Robot.txt help

    khemraj, Jul 30, 2010
    Replies:
    3
    Views:
    1,203
    bieberfc.com
    Aug 9, 2010
  15. astrohope

    What to Allow and Disallow on robots.txt

    astrohope, Jul 5, 2010
    Replies:
    6
    Views:
    2,210
    joseph pop
    Aug 5, 2010
  16. PuneetJvw

    How bad is it to NOT have a robots.txt file

    PuneetJvw, Jan 16, 2010 ... 2
    Replies:
    22
    Views:
    5,893
    smartamit04
    Aug 4, 2010
  17. catilucky

    robots.txt

    catilucky, Aug 2, 2010
    Replies:
    1
    Views:
    1,069
    zacharooni
    Aug 3, 2010
  18. Talker

    Robots.txt for Vbulletin

    Talker, Oct 13, 2006
    Replies:
    11
    Views:
    6,894
    zinghana
    Jul 31, 2010
  19. life31

    Please help me in listing the useful bots or user agents.

    life31, Jun 15, 2009
    Replies:
    3
    Views:
    2,055
    zero_124
    Jul 27, 2010
  20. Neil UK

    Robots.txt and Google crawl error

    Neil UK, Jul 10, 2010
    Replies:
    6
    Views:
    2,165
    mattstevens40
    Jul 27, 2010
  21. internet money making

    is it necessay robots.txt

    internet money making, Apr 27, 2010
    Replies:
    3
    Views:
    1,300
    travelguides
    Jul 25, 2010
  22. mobixone

    Please Guid me

    mobixone, Apr 16, 2009
    Replies:
    9
    Views:
    2,863
    newlogo
    Jul 20, 2010
  23. Coldfaun

    Problem with robots.txt

    Coldfaun, Jul 12, 2010
    Replies:
    14
    Views:
    1,715
    Groovystar
    Jul 17, 2010
  24. cDc

    Blocking certain querystring parameters

    cDc, Jul 13, 2010
    Replies:
    3
    Views:
    1,202
    manish.chauhan
    Jul 15, 2010
  25. jnelson563

    Help with blocking affiliate URLs

    jnelson563, Jun 17, 2010 ... 2
    Replies:
    23
    Views:
    3,753
    manish.chauhan
    Jul 12, 2010
  26. fadetoblack22

    Robots.txt not working for wordpress.

    fadetoblack22, Jul 8, 2010
    Replies:
    6
    Views:
    2,321
    fadetoblack22
    Jul 9, 2010
  27. Chalkywhite

    Blog logs and robots.txt

    Chalkywhite, Jul 8, 2010
    Replies:
    3
    Views:
    1,111
    manish.chauhan
    Jul 9, 2010
  28. wan.ariffuddin

    How do i remove my history from INTERNET ARCHIVE WAYBACK MACHINE

    wan.ariffuddin, Jul 8, 2010
    Replies:
    1
    Views:
    7,733
    manish.chauhan
    Jul 8, 2010
  29. arcademan

    block root directory and all subdirectories?

    arcademan, Jun 29, 2010
    Replies:
    2
    Views:
    1,546
    manish.chauhan
    Jul 7, 2010
  30. durgesh871

    How to analyze robots.txt for errors ?

    durgesh871, Jun 30, 2010
    Replies:
    3
    Views:
    1,312
    alexeyd
    Jul 2, 2010
  31. arcademan

    block root directory and all subdirectories?

    arcademan, Jun 29, 2010
    Replies:
    1
    Views:
    1,043
    manish.chauhan
    Jun 30, 2010
  32. milan.mysterio

    what is robot.txt and how is it important for SEO

    milan.mysterio, Jun 8, 2010
    Replies:
    8
    Views:
    1,693
    edpatton
    Jun 29, 2010
  33. Groovystar

    Need help on robots.txt

    Groovystar, Jun 24, 2010
    Replies:
    2
    Views:
    1,111
    Groovystar
    Jun 25, 2010
  34. Rakhi Chowdhary

    My Site Shows many pages which I don't want search engine to crawl

    Rakhi Chowdhary, Jun 22, 2010
    Replies:
    3
    Views:
    1,326
    manish.chauhan
    Jun 24, 2010
  35. Charlee22

    Pattern Matching in Robots.txt !

    Charlee22, Jun 22, 2010
    Replies:
    1
    Views:
    1,316
    manish.chauhan
    Jun 23, 2010
  36. Tanha Karim

    Is robots.txt file decrease our Adsense Revenue ?

    Tanha Karim, Jun 3, 2010
    Replies:
    4
    Views:
    1,463
    manish.chauhan
    Jun 22, 2010
  37. zinghana

    Robots.txt advice needed

    zinghana, Jun 18, 2010
    Replies:
    1
    Views:
    991
    manish.chauhan
    Jun 20, 2010
  38. SearchBliss

    Google's Robots.txt Tip

    SearchBliss, May 30, 2010
    Replies:
    5
    Views:
    1,509
    manish.chauhan
    Jun 13, 2010
  39. cahyoaja

    robot.txt for blogspot

    cahyoaja, May 8, 2010
    Replies:
    1
    Views:
    1,083
    manish.chauhan
    Jun 11, 2010
  40. Ralarash

    Robots txt file permission

    Ralarash, Apr 30, 2010
    Replies:
    9
    Views:
    22,434
    manish.chauhan
    Jun 11, 2010
  41. thosecars82

    robots.txt how prevent pages under website's root folder from being listed by Google

    thosecars82, Jun 3, 2010
    Replies:
    1
    Views:
    1,148
    manish.chauhan
    Jun 11, 2010
  42. No_name

    URL restriction tips

    No_name, Jun 2, 2010
    Replies:
    1
    Views:
    1,200
    manish.chauhan
    Jun 11, 2010
  43. 1rapid

    help make a robots.txt

    1rapid, Jun 6, 2010
    Replies:
    3
    Views:
    1,075
    manish.chauhan
    Jun 11, 2010
  44. ysribharath

    am i correct ?

    ysribharath, Jun 10, 2010
    Replies:
    1
    Views:
    941
    manish.chauhan
    Jun 10, 2010
  45. internet money making

    robot.txt is necessary ?

    internet money making, Apr 26, 2010
    Replies:
    10
    Views:
    1,563
    aeroz1
    Jun 7, 2010
  46. Imozeb

    Is this robots.txt file correct?

    Imozeb, May 5, 2010
    Replies:
    10
    Views:
    1,628
    luckzz1000
    May 30, 2010
  47. internet money making

    Robot tag

    internet money making, Apr 29, 2010
    Replies:
    4
    Views:
    1,097
    lonew
    May 29, 2010
  48. mithu.singh
    Locked

    What is robot.txt files.

    mithu.singh, May 22, 2010
    Replies:
    13
    Views:
    1,330
    goldenminds
    May 25, 2010
  49. jack_sparrow

    Help needed - robts.txt and sitemap

    jack_sparrow, Aug 3, 2007 ... 3 4 5
    Replies:
    89
    Views:
    23,414
    kevoh99
    May 25, 2010
  50. weathor

    Too many googlebot hits

    weathor, Apr 6, 2010 ... 2
    Replies:
    25
    Views:
    6,699
    kevoh99
    May 25, 2010
Showing threads 301 to 350 of 911

Thread Display Options

Loading...
Page 7 of 19