robots.txt

Find help with setting up your robots.txt file on your web site.

Page 16 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↑
  1. TheSyndicate

    How do you get archiver to READ my robots?

    TheSyndicate, Mar 31, 2012
    Replies:
    6
    Views:
    3,396
    TheSyndicate
    Apr 27, 2012
  2. satish nandanwar

    How we use robot.txt file for our site pages?

    satish nandanwar, May 9, 2012
    Replies:
    1
    Views:
    2,964
    bsdstr
    May 17, 2012
  3. joerudy1988

    robot.txt to allow everything

    joerudy1988, May 3, 2012
    Replies:
    3
    Views:
    3,512
    joerudy1988
    May 17, 2012
  4. gdtechind

    How to Stop Web Robot from Engines other than G and Y and MSN

    gdtechind, Oct 27, 2005
    Replies:
    9
    Views:
    8,353
    Spearheadltd
    Jun 1, 2012
  5. riteshvyas52

    What is robots.txt ?

    riteshvyas52, May 4, 2012
    Replies:
    5
    Views:
    3,197
    JFSG
    Jun 3, 2012
  6. AlisaRozzy

    Help regarding robots.txt file for my website

    AlisaRozzy, May 19, 2012
    Replies:
    1
    Views:
    2,885
    war_machine
    Jun 3, 2012
  7. matthecool

    Blocking .php from google in robots.txt

    matthecool, Jun 4, 2012
    Replies:
    1
    Views:
    2,750
    digitalpoint
    Jun 4, 2012
  8. hvwebseo

    How to stop crawling of multi language site

    hvwebseo, Jun 5, 2012
    Replies:
    1
    Views:
    2,706
    RoseHosting
    Jun 5, 2012
  9. mariush

    Need advice on how to block certain page using robots.txt

    mariush, Aug 12, 2006
    Replies:
    7
    Views:
    10,594
    learn2success
    Jun 6, 2012
  10. iGeekSoftware

    Idea for robots.txt selective restrictions [done easy]

    iGeekSoftware, Jun 14, 2012
    Replies:
    1
    Views:
    3,075
    atxsurf
    Jul 3, 2012
  11. russel_shuvo2

    Robot.txt block my major part

    russel_shuvo2, Jun 11, 2012
    Replies:
    2
    Views:
    3,122
    atxsurf
    Jul 3, 2012
  12. kartarsingh

    I need anyone expert who can create robots.txt file for myBB forum

    kartarsingh, Mar 17, 2012
    Replies:
    3
    Views:
    6,801
    Pepdeal
    Jul 28, 2012
  13. dropdrop

    Robots.txt only homepage

    dropdrop, Jul 6, 2012
    Replies:
    4
    Views:
    3,131
    mr.siiin
    Jul 30, 2012
  14. thundercow

    How to use Robots.txt to get only front page indexed?

    thundercow, Dec 12, 2008
    Replies:
    6
    Views:
    4,573
    Jason0078
    Jul 30, 2012
  15. PeterA

    What does this robots.txt file do?

    PeterA, Jan 15, 2010
    Replies:
    5
    Views:
    5,233
    andydxgame
    Aug 3, 2012
  16. Alex Decosta

    How Robots.txt works

    Alex Decosta, Apr 14, 2012
    Replies:
    6
    Views:
    4,345
    XpErT
    Aug 5, 2012
  17. idoz1985

    Pages with duplicate title tags

    idoz1985, Aug 2, 2012
    Replies:
    5
    Views:
    3,375
    ryan_uk
    Aug 5, 2012
  18. BuySellBazaar

    URL Redirection, Does Robot.txt has something to do with it?

    BuySellBazaar, Aug 12, 2012
    Replies:
    1
    Views:
    2,565
    sarahk
    Aug 12, 2012
  19. cloudseoguy

    search engine specific robot.txt [HELP]

    cloudseoguy, Aug 19, 2012
    Replies:
    0
    Views:
    2,154
    cloudseoguy
    Aug 19, 2012
  20. Rajnish357

    What this robot.txt will do ??

    Rajnish357, Mar 26, 2012
    Replies:
    8
    Views:
    3,955
    rjcipa
    Aug 21, 2012
  21. Cuekal

    Robots.txt - Need Help!

    Cuekal, Aug 28, 2012
    Replies:
    2
    Views:
    2,668
    singhbadshah
    Aug 28, 2012
  22. cuteboy

    Staging IP indexed by Google

    cuteboy, Sep 4, 2012
    Replies:
    0
    Views:
    2,926
    cuteboy
    Sep 4, 2012
  23. nitu99

    Unable to delete robots.txt file

    nitu99, Apr 25, 2012
    Replies:
    4
    Views:
    6,181
    PaulCpR
    Sep 5, 2012
  24. pooja111

    Robots.txt was inaccessible

    pooja111, Sep 15, 2012
    Replies:
    0
    Views:
    3,015
    pooja111
    Sep 15, 2012
  25. DevAdelicia

    What is Robots.txt ?

    DevAdelicia, Nov 17, 2011
    Replies:
    15
    Views:
    6,202
    xiaokaige
    Sep 18, 2012
  26. duglas1978
    Locked

    What is robot.text file ?

    duglas1978, Sep 26, 2012
    Replies:
    3
    Views:
    2,324
    duglas1978
    Sep 26, 2012
  27. alvinchua91

    How To Delete Robots.txt from Blogger Own Domain?

    alvinchua91, Sep 29, 2012
    Replies:
    3
    Views:
    2,789
    ramcharantej
    Sep 30, 2012
  28. alvinchua91

    will User-agent: * Disallow: Allow: / result in duplicate content issues?

    alvinchua91, Oct 5, 2012
    Replies:
    5
    Views:
    2,832
    Traditione
    Oct 7, 2012
  29. techfreakd

    How to disallow Google crawl link.???

    techfreakd, May 18, 2012 ... 2
    Replies:
    24
    Views:
    8,505
    Max796
    Oct 10, 2012
  30. erebussymk

    Is it go to index wordpress Tag or not ?

    erebussymk, Oct 11, 2011
    Replies:
    6
    Views:
    4,554
    manish.chauhan
    Oct 12, 2012
  31. Sunil Keshari

    Robot.txt file crawling pages very fast

    Sunil Keshari, Apr 26, 2012
    Replies:
    6
    Views:
    4,396
    selectaupairs
    Oct 30, 2012
  32. GENIUSTECH

    in a website robots.txt

    GENIUSTECH, Oct 26, 2012
    Replies:
    3
    Views:
    2,187
    dimitarang
    Oct 30, 2012
  33. tutordelphia

    robots.txt files with similar names

    tutordelphia, Nov 1, 2012
    Replies:
    1
    Views:
    1,945
    adwordmedia
    Nov 5, 2012
  34. offsitenoc

    How to Download google robot.txt file?

    offsitenoc, Jul 26, 2010
    Replies:
    3
    Views:
    11,008
    -[z]-
    Nov 7, 2012
  35. gaush007

    How to block .pdf or any documents from google bots apearing in search rankings

    gaush007, Oct 9, 2012
    Replies:
    5
    Views:
    2,777
    -[z]-
    Nov 7, 2012
  36. TheSyndicate

    Should i just block Baidu and Yandex?

    TheSyndicate, Sep 20, 2012
    Replies:
    3
    Views:
    2,878
    -[z]-
    Nov 7, 2012
  37. TheSyndicate

    ia_archiver is not listen to Robots.txt can i block there UP?

    TheSyndicate, Jul 10, 2012
    Replies:
    5
    Views:
    4,327
    -[z]-
    Nov 7, 2012
  38. doonn

    Please review this file

    doonn, Feb 2, 2012
    Replies:
    7
    Views:
    4,547
    -[z]-
    Nov 7, 2012
  39. praveen1

    Robots Error in Free Blogger---HELP?

    praveen1, Nov 6, 2012
    Replies:
    2
    Views:
    2,258
    praveen1
    Nov 9, 2012
  40. clive2

    robots.txt and redirected url

    clive2, May 8, 2012
    Replies:
    1
    Views:
    3,794
    danish001
    Nov 18, 2012
  41. p.caspian

    Sitemap using robot.txt !!

    p.caspian, Mar 21, 2012
    Replies:
    3
    Views:
    3,614
    danish001
    Nov 18, 2012
  42. alka007

    What is Robots.txt ?

    alka007, Feb 23, 2012 ... 2
    Replies:
    32
    Views:
    9,022
    sahostking
    Nov 22, 2012
  43. sandegownc

    what is the code to write robots.txt files ?

    sandegownc, Nov 21, 2012
    Replies:
    2
    Views:
    2,297
    kashishjain
    Nov 24, 2012
  44. Adhe Carere

    Problem With Robot.txt

    Adhe Carere, May 30, 2012
    Replies:
    6
    Views:
    4,071
    kashishjain
    Nov 24, 2012
  45. claudiajay

    Please suggest if my robots.text file is right.

    claudiajay, Nov 23, 2012
    Replies:
    4
    Views:
    2,837
    claudiajay
    Nov 26, 2012
  46. claudiajay

    100 Images on my e-commerce website

    claudiajay, Nov 23, 2012
    Replies:
    7
    Views:
    2,546
    ryan_uk
    Nov 26, 2012
  47. yogeshseo

    Robots.txt File Format is Right or wrong?

    yogeshseo, Mar 6, 2012
    Replies:
    7
    Views:
    4,526
    Renatus
    Dec 3, 2012
  48. geniusoptimizer

    Can't Remove Robot.txt file from Blogger

    geniusoptimizer, Sep 25, 2012
    Replies:
    7
    Views:
    3,532
    ryan_uk
    Dec 4, 2012
  49. gedarnet

    i'm need best robots.txt for wordpress photographers

    gedarnet, Nov 18, 2012
    Replies:
    12
    Views:
    3,273
    gedarnet
    Dec 7, 2012
  50. Dee2007

    Folder Question

    Dee2007, Dec 9, 2012
    Replies:
    0
    Views:
    1,652
    Dee2007
    Dec 9, 2012
Showing threads 751 to 800 of 911

Thread Display Options

Loading...
Page 16 of 19