robots.txt

Find help with setting up your robots.txt file on your web site.

Page 5 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. tradester

    robots.txt to prevent search engines from crawling/ indexing a page or folder

    tradester, Oct 13, 2011
    Replies:
    5
    Views:
    3,210
    bobwarner01
    Nov 4, 2011
  2. OSSEO

    How to restrict url from robots.txt file

    OSSEO, Sep 7, 2011
    Replies:
    11
    Views:
    4,007
    MyWebsiteNow
    Oct 17, 2011
  3. REtec

    Sub-domains and Robots.txt

    REtec, Sep 26, 2011
    Replies:
    2
    Views:
    3,022
    REtec
    Sep 29, 2011
  4. simplythewebst

    Some solution to prevent indexing my site except robots.txt

    simplythewebst, Sep 26, 2011
    Replies:
    1
    Views:
    2,645
    manig4
    Sep 27, 2011
  5. nikmahajan

    Blogger Robots.txt Problem- Google is not crawling

    nikmahajan, Aug 5, 2008
    Replies:
    8
    Views:
    10,414
    g4ngst3r
    Sep 25, 2011
  6. gogetit

    Robots.txt crawl error

    gogetit, Sep 20, 2011
    Replies:
    1
    Views:
    3,256
    aruninfo
    Sep 23, 2011
  7. Kalibaster

    Robots.txt for wordpress autoblog setup.

    Kalibaster, Sep 21, 2011
    Replies:
    0
    Views:
    3,241
    Kalibaster
    Sep 21, 2011
  8. www.worldofdth.in

    robot..txt file generator website

    www.worldofdth.in, Sep 6, 2011
    Replies:
    4
    Views:
    3,206
    yummysilent
    Sep 14, 2011
  9. VijayanKumar

    What Is Robots.txt?

    VijayanKumar, Jun 9, 2011 ... 2
    Replies:
    32
    Views:
    9,495
    webgfx
    Sep 14, 2011
  10. Rage

    how to create a page that search engine dont search???

    Rage, May 23, 2006
    Replies:
    5
    Views:
    4,270
    sophie1986
    Sep 5, 2011
  11. Gangakarthik

    Disable Feed pages

    Gangakarthik, Aug 30, 2011
    Replies:
    1
    Views:
    2,876
    babushkyn
    Sep 2, 2011
  12. Latestnewsin

    Urgently Google Rebot Help

    Latestnewsin, Aug 28, 2011
    Replies:
    1
    Views:
    2,735
    babushkyn
    Sep 2, 2011
  13. Ratman2050

    Is 32,000 Alexa Rank Good?

    Ratman2050, Oct 8, 2006 ... 2
    Replies:
    35
    Views:
    17,141
    icyfiren
    Aug 26, 2011
  14. Arcos

    How to STOP indexing web pages

    Arcos, Dec 23, 2005
    Replies:
    14
    Views:
    8,395
    jhon786
    Aug 25, 2011
  15. bluestar

    How to set right crawler errors?

    bluestar, Aug 1, 2011
    Replies:
    8
    Views:
    3,977
    jhon786
    Aug 25, 2011
  16. LanceT

    Will robots.txt remove my crawl errors

    LanceT, Aug 9, 2011
    Replies:
    1
    Views:
    3,421
    Chuman
    Aug 14, 2011
  17. Planet_x

    Block Querystring for all user-agents

    Planet_x, Aug 2, 2011
    Replies:
    1
    Views:
    3,546
    manish.chauhan
    Aug 5, 2011
  18. U-Tag

    My blog "robots.txt" contain any indexing error?

    U-Tag, Aug 4, 2011
    Replies:
    2
    Views:
    2,928
    U-Tag
    Aug 5, 2011
  19. xenoo

    Using Robots.txt for image hosting site?

    xenoo, Jun 24, 2011
    Replies:
    1
    Views:
    3,150
    manish.chauhan
    Aug 1, 2011
  20. Fking

    The latest posts page in WP brings duplication issues.

    Fking, Jul 8, 2011
    Replies:
    1
    Views:
    2,833
    manish.chauhan
    Aug 1, 2011
  21. jducey1234

    Using robots.txt for better search engine placement

    jducey1234, Jun 1, 2011
    Replies:
    10
    Views:
    4,041
    manish.chauhan
    Aug 1, 2011
  22. electroze

    Help! Yahoo crawler is sucking my bandwidth - 8,779 queries in last 3 days!

    electroze, Jul 20, 2011
    Replies:
    1
    Views:
    2,962
    manish.chauhan
    Aug 1, 2011
  23. i_am_dhaval

    is am i right with robots.txt file

    i_am_dhaval, Jul 11, 2011
    Replies:
    6
    Views:
    3,139
    manish.chauhan
    Aug 1, 2011
  24. NickMP3

    block all sub domains

    NickMP3, Jul 10, 2011
    Replies:
    3
    Views:
    3,652
    manish.chauhan
    Aug 1, 2011
  25. aj845

    unknown bots.

    aj845, Jul 6, 2007
    Replies:
    12
    Views:
    11,361
    cquinndesign
    Jul 3, 2011
  26. mjindustry

    Is this going to work? (robots.txt)

    mjindustry, May 3, 2011
    Replies:
    8
    Views:
    3,778
    upendraets
    Jun 25, 2011
  27. shaun_ru_th

    correct 301 redirect methord???

    shaun_ru_th, May 31, 2011
    Replies:
    5
    Views:
    3,578
    upendraets
    Jun 25, 2011
  28. micmic

    Flash files in robots.txt

    micmic, Apr 27, 2007
    Replies:
    5
    Views:
    6,343
    jbtooloo
    Jun 22, 2011
  29. upendraets

    how we make Robot.txt and how we add in our site?

    upendraets, Jun 17, 2011
    Replies:
    1
    Views:
    3,031
    seodash
    Jun 17, 2011
  30. sandeeppandey

    Robot.Txt And RSS

    sandeeppandey, Jun 2, 2011
    Replies:
    4
    Views:
    901
    WishBone
    Jun 7, 2011
  31. prclicks

    Create robots.txt File, Need Your Suggestion

    prclicks, Jun 3, 2011
    Replies:
    0
    Views:
    2,713
    prclicks
    Jun 3, 2011
  32. AlwaysThinking

    How to stop Gugl from indexing https urls as duplicates in wordpress since adding SSL

    AlwaysThinking, May 9, 2011
    Replies:
    4
    Views:
    3,683
    thejimy
    May 29, 2011
  33. premierfabio

    Robots Problem

    premierfabio, May 23, 2011
    Replies:
    6
    Views:
    3,132
    RamCity
    May 24, 2011
  34. mlao

    Disallow Sub-domain From Robots.txt ???

    mlao, May 7, 2011
    Replies:
    8
    Views:
    6,366
    manish.chauhan
    May 24, 2011
  35. egreenr2

    robots.txtx support

    egreenr2, May 12, 2011
    Replies:
    1
    Views:
    2,561
    manish.chauhan
    May 18, 2011
  36. guzelonline

    restrict co.cc hack with robots.txt

    guzelonline, May 9, 2011
    Replies:
    1
    Views:
    3,120
    manish.chauhan
    May 18, 2011
  37. vnpatra2

    robots.txt help

    vnpatra2, May 6, 2011
    Replies:
    4
    Views:
    2,747
    manish.chauhan
    May 18, 2011
  38. frustik

    what disallow or allow in a wp blog

    frustik, May 17, 2011
    Replies:
    1
    Views:
    2,474
    manish.chauhan
    May 18, 2011
  39. style fashion

    Best Robots.txt For Wordpress Blog ....

    style fashion, Oct 1, 2009
    Replies:
    13
    Views:
    10,693
    mohisha
    May 10, 2011
  40. covetousrat

    Can someone check my Robot.txt My site is still not indexed yet.

    covetousrat, Mar 27, 2011
    Replies:
    11
    Views:
    4,328
    us2006
    May 5, 2011
  41. dkv

    Some posts Restricted by robots.txt in Google webmaster tool

    dkv, Mar 5, 2011
    Replies:
    3
    Views:
    3,161
    manish.chauhan
    May 4, 2011
  42. shakac

    google maybe banneding my account Help me in robots.txt file

    shakac, Mar 9, 2011
    Replies:
    2
    Views:
    2,820
    manish.chauhan
    May 4, 2011
  43. besbesmany

    please help problem blogger restricted all my posts

    besbesmany, Feb 24, 2011
    Replies:
    2
    Views:
    2,834
    manish.chauhan
    May 4, 2011
  44. rajeev_seo

    Robots.tst generate

    rajeev_seo, Apr 17, 2011
    Replies:
    1
    Views:
    2,452
    manish.chauhan
    May 4, 2011
  45. harryoxford

    Robots.txt

    harryoxford, Feb 10, 2011
    Replies:
    14
    Views:
    3,780
    manish.chauhan
    May 4, 2011
  46. allanswagger

    Robots.txt

    allanswagger, Sep 2, 2010
    Replies:
    17
    Views:
    4,221
    manish.chauhan
    May 4, 2011
  47. clive2

    robots.txt explanation need?

    clive2, Apr 25, 2011
    Replies:
    2
    Views:
    2,881
    manish.chauhan
    May 4, 2011
  48. tb1234

    unknown bots crawling? how are they and what do they want?

    tb1234, Jun 7, 2006
    Replies:
    4
    Views:
    3,607
    manish.chauhan
    May 3, 2011
  49. gagan.kr90

    What is Robot.txt File?

    gagan.kr90, Apr 25, 2011
    Replies:
    3
    Views:
    2,372
    manish.chauhan
    May 3, 2011
  50. sarahturned

    Help Understanding Robots.txt basic commands

    sarahturned, Dec 30, 2010
    Replies:
    5
    Views:
    3,306
    brad.smith4321
    Apr 19, 2011
Showing threads 201 to 250 of 911

Thread Display Options

Loading...
Page 5 of 19