Social Buttons
  1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 1 of 18
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. infopages

    Help with Robots.txt

    infopages, Sep 29, 2017
    Replies:
    1
    Views:
    522
    qwikad.com
    Sep 29, 2017
  2. Divvy

    Help with robots.txt file

    Divvy, Jun 14, 2017
    Replies:
    4
    Views:
    949
    Arun Kumar Soni
    Sep 24, 2017
  3. rants1990

    Vbulletin 4 robots txt?

    rants1990, Sep 19, 2017
    Replies:
    3
    Views:
    268
    sarahk
    Sep 19, 2017
  4. pradip200

    Robots.txt for SMF Forum

    pradip200, Mar 23, 2013
    Replies:
    4
    Views:
    12,189
    Svoboda
    Jun 30, 2017
  5. seoworld1234

    How to check robots.txt on site?

    seoworld1234, Apr 21, 2008 ... 2
    Replies:
    29
    Views:
    60,876
    Stephen Thomas
    May 19, 2017
  6. wartes

    Robot.txt for Google (error)

    wartes, Apr 10, 2017
    Replies:
    6
    Views:
    3,197
    Md.Saddam Hossain
    May 7, 2017
  7. Mubashir Ahmad

    Robot.txt or remove the page

    Mubashir Ahmad, Mar 21, 2017
    Replies:
    6
    Views:
    1,893
    Tutorials Feed
    Apr 19, 2017
  8. PaRaDiGm13

    Robots.txt Question

    PaRaDiGm13, Mar 16, 2016
    Replies:
    4
    Views:
    2,698
    Cado Magenge
    Feb 28, 2017
  9. smokey99

    Noindex and robots disallow

    smokey99, Dec 22, 2016
    Replies:
    2
    Views:
    1,157
    PoPSiCLe
    Feb 24, 2017
  10. hmansfield

    List of Bad Bots to disallow?

    hmansfield, May 20, 2007
    Replies:
    14
    Views:
    14,689
    Sergio Minozzi
    Feb 22, 2017
  11. Abdullah adil

    robot.txt

    Abdullah adil, Feb 11, 2017
    Replies:
    6
    Views:
    4,170
    sarahk
    Feb 16, 2017
  12. kabali

    importance of robots.txt

    kabali, Sep 27, 2016
    Replies:
    3
    Views:
    1,244
    highphiverr
    Feb 4, 2017
  13. Boxwell

    Can someone tell me what is wrong with my robots.txt file?

    Boxwell, Nov 24, 2016
    Replies:
    2
    Views:
    1,466
    GameO.Over
    Nov 24, 2016
  14. dinrock

    I have default robots.txt and asking how to customize it

    dinrock, Oct 13, 2016
    Replies:
    0
    Views:
    1,268
    dinrock
    Oct 13, 2016
  15. shoprat

    Problem with file

    shoprat, Sep 23, 2016
    Replies:
    2
    Views:
    819
    JeffMichaels
    Sep 23, 2016
  16. Noobnl

    Disallow google to index specific tekst with Robot.txt on webpage?

    Noobnl, Apr 10, 2016
    Replies:
    4
    Views:
    1,865
    Furquan Ahmed
    Aug 25, 2016
  17. Official Gates

    What is the correct way to write my “robots.txt” file?

    Official Gates, Aug 16, 2016
    Replies:
    0
    Views:
    1,063
    Official Gates
    Aug 16, 2016
  18. sensor88

    Addon Domains Showing With Main Domain In URL

    sensor88, Aug 9, 2016
    Replies:
    0
    Views:
    998
    sensor88
    Aug 9, 2016
  19. ketanseo23

    Robots.txt Issues

    ketanseo23, Apr 14, 2016
    Replies:
    1
    Views:
    1,807
    MayurKawale
    Aug 4, 2016
  20. irfansyah

    Problem mobile friendly test

    irfansyah, Jun 23, 2016
    Replies:
    8
    Views:
    1,354
    Host Little
    Jul 14, 2016
  21. soseooooo

    Block a folder except some URL's

    soseooooo, Mar 9, 2016
    Replies:
    1
    Views:
    2,105
    MayurKawale
    Jun 29, 2016
  22. Angelabailey

    Robots.txt

    Angelabailey, Nov 15, 2013
    Replies:
    15
    Views:
    14,317
    Panda123
    Mar 27, 2016
  23. Xfactormaster

    Robots.txt has 'Disallow: /forums/-/' line, what is the meaning

    Xfactormaster, Mar 1, 2016
    Replies:
    2
    Views:
    2,583
    stilliard
    Mar 7, 2016
  24. Kushal Biswas

    Robots.txt Help to Block a Particular File Name

    Kushal Biswas, Feb 3, 2016
    Replies:
    1
    Views:
    6,714
    Anubhav-soin
    Feb 25, 2016
  25. Jeffr2014

    Do you block Majestic bots?

    Jeffr2014, Apr 9, 2015
    Replies:
    4
    Views:
    9,962
    neteater
    Feb 20, 2016
  26. Harris96

    Robots.txt error

    Harris96, Jan 24, 2016
    Replies:
    1
    Views:
    3,078
    Nigel Lew
    Jan 24, 2016
  27. anna30

    How to add robots.txt file in Magento

    anna30, Oct 16, 2015
    Replies:
    1
    Views:
    4,955
    paco78
    Oct 16, 2015
  28. gree124

    robots.txt please check

    gree124, Nov 27, 2014
    Replies:
    7
    Views:
    7,315
    rajendra1682
    Oct 6, 2015
  29. MoneyPlanters

    Why cant Google access my robots.txt ?????

    MoneyPlanters, Sep 27, 2015
    Replies:
    1
    Views:
    7,598
    Mit Thakkar
    Sep 27, 2015
  30. web_master

    i can't stop Baidu, pls help!!!

    web_master, Dec 27, 2011
    Replies:
    7
    Views:
    9,402
    RoundShots
    Sep 11, 2015
  31. MarcL

    Fix to ban google search from Cloaking and/or sneaky redirects

    MarcL, Jul 30, 2015
    Replies:
    4
    Views:
    10,384
    MarcL
    Aug 3, 2015
  32. full_time_writer

    Robot text and partial fetch issues - need suggestions

    full_time_writer, Jul 28, 2015
    Replies:
    0
    Views:
    5,205
    full_time_writer
    Jul 28, 2015
  33. Marker1982

    Very Strange Problem

    Marker1982, Jul 21, 2015
    Replies:
    3
    Views:
    9,176
    Jacob.k
    Jul 25, 2015
  34. Prasseo

    How to create a robots file for PHP webpage to crawl

    Prasseo, Jan 12, 2015
    Replies:
    3
    Views:
    7,109
    abinjoy
    Jul 14, 2015
  35. danieloffice

    how to block all other BOT except Google, MSN, Yahoo

    danieloffice, Jan 9, 2008 ... 2
    Replies:
    23
    Views:
    47,816
    moneyroad
    Jul 10, 2015
  36. myfastrupee

    Robot.txt file not found on my blog

    myfastrupee, Jun 21, 2013
    Replies:
    4
    Views:
    9,469
    meethi jain
    Jun 3, 2015
  37. scott2342

    How do I unblock my robots.txt

    scott2342, Nov 24, 2014
    Replies:
    3
    Views:
    7,365
    meethi jain
    Jun 3, 2015
  38. Minesh Parikh

    Blocked Url's by robots.txt in google webmaster tool how to fix it?

    Minesh Parikh, Apr 6, 2015
    Replies:
    0
    Views:
    9,888
    Minesh Parikh
    Apr 6, 2015
  39. enikram

    How to control when a bot(yahoo,google) can crawl a website

    enikram, Mar 18, 2015
    Replies:
    4
    Views:
    10,876
    Nigel Lew
    Mar 18, 2015
  40. Steviebone

    Robots.txt code to disallow by default but allow domain name

    Steviebone, Mar 4, 2015
    Replies:
    0
    Views:
    7,112
    Steviebone
    Mar 4, 2015
  41. Mrinal_Roy

    Robots.txt is blocking my labels

    Mrinal_Roy, Feb 5, 2015
    Replies:
    1
    Views:
    7,795
    vgopalsujith
    Feb 5, 2015
  42. Prasseo

    Will robots file help to cache the content in the web page

    Prasseo, Jan 12, 2015
    Replies:
    1
    Views:
    7,222
    Sycrid
    Jan 18, 2015
  43. wayne_hoton

    How to create robots.txt for bookmarking website. Is it necessary?

    wayne_hoton, Jan 11, 2015
    Replies:
    1
    Views:
    7,190
    wayne_hoton
    Jan 14, 2015
  44. Jaden Alvin

    Webmaster Blocked URL's Understanding Issue

    Jaden Alvin, Jul 9, 2014
    Replies:
    19
    Views:
    14,991
    Jaden Alvin
    Nov 24, 2014
  45. Olvia

    can give an example of a good robots.txt to wordpress

    Olvia, Nov 13, 2014
    Replies:
    0
    Views:
    7,109
    Olvia
    Nov 13, 2014
  46. aleale

    Is my robots.txt working fine?

    aleale, Sep 10, 2014
    Replies:
    2
    Views:
    12,737
    maddenitrous
    Oct 3, 2014
  47. gvinaykumar

    How to block a specific folder in robots.txt file?

    gvinaykumar, Jul 17, 2014
    Replies:
    0
    Views:
    8,217
    gvinaykumar
    Jul 17, 2014
  48. linkladdy

    Error - URLs restricted by robots.txt errors How Fix IT

    linkladdy, Jul 2, 2014
    Replies:
    1
    Views:
    8,539
    evildon
    Jul 2, 2014
  49. David Lowe

    Can I add specific urls to robots.txt to stop google indexing them?

    David Lowe, Jun 25, 2013
    Replies:
    2
    Views:
    9,205
    BabyDoll123
    Jun 19, 2014
  50. cathandrew

    How to clear these errors in robots.txt?

    cathandrew, Feb 25, 2014
    Replies:
    1
    Views:
    8,327
    BabyDoll123
    Jun 19, 2014
Showing threads 1 to 50 of 898

Thread Display Options

Loading...
Page 1 of 18