Social Buttons
  1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 1 of 18
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. shoprat

    Problem with file

    shoprat, Sep 23, 2016 at 8:24 AM
    Replies:
    2
    Views:
    90
    JeffMichaels
    Sep 23, 2016 at 12:07 PM
  2. Noobnl

    Disallow google to index specific tekst with Robot.txt on webpage?

    Noobnl, Apr 10, 2016
    Replies:
    4
    Views:
    1,110
    Furquan Ahmed
    Aug 25, 2016
  3. Official Gates

    What is the correct way to write my “robots.txt” file?

    Official Gates, Aug 16, 2016
    Replies:
    0
    Views:
    201
    Official Gates
    Aug 16, 2016
  4. PaRaDiGm13

    Robots.txt Question

    PaRaDiGm13, Mar 16, 2016
    Replies:
    3
    Views:
    1,540
    Alex D
    Aug 10, 2016
  5. sensor88

    Addon Domains Showing With Main Domain In URL

    sensor88, Aug 9, 2016
    Replies:
    0
    Views:
    168
    sensor88
    Aug 9, 2016
  6. ketanseo23

    Robots.txt Issues

    ketanseo23, Apr 14, 2016
    Replies:
    1
    Views:
    1,047
    MayurKawale
    Aug 4, 2016
  7. irfansyah

    Problem mobile friendly test

    irfansyah, Jun 23, 2016
    Replies:
    8
    Views:
    534
    Host Little
    Jul 14, 2016
  8. soseooooo

    Block a folder except some URL's

    soseooooo, Mar 9, 2016
    Replies:
    1
    Views:
    1,449
    MayurKawale
    Jun 29, 2016
  9. Angelabailey

    Robots.txt

    Angelabailey, Nov 15, 2013
    Replies:
    15
    Views:
    13,376
    Panda123
    Mar 27, 2016
  10. Xfactormaster

    Robots.txt has 'Disallow: /forums/-/' line, what is the meaning

    Xfactormaster, Mar 1, 2016
    Replies:
    2
    Views:
    1,792
    stilliard
    Mar 7, 2016
  11. Kushal Biswas

    Robots.txt Help to Block a Particular File Name

    Kushal Biswas, Feb 3, 2016
    Replies:
    1
    Views:
    2,200
    Anubhav-soin
    Feb 25, 2016
  12. Jeffr2014

    Do you block Majestic bots?

    Jeffr2014, Apr 9, 2015
    Replies:
    4
    Views:
    8,475
    neteater
    Feb 20, 2016
  13. Harris96

    Robots.txt error

    Harris96, Jan 24, 2016
    Replies:
    1
    Views:
    2,347
    Nigel Lew
    Jan 24, 2016
  14. anna30

    How to add robots.txt file in Magento

    anna30, Oct 16, 2015
    Replies:
    1
    Views:
    4,282
    paco78
    Oct 16, 2015
  15. gree124

    robots.txt please check

    gree124, Nov 27, 2014
    Replies:
    7
    Views:
    6,688
    rajendra1682
    Oct 6, 2015
  16. MoneyPlanters

    Why cant Google access my robots.txt ?????

    MoneyPlanters, Sep 27, 2015
    Replies:
    1
    Views:
    3,279
    Mit Thakkar
    Sep 27, 2015
  17. web_master

    i can't stop Baidu, pls help!!!

    web_master, Dec 27, 2011
    Replies:
    7
    Views:
    8,450
    RoundShots
    Sep 11, 2015
  18. MarcL

    Fix to ban google search from Cloaking and/or sneaky redirects

    MarcL, Jul 30, 2015
    Replies:
    4
    Views:
    5,787
    MarcL
    Aug 3, 2015
  19. full_time_writer

    Robot text and partial fetch issues - need suggestions

    full_time_writer, Jul 28, 2015
    Replies:
    0
    Views:
    4,554
    full_time_writer
    Jul 28, 2015
  20. Marker1982

    Very Strange Problem

    Marker1982, Jul 21, 2015
    Replies:
    3
    Views:
    4,788
    Jacob.k
    Jul 25, 2015
  21. Prasseo

    How to create a robots file for PHP webpage to crawl

    Prasseo, Jan 12, 2015
    Replies:
    3
    Views:
    6,464
    abinjoy
    Jul 14, 2015
  22. danieloffice

    how to block all other BOT except Google, MSN, Yahoo

    danieloffice, Jan 9, 2008 ... 2
    Replies:
    23
    Views:
    40,483
    moneyroad
    Jul 10, 2015
  23. myfastrupee

    Robot.txt file not found on my blog

    myfastrupee, Jun 21, 2013
    Replies:
    4
    Views:
    8,783
    meethi jain
    Jun 3, 2015
  24. scott2342

    How do I unblock my robots.txt

    scott2342, Nov 24, 2014
    Replies:
    3
    Views:
    6,686
    meethi jain
    Jun 3, 2015
  25. Minesh Parikh

    Blocked Url's by robots.txt in google webmaster tool how to fix it?

    Minesh Parikh, Apr 6, 2015
    Replies:
    0
    Views:
    5,528
    Minesh Parikh
    Apr 6, 2015
  26. enikram

    How to control when a bot(yahoo,google) can crawl a website

    enikram, Mar 18, 2015
    Replies:
    4
    Views:
    6,070
    Nigel Lew
    Mar 18, 2015
  27. Steviebone

    Robots.txt code to disallow by default but allow domain name

    Steviebone, Mar 4, 2015
    Replies:
    0
    Views:
    6,482
    Steviebone
    Mar 4, 2015
  28. Mrinal_Roy

    Robots.txt is blocking my labels

    Mrinal_Roy, Feb 5, 2015
    Replies:
    1
    Views:
    6,955
    vgopalsujith
    Feb 5, 2015
  29. Prasseo

    Will robots file help to cache the content in the web page

    Prasseo, Jan 12, 2015
    Replies:
    1
    Views:
    6,563
    Sycrid
    Jan 18, 2015
  30. wayne_hoton

    How to create robots.txt for bookmarking website. Is it necessary?

    wayne_hoton, Jan 11, 2015
    Replies:
    1
    Views:
    6,562
    wayne_hoton
    Jan 14, 2015
  31. Jaden Alvin

    Webmaster Blocked URL's Understanding Issue

    Jaden Alvin, Jul 9, 2014
    Replies:
    19
    Views:
    14,214
    Jaden Alvin
    Nov 24, 2014
  32. Olvia

    can give an example of a good robots.txt to wordpress

    Olvia, Nov 13, 2014
    Replies:
    0
    Views:
    6,500
    Olvia
    Nov 13, 2014
  33. aleale

    Is my robots.txt working fine?

    aleale, Sep 10, 2014
    Replies:
    2
    Views:
    8,242
    maddenitrous
    Oct 3, 2014
  34. gvinaykumar

    How to block a specific folder in robots.txt file?

    gvinaykumar, Jul 17, 2014
    Replies:
    0
    Views:
    7,549
    gvinaykumar
    Jul 17, 2014
  35. linkladdy

    Error - URLs restricted by robots.txt errors How Fix IT

    linkladdy, Jul 2, 2014
    Replies:
    1
    Views:
    7,787
    evildon
    Jul 2, 2014
  36. David Lowe

    Can I add specific urls to robots.txt to stop google indexing them?

    David Lowe, Jun 25, 2013
    Replies:
    2
    Views:
    8,587
    BabyDoll123
    Jun 19, 2014
  37. cathandrew

    How to clear these errors in robots.txt?

    cathandrew, Feb 25, 2014
    Replies:
    1
    Views:
    7,702
    BabyDoll123
    Jun 19, 2014
  38. mjfnd

    how to dissalow an image?

    mjfnd, Apr 29, 2014
    Replies:
    1
    Views:
    7,592
    BabyDoll123
    Jun 19, 2014
  39. BabyDoll123

    Magento Robots.txt file

    BabyDoll123, Jun 12, 2014
    Replies:
    3
    Views:
    8,348
    Nigel Lew
    Jun 14, 2014
  40. imaginexhacked

    Blocking auto generated pages with robots.txt

    imaginexhacked, Jun 9, 2014
    Replies:
    1
    Views:
    7,627
    O-D-T
    Jun 9, 2014
  41. jiangnancun

    How can i find those website in which robots.txt disallow google spider?

    jiangnancun, May 13, 2014
    Replies:
    2
    Views:
    8,100
    neroux
    May 13, 2014
  42. James101

    What is robot.txt file?

    James101, Apr 1, 2014
    Replies:
    4
    Views:
    8,531
    John Dave
    Apr 2, 2014
  43. garry420

    I have at-last found site that creates xml file of https sites

    garry420, Mar 8, 2014
    Replies:
    0
    Views:
    7,405
    garry420
    Mar 8, 2014
  44. bright776

    how to disallow a folder from being indexed

    bright776, Feb 1, 2014
    Replies:
    3
    Views:
    8,110
    Ehteshamshaikh
    Mar 7, 2014
  45. cathandrew

    Is it necessary to specify sitemap.xml in robots.txt file

    cathandrew, Feb 25, 2014
    Replies:
    1
    Views:
    8,037
    Ehteshamshaikh
    Feb 27, 2014
  46. manish_khanna

    Things you should be doing to block bad BOTS.

    manish_khanna, Feb 13, 2014
    Replies:
    0
    Views:
    7,738
    manish_khanna
    Feb 13, 2014
  47. SebastienD

    Dissalow folder with the same name as allowed folder

    SebastienD, Jan 15, 2014
    Replies:
    2
    Views:
    8,651
    Nette
    Feb 2, 2014
  48. vatu

    Robots.txt File Help

    vatu, Jan 15, 2014
    Replies:
    8
    Views:
    7,820
    alsgroup
    Jan 23, 2014
  49. Sunil Keshari

    Robots.txt code

    Sunil Keshari, Jan 9, 2014
    Replies:
    3
    Views:
    9,343
    ashokdhan
    Jan 14, 2014
  50. Vel

    I need help on how to Noindex categories, tags, author, archive

    Vel, Jul 25, 2013
    Replies:
    6
    Views:
    8,148
    tymsh
    Dec 9, 2013
Showing threads 1 to 50 of 888

Thread Display Options

Loading...
Page 1 of 18