robots.txt

Find help with setting up your robots.txt file on your web site.

Page 13 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. AlfaNet

    Help with using wildcard for dynamic URL

    AlfaNet, Apr 29, 2008
    Replies:
    3
    Views:
    1,553
    manish.chauhan
    Jun 13, 2008
  2. seoworld1234

    How to create robots.txt

    seoworld1234, May 27, 2008
    Replies:
    4
    Views:
    1,357
    Dayton Internet Marketing
    Jun 12, 2008
  3. dealtoplay

    Should I even have a robots.txt?

    dealtoplay, Jun 3, 2008
    Replies:
    6
    Views:
    1,437
    Dayton Internet Marketing
    Jun 12, 2008
  4. nonflasher

    How to put robots.txt

    nonflasher, Feb 9, 2006
    Replies:
    13
    Views:
    3,666
    Hunnigs
    Jun 6, 2008
  5. SuperNovaArcade

    All About Robots.txt

    SuperNovaArcade, Jun 4, 2008
    Replies:
    0
    Views:
    1,641
    SuperNovaArcade
    Jun 4, 2008
  6. lim (x² - 5x³) = -∞

    robots.txt - How to ban everyone except Google

    lim (x² - 5x³) = -∞, Mar 27, 2008 ... 2
    Replies:
    21
    Views:
    4,638
    dermax
    Jun 2, 2008
  7. gunakesh

    how can i get the list of bad robots??????

    gunakesh, May 13, 2008
    Replies:
    6
    Views:
    1,434
    manish.chauhan
    May 30, 2008
  8. domainer_10

    HTTPS getting indexed instead of HTTP on wordpress

    domainer_10, May 26, 2008
    Replies:
    1
    Views:
    991
    domainer_10
    May 26, 2008
  9. m021478

    Question on using robots.txt to block Googlebot

    m021478, May 20, 2008
    Replies:
    8
    Views:
    1,960
    manish.chauhan
    May 26, 2008
  10. vinug

    Robots.txt file vs robots tag

    vinug, May 21, 2008
    Replies:
    2
    Views:
    1,612
    manish.chauhan
    May 21, 2008
  11. ManchesterTech

    Robots.txt...what do I want from it?...

    ManchesterTech, May 20, 2008
    Replies:
    2
    Views:
    1,106
    manish.chauhan
    May 20, 2008
  12. risoknop

    Review my robots.txt

    risoknop, May 20, 2008
    Replies:
    6
    Views:
    1,315
    manish.chauhan
    May 20, 2008
  13. SeoVeteran33

    Wicked Dump Robots Question

    SeoVeteran33, May 17, 2008
    Replies:
    2
    Views:
    1,178
    manish.chauhan
    May 20, 2008
  14. nitu

    Use of robots.txt

    nitu, May 3, 2008
    Replies:
    3
    Views:
    1,219
    fifty-one
    May 17, 2008
  15. mnymkr

    robot.txt wildcards are they allowed?

    mnymkr, May 4, 2008
    Replies:
    2
    Views:
    3,180
    Trusted Writer
    May 13, 2008
  16. akhilesh243

    Ideal Robots.txt file for wordpress website

    akhilesh243, Nov 19, 2007
    Replies:
    8
    Views:
    6,206
    arunsubru
    May 12, 2008
  17. bryanw

    Help needed - ban robots from crawling redirect script

    bryanw, May 7, 2008
    Replies:
    9
    Views:
    1,439
    Trusted Writer
    May 7, 2008
  18. Molon Lave

    Bandwidth problem due to bots

    Molon Lave, Apr 10, 2008
    Replies:
    4
    Views:
    2,098
    manish.chauhan
    May 7, 2008
  19. lateuk

    Blocking a forum with robots.txt

    lateuk, May 1, 2008
    Replies:
    1
    Views:
    1,147
    manish.chauhan
    May 1, 2008
  20. smoked1

    Help with robots.txt and dynamic URLs

    smoked1, Apr 25, 2008
    Replies:
    3
    Views:
    1,398
    smoked1
    Apr 28, 2008
  21. abcdefGARY

    quick question about robots.txt

    abcdefGARY, Feb 9, 2008
    Replies:
    9
    Views:
    1,771
    manish.chauhan
    Apr 20, 2008
  22. TheDutchMan

    Automatic no-follow???

    TheDutchMan, Apr 13, 2008
    Replies:
    6
    Views:
    1,370
    henery008
    Apr 17, 2008
  23. IndianMonk

    Need help In my robots.txt

    IndianMonk, Apr 10, 2008
    Replies:
    4
    Views:
    1,183
    manish.chauhan
    Apr 13, 2008
  24. sudip_dg77

    Robot.txt?

    sudip_dg77, Apr 4, 2008
    Replies:
    6
    Views:
    1,442
    SubmitShop
    Apr 10, 2008
  25. movvadinesh

    What is robot.txt and how to add this to blog

    movvadinesh, Feb 2, 2008
    Replies:
    4
    Views:
    2,223
    manish.chauhan
    Apr 9, 2008
  26. Maciej Ziemczonek

    robots.txt and IP addresses

    Maciej Ziemczonek, Apr 8, 2008
    Replies:
    2
    Views:
    1,280
    Maciej Ziemczonek
    Apr 8, 2008
  27. gogo2005

    Error on my site Robots could block them?

    gogo2005, Dec 19, 2007
    Replies:
    4
    Views:
    2,075
    manish.chauhan
    Apr 7, 2008
  28. kie

    Prohibit SE to crawl subdomain

    kie, Jan 23, 2008
    Replies:
    7
    Views:
    1,654
    manish.chauhan
    Apr 7, 2008
  29. Giorgi

    how to completly block engine from crawling my site?

    Giorgi, Jan 16, 2008
    Replies:
    15
    Views:
    2,196
    manish.chauhan
    Apr 7, 2008
  30. Malmsteen

    Quick Question on a robots.txt file - help!?

    Malmsteen, Feb 29, 2008
    Replies:
    3
    Views:
    1,262
    manish.chauhan
    Apr 6, 2008
  31. angellica2017

    Dangerous Crawler and User Agent! Must Read!

    angellica2017, Sep 19, 2006 ... 2
    Replies:
    22
    Views:
    11,865
    manish.chauhan
    Apr 6, 2008
  32. Death4you

    [.htaccess and bad bots list] robots.txt of course not works

    Death4you, Mar 22, 2008
    Replies:
    4
    Views:
    2,089
    manish.chauhan
    Apr 6, 2008
  33. ialwaysforget

    robot.txt help

    ialwaysforget, Mar 13, 2008
    Replies:
    7
    Views:
    1,557
    manish.chauhan
    Apr 6, 2008
  34. HomeRun

    How can I block / disallow this spider?

    HomeRun, Mar 20, 2008
    Replies:
    2
    Views:
    1,167
    manish.chauhan
    Apr 5, 2008
  35. crazyryan

    Stop ANY search engine indexing my site

    crazyryan, Feb 2, 2008
    Replies:
    7
    Views:
    2,191
    manish.chauhan
    Apr 4, 2008
  36. BigJuicy

    I'm a NOOB google wont spider me =(

    BigJuicy, Dec 11, 2007
    Replies:
    5
    Views:
    1,507
    Dreads
    Apr 4, 2008
  37. HomeRun

    Is this a valid robots.txt file?

    HomeRun, Mar 17, 2008
    Replies:
    6
    Views:
    1,159
    manish.chauhan
    Apr 4, 2008
  38. templates

    ? abouta bot

    templates, Feb 23, 2008
    Replies:
    2
    Views:
    1,382
    manish.chauhan
    Apr 4, 2008
  39. TheSites

    How to block out sessions from being indexed

    TheSites, Mar 29, 2008
    Replies:
    1
    Views:
    1,041
    manish.chauhan
    Apr 4, 2008
  40. coolfriend

    Method to edit "robots.txt" in blogspot

    coolfriend, Mar 25, 2008
    Replies:
    3
    Views:
    2,302
    manish.chauhan
    Apr 4, 2008
  41. artboy

    How to get robots.txt re-crawled by google/yahoo, etc?

    artboy, Mar 10, 2008
    Replies:
    7
    Views:
    1,599
    manish.chauhan
    Apr 4, 2008
  42. HomeRun

    Is this a good robots.txt for vbulletin?

    HomeRun, Mar 25, 2008
    Replies:
    2
    Views:
    1,324
    FightRice
    Mar 31, 2008
  43. warley

    Help Required

    warley, Mar 11, 2008
    Replies:
    2
    Views:
    865
    warley
    Mar 11, 2008
  44. jtothek3030

    How do i stop Internet Shinchakubin?

    jtothek3030, May 11, 2007
    Replies:
    13
    Views:
    4,607
    scoopy82
    Mar 10, 2008
  45. mytechguru

    URLs restricted by robots.txt

    mytechguru, Mar 6, 2008
    Replies:
    4
    Views:
    1,726
    LittleJonSupportSite
    Mar 7, 2008
  46. coletrickle

    Robots.txt not visible via HTTP for wordpress blog...

    coletrickle, Feb 11, 2008
    Replies:
    4
    Views:
    1,299
    coletrickle
    Feb 18, 2008
  47. Jinx3000

    First time setting up robots.txt

    Jinx3000, Jan 14, 2008
    Replies:
    7
    Views:
    1,438
    catanich
    Feb 2, 2008
  48. Jon12345

    What does this robots.txt text mean?

    Jon12345, Jan 8, 2008
    Replies:
    7
    Views:
    1,882
    catanich
    Feb 2, 2008
  49. frio

    Disallow

    frio, Dec 20, 2007
    Replies:
    7
    Views:
    2,260
    catanich
    Feb 2, 2008
  50. nshadab

    Help required for ROBOTS.txt?

    nshadab, Jan 22, 2008
    Replies:
    4
    Views:
    1,058
    catanich
    Feb 2, 2008
Showing threads 601 to 650 of 911

Thread Display Options

Loading...
Page 13 of 19