1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

WebmasterWorld Out Of Google

Discussion in 'General Chat' started by pachecus, Nov 23, 2005.

0
Page 1 of 8
  1. #1
    pachecus, Nov 23, 2005 IP
    Dekker likes this.
  2. ozami

    ozami Peon

    Messages:
    89
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #2
    wow thats quite a bold move. It would worry me that the forums would die down though as no new people would be comming through the engines.

    Still - it takes guts to ban all the engines from your site.
     
    ozami, Nov 23, 2005 IP
  3. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,333
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #3
    I'm sure they must know what they are doing, but the good I see it in it for them doesn't outweigh the bad. I'm (personally) too stupid to see the overall good in it I guess.
     
    digitalpoint, Nov 23, 2005 IP
  4. The Webmaster

    The Webmaster IdeasOfOne

    Messages:
    9,516
    Likes Received:
    718
    Best Answers:
    0
    Trophy Points:
    360
    #4
    That is strange..very strange.
    But why google has done it?? :confused:
     
    The Webmaster, Nov 23, 2005 IP
  5. torunforever

    torunforever Peon

    Messages:
    414
    Likes Received:
    36
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I found WMW via search engine results, and probably wouldn't have known about it otherwise. I consider myself web savvy, but sometimes I just miss out on websites, no matter how popular they are.

    Brett set the WMW robots.txt to disallow all robots. Read the link pachecus posted.
     
    torunforever, Nov 23, 2005 IP
  6. ferret77

    ferret77 Heretic

    Messages:
    5,276
    Likes Received:
    230
    Best Answers:
    0
    Trophy Points:
    0
    #6
    its there a way to just block most "rogue spiders"

    it seems sort of extreme

    and won't most "rogue spiders" ingnore the robots

    I have site rippers and email havesters that don't give a shit about robots.txt
     
    ferret77, Nov 23, 2005 IP
  7. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,333
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #7
    Of course you could... One way would be to require a certain cookie that's generated with an md5 hashing function in JavaScript. Then if the spider doesn't do JavaScript and cookies, you could just block them. Would of course block some users that don't have cookies and JavaScript enabled, but that solution seems better to me. {shrug}
     
    digitalpoint, Nov 23, 2005 IP
  8. forumbulge

    forumbulge Guest

    Messages:
    483
    Likes Received:
    24
    Best Answers:
    0
    Trophy Points:
    0
    #8
    No loss.. Im not a fan of that site anyway. Anyone that wants to go there will have it bookmarked.
    I dont really see how it will affect them.
     
    forumbulge, Nov 23, 2005 IP
  9. Nintendo

    Nintendo ♬ King of da Wackos ♬

    Messages:
    12,890
    Likes Received:
    1,064
    Best Answers:
    0
    Trophy Points:
    430
    #9
    Because Brett Tabke wanted to be banned!!!!! :D

    Well, that didn't take long. I wrote on Monday about how WebmasterWorld head Brett Tabke decided to ban all search spiders including those from the major search engines in an effort to combat bandwidth loss and server sluggishness due to rogue spiders. Brett figured he had about 60 days until he'd see pages get dropped. It took two.

    I say they get listed again causing the site to get slow as a slug and then every one from WMW comes here!! :D

    er, didn't they just 'upgrade' the servers!!!! :D
     
    Nintendo, Nov 23, 2005 IP
  10. INV

    INV Peon

    Messages:
    1,686
    Likes Received:
    101
    Best Answers:
    0
    Trophy Points:
    0
    #10
    If anybody checked WMW They would see

    <META NAME="GOOGLEBOT" CONTENT="NOARCHIVE">
    <META NAME="ROBOTS" CONTENT="NOARCHIVE">
    :D I still dont get this move really.
     
    INV, Nov 23, 2005 IP
  11. lorien1973

    lorien1973 Notable Member

    Messages:
    12,206
    Likes Received:
    601
    Best Answers:
    0
    Trophy Points:
    260
    #11
    blog says it all to me. The reasoning is flawed, but at least a site is finally removed from google by choice.

    Seeing as how WMW is so close to google, I bet there was a little manual removal there, just to speed up this experiment - which I assume it is.
     
    lorien1973, Nov 23, 2005 IP
  12. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,333
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #12
    digitalpoint, Nov 23, 2005 IP
  13. Crazy_Rob

    Crazy_Rob I seen't it!

    Messages:
    13,157
    Likes Received:
    1,366
    Best Answers:
    0
    Trophy Points:
    360
    #13
    Google is the only good way to search WMW.

    How strange!
     
    Crazy_Rob, Nov 23, 2005 IP
  14. Dekker

    Dekker Peon

    Messages:
    4,185
    Likes Received:
    287
    Best Answers:
    0
    Trophy Points:
    0
    #14
    i still can't grasp the reason for this..............
     
    Dekker, Nov 23, 2005 IP
  15. ronmojohny

    ronmojohny Active Member

    Messages:
    729
    Likes Received:
    20
    Best Answers:
    0
    Trophy Points:
    68
    #15
    copy this code to a local html file.. the yahoo search still works for WW

    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
    <html>
    <head>
    <title>
    Untitled
    </title>
    </head>
    <body>
    <table>
    <tr>
    <td>
    <form action=http://www.google.com/search target=_blank>
    <input TYPE="text" name="q">
    <input type="submit" value="Google">
    <input type="hidden" name="sitesearch" value="webmasterworld.com">
    </form>
    </td>
    <td>
    <form action=http://search.yahoo.com/search target=_blank>
    <input TYPE="text" name="p">
    <input type="submit" value="Yahoo">
    <input type="hidden" name="vs" value="webmasterworld.com">
    </form>
    </td>
    <td>
    <form action=http://search.msn.com/results.aspx target=_blank>
    <input TYPE="text" name="q">
    <input type="submit" value="MSN">
    <input type="hidden" name="q1" value="site:webmasterworld.com">
    </form>
    </td>
    </tr>
    </table>
    </body>
    </html>


    -RonMo
     
    ronmojohny, Nov 23, 2005 IP
  16. aeiouy

    aeiouy Peon

    Messages:
    2,876
    Likes Received:
    275
    Best Answers:
    0
    Trophy Points:
    0
    #16
    Came across an interesting thread at WebMaster World. They are taking some extra-ordinary measures to try and reduce the number of rogue bots that are accessing their site. They are going to eliminate all robots from accessing the site completely.

    Apparently while at PubCon they had 12,000,000 pageviews attributable to Rogue Bots.

    This means none of the search engines will have their site or pages indexed.

    The question I pose is how many of you feel this is a problem for some or all of your sites? Are you seeing a substantial number of activity from bots that provide no real benefit and generate no real traffic? If so, what is your strategy for dealing with it?

    The WMW thread is here:

    http://www.webmasterworld.com/forum9/9593.htm
     
    aeiouy, Nov 23, 2005 IP
  17. Dekker

    Dekker Peon

    Messages:
    4,185
    Likes Received:
    287
    Best Answers:
    0
    Trophy Points:
    0
    #17
    Well, I'm not sure how to monitor bots completely, but from my AWstats, I only have about 6 different bots visit my site.
     
    Dekker, Nov 23, 2005 IP
  18. webdesign2005

    webdesign2005 Guest

    Messages:
    26
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #18
    I use .htaccess to block them by user agent and enviromental variables. They get a 403 page.
     
    webdesign2005, Nov 23, 2005 IP
  19. aeiouy

    aeiouy Peon

    Messages:
    2,876
    Likes Received:
    275
    Best Answers:
    0
    Trophy Points:
    0
    #19
    aeiouy, Nov 23, 2005 IP
  20. Crazy_Rob

    Crazy_Rob I seen't it!

    Messages:
    13,157
    Likes Received:
    1,366
    Best Answers:
    0
    Trophy Points:
    360
    #20
    Crazy_Rob, Nov 23, 2005 IP
Page 1 of 8