1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

WebmasterWorld Out Of Google

Discussion in 'General Chat' started by pachecus, Nov 23, 2005.

  1. Dekker

    Dekker Peon

    Messages:
    4,185
    Likes Received:
    286
    Best Answers:
    0
    Trophy Points:
    0
    #61
    was webmasterworld slow prior to this? i went on a few times and it was always fast..
    SEMrush
     
    Dekker, Nov 25, 2005 IP
    SEMrush
  2. aeiouy

    aeiouy Peon

    Messages:
    2,877
    Likes Received:
    275
    Best Answers:
    0
    Trophy Points:
    0
    #62
    Yeah but I think Brett has said that the robots.txt is just one aspect. The real way they are stopping the bots is with cookies etc required for everyone going to the forums.

    I don't that it will hurt WMW that much for a few reasons:

    The way the site worked before was very newbie unfriendly and made it look like you had to pay them money to get access to the information, even when doing a random search in Google.

    They have always been a one-way resource. Since they don't allow linking in posts, it stifles discussion a bit but also keeps traffic from out-flowing. All the while other webmaster resources, including DP, don't have any issue getting links.

    I imagine a search through all the DP posts would turn up more than a handful of links to WMW threads. So they will still end up getting second-hand traffic from other resources.

    I don't know enough about the things involved to make an educated guess myself. My first impression, though, is that there have to be other ways to potentially do this without getting into outright cloaking of pages.

    Apparently the reason why they fell out of google so fast is many people use d the URL Removal tool on them once they posted about the robots.txt change. Google seems very efficient at dropping sites.
     
    aeiouy, Nov 25, 2005 IP
  3. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,283
    Likes Received:
    2,599
    Best Answers:
    460
    Trophy Points:
    710
    Digital Goods:
    29
    #63
    digitalpoint, Nov 25, 2005 IP
  4. aeiouy

    aeiouy Peon

    Messages:
    2,877
    Likes Received:
    275
    Best Answers:
    0
    Trophy Points:
    0
    #64
    That is pretty bad. I think he may have under-estimated the percentage of casual traffic that passes through his site.

    I imagine the same is true here. While there is a large number of regular users, I imagine the core of the traffic is from more casual people. Who probably often first come across such resources via search engines.
     
    aeiouy, Nov 25, 2005 IP
  5. Dekker

    Dekker Peon

    Messages:
    4,185
    Likes Received:
    286
    Best Answers:
    0
    Trophy Points:
    0
    #65
    I found DP by google for something like "how setup apache"
     
    Dekker, Nov 25, 2005 IP
  6. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,283
    Likes Received:
    2,599
    Best Answers:
    460
    Trophy Points:
    710
    Digital Goods:
    29
    #66
    I'm sure they knew the amount of traffic he gets from search engines before they did it (they aren't stupid). See this post.
     
    digitalpoint, Nov 25, 2005 IP
  7. Will.Spencer

    Will.Spencer NetBuilder

    Messages:
    14,789
    Likes Received:
    1,040
    Best Answers:
    0
    Trophy Points:
    375
    #67
    Hmmm... stupid.... hmmm...

    Can we settle on maniacally insane?
     
    Will.Spencer, Nov 25, 2005 IP
    minstrel likes this.
  8. lorien1973

    lorien1973 Notable Member

    Messages:
    12,210
    Likes Received:
    603
    Best Answers:
    0
    Trophy Points:
    260
    #68
    Can you commiserate with him though and understand the reasoning? Do you keep track of how much bandwidth spiders eat up (as a percentage)? Have you ever even -come close- to doing something similar with this forum?
     
    lorien1973, Nov 25, 2005 IP
  9. Will.Spencer

    Will.Spencer NetBuilder

    Messages:
    14,789
    Likes Received:
    1,040
    Best Answers:
    0
    Trophy Points:
    375
    #69
    Will.Spencer, Nov 25, 2005 IP
  10. Dekker

    Dekker Peon

    Messages:
    4,185
    Likes Received:
    286
    Best Answers:
    0
    Trophy Points:
    0
    #70
    Results 11 - 20 of about 2,090,000 for webmasterworld
     
    Dekker, Nov 25, 2005 IP
  11. Dekker

    Dekker Peon

    Messages:
    4,185
    Likes Received:
    286
    Best Answers:
    0
    Trophy Points:
    0
    #71
    so i'm guessing it's not permanent...and they'll get reindexed fast
     
    Dekker, Nov 25, 2005 IP
  12. Will.Spencer

    Will.Spencer NetBuilder

    Messages:
    14,789
    Likes Received:
    1,040
    Best Answers:
    0
    Trophy Points:
    375
    #72
    Unless Google makes a special exception for him, it's a 180 day wait once you use the Google Automatic URL Removal System.
     
    Will.Spencer, Nov 25, 2005 IP
  13. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,283
    Likes Received:
    2,599
    Best Answers:
    460
    Trophy Points:
    710
    Digital Goods:
    29
    #73
    It's not my forum, I'm not going to tell him how to run it.

    It's not much (21.5% of bandwidth in this forum goes to spiders/bots)... msnbot is the worst bandwidth sucker though. This month (so far) 10.25GB went to msnbot, 4.86GB went to Gigabot, 4.11GB to Yahoo Slurp, 2.26GB to Googlebot (then a bunch of unknown bots that were under 1GB). Google definitely is the most efficient bot (maintaining self-control on how much to suck down as well as using 1.1 with compression to reduce bandwidth by about 85%). And no... never even crossed my mind to block bots.
     
    digitalpoint, Nov 25, 2005 IP
  14. nevetS

    nevetS Evolving Dragon

    Messages:
    2,544
    Likes Received:
    211
    Best Answers:
    0
    Trophy Points:
    135
    #74
    Wow. It goes to show that even someone who has been around for years and should know the ins and outs of what to do and what not to do can make a huge mistake every once in a while.

    Maybe it will be good for them by causing a shift in the signal to noise ratio, but any positive affects will be anomalies.

    Why would you run an experiment like that on one of your important sites? I know that as webmasters we do a lot of testing and a lot of playing around with little things, but you'd have to be on no sleep for 3 days and drunk to get a site like that removed from google on a whim.
     
    nevetS, Nov 25, 2005 IP
  15. Will.Spencer

    Will.Spencer NetBuilder

    Messages:
    14,789
    Likes Received:
    1,040
    Best Answers:
    0
    Trophy Points:
    375
    #75
    One word: Anti-depressants.
     
    Will.Spencer, Nov 25, 2005 IP
  16. markhutch

    markhutch Peon

    Messages:
    357
    Likes Received:
    22
    Best Answers:
    0
    Trophy Points:
    0
    #76
    I problem with this strategy is that when regular users drift away there will be no new ones to take their place.
     
    markhutch, Nov 25, 2005 IP
  17. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #77
    How about arrogant? Thinking he owns the SEO forum world and no longer needs Google?
     
    minstrel, Nov 25, 2005 IP
  18. Design Agent

    Design Agent Peon

    Messages:
    3,061
    Likes Received:
    154
    Best Answers:
    0
    Trophy Points:
    0
    #78
    Something is missing in all this.
    WMW has been around so long + BT must have plenty of good advice from talented people. I cant imagine any of this being a mistake - the idea and possible repercussion must have been considered. Risky, but not a mistake, though it may turn out that way.

    Im not tech minded enough to work the details though + I dont care :) Ive never been able to deal with the design/ layout anyway.
     
    Design Agent, Nov 25, 2005 IP
  19. Dekker

    Dekker Peon

    Messages:
    4,185
    Likes Received:
    286
    Best Answers:
    0
    Trophy Points:
    0
    #79
    me neither...I hate how they make you register for most of the threads..and they randomly give you a password so I forgot it and gave up :p

    i'm sure he's thought this through though..
     
    Dekker, Nov 25, 2005 IP
  20. Crazy_Rob

    Crazy_Rob I seen't it!

    Messages:
    13,155
    Likes Received:
    1,366
    Best Answers:
    0
    Trophy Points:
    0
    #80

    Actually, I don't think he has. If you go over there and read the thread, it's clear that he wasn't expecting this.

    WMW has been a great resource over the years and I'm sure it will be "fine".
     
    Crazy_Rob, Nov 25, 2005 IP