1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

WebmasterWorld Out Of Google

Discussion in 'General Chat' started by pachecus, Nov 23, 2005.

  1. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,283
    Likes Received:
    2,599
    Best Answers:
    460
    Trophy Points:
    710
    Digital Goods:
    29
    #41
    Teach them a lesson about *what* though? I actually get the impression it's really about rogue bots... I just think it's not the best way to handle it myself. That's why I'm interested to fast forward this experiment a few months into the future and see what becomes of it.
    SEMrush
     
    digitalpoint, Nov 23, 2005 IP
    SEMrush
  2. markhutch

    markhutch Peon

    Messages:
    357
    Likes Received:
    22
    Best Answers:
    0
    Trophy Points:
    0
    #42
    I believe the bad bot story is a smoke screen myself. Have you ever known a bad bot to obey a robots.txt file? I haven't. Most don't even check for it. We all deal with them everyday in our own way. I certainly am not going to ban good search engines because of the bad. Why would anyone in their right mind think that a bad bot would obey a complete ban robots txt file when they won't obey the one directed right at them in the first place? This whole story is not to be believed. Brett is NOT stupid and he knows that bad bots don't obey robots.txt and never have. I've been at the game since 1996 and there is more here than meets the eye. If you have bad customers, you don't close your whole business down because of the bad one, you find way to eliminate the bad apples and cause an little damage as possible to your good customers.

    Brett did change ISP's recently because of a router crash in Chicago. He may be, for the first time, paying over bandwidth charges which might be catching his attention for the first time. There are honest ways to make money off a good forum that gets this much traffic. However, he might not want to admit that he's scared to death facing these extra bandwidth fees for the first time. Everything we are talking about here is speculation, but what's going on over at WMW isn't speculation, it's real life for folks like me.
     
    markhutch, Nov 23, 2005 IP
    digitalpoint likes this.
  3. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,283
    Likes Received:
    2,599
    Best Answers:
    460
    Trophy Points:
    710
    Digital Goods:
    29
    #43
    Yeah that's true... a rogue bot certainly isn't going to adhere to a robots.txt file directive. In that case it beats the hell out of me... :)
     
    digitalpoint, Nov 23, 2005 IP
  4. markhutch

    markhutch Peon

    Messages:
    357
    Likes Received:
    22
    Best Answers:
    0
    Trophy Points:
    0
    #44
    I still believe that the truth in not the bandwidth question, but something that happened at Pubcon. I heard that Brett personally interview Matt Cutts in Las Vegas during one of their sessions and asked him pointed questions about some of Googles practices including the so called "Sandbox" effect, which Cutts said didn't exist. Most Webmaster do believe that something like a sandbox does exist. I don't think this is the reason, but something happened there that didn't go well for Brett and/or WMW. We may never know, but I have found that I'm spending more time here now in this forum which isn't a bad thing! :)
     
    markhutch, Nov 23, 2005 IP
  5. Crazy_Rob

    Crazy_Rob I seen't it!

    Messages:
    13,155
    Likes Received:
    1,366
    Best Answers:
    0
    Trophy Points:
    0
    #45
    I guess it's an attempt to lower bandwidth usage. Looks like it's been working! ;)

    I don't know- maybe there's more to the story. Brett's been around a long time.
     
    Crazy_Rob, Nov 23, 2005 IP
  6. markhutch

    markhutch Peon

    Messages:
    357
    Likes Received:
    22
    Best Answers:
    0
    Trophy Points:
    0
    #46
    Any how, I'm out of this thread. I said what I had to say and now I'm moving on to other more profitable things. Best of luck, Brett!
     
    markhutch, Nov 23, 2005 IP
  7. latehorn

    latehorn Guest

    Messages:
    4,677
    Likes Received:
    238
    Best Answers:
    0
    Trophy Points:
    0
    #47
    Yea it sucks when SEs eats up bandwidth..

    I wish there was a way to limit the number of spiderings in robots.txt
    etc.. max 100 per SE
     
    latehorn, Nov 23, 2005 IP
  8. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #48
    Another bad move. That's way overpriced. maybe in the 99 cent bargain bin but no one is going to pay $9.95 for that...

    Too long. He's become a tail that thinks he can wag the dog.
     
    minstrel, Nov 23, 2005 IP
  9. lorien1973

    lorien1973 Notable Member

    Messages:
    12,210
    Likes Received:
    603
    Best Answers:
    0
    Trophy Points:
    260
    #49
    Seems like good way to get free publicity, if nothing else, to me. Worked like a charm so far.
     
    lorien1973, Nov 24, 2005 IP
  10. latehorn

    latehorn Guest

    Messages:
    4,677
    Likes Received:
    238
    Best Answers:
    0
    Trophy Points:
    0
    #50
    ^^ haha yeah probably
     
    latehorn, Nov 24, 2005 IP
  11. dvduval

    dvduval Notable Member

    Messages:
    3,365
    Likes Received:
    355
    Best Answers:
    1
    Trophy Points:
    260
    #51
    I'm a little late reading this, but I wanted to also express my shock and dsbelief. Just last month they moved away from Westhost and got a dedicated server. Surely, this new server can handle a few roque bots hitting them, and surely large bandwidth hogs can be properly dealt with. I think somehow Brett has gotten too obsessed in some way. It seems crazy to me.

    If he needs a little better server, a better firewall, and 2 or 3 people to help manage traffic, etc., all he needs to do is sell a few ads on his site. It makes no sense. I'm wondering if he somehow is scared of success, and subconsiously does things to prevent success.
     
    dvduval, Nov 24, 2005 IP
    Crazy_Rob likes this.
  12. Solicitors Mortgages

    Solicitors Mortgages Well-Known Member

    Messages:
    2,217
    Likes Received:
    139
    Best Answers:
    0
    Trophy Points:
    103
    #52
    is it a way of purging googles index?

    if he changes the robot text back to what it was surely they would get spidered normally??

    and he would go straight back to his normal PR..... or greater (seeing how many more links this has generated.)

    or can google (calling his bluff) turn round and say "No, you are out, stay out?"
     
    Solicitors Mortgages, Nov 24, 2005 IP
  13. markhutch

    markhutch Peon

    Messages:
    357
    Likes Received:
    22
    Best Answers:
    0
    Trophy Points:
    0
    #53
    I believe Google makes a domain wait at least 180 days before reindexing after they have been removed. In that time period considering the competition from places like this forum, they might be hurt more than any new linking strategy would help. Time will tell.
     
    markhutch, Nov 24, 2005 IP
  14. Solicitors Mortgages

    Solicitors Mortgages Well-Known Member

    Messages:
    2,217
    Likes Received:
    139
    Best Answers:
    0
    Trophy Points:
    103
    #54
    excuse my ignorance on this issue, but is that official?
     
    Solicitors Mortgages, Nov 24, 2005 IP
  15. Dekker

    Dekker Peon

    Messages:
    4,185
    Likes Received:
    286
    Best Answers:
    0
    Trophy Points:
    0
    #55
    I really don't think anything we know is official :p Even the official stuff is up to interpretation lol
     
    Dekker, Nov 24, 2005 IP
  16. markhutch

    markhutch Peon

    Messages:
    357
    Likes Received:
    22
    Best Answers:
    0
    Trophy Points:
    0
    #56
    http://www.google.com/webmasters/remove.html

    Apparently, some helpful webmasters submitted the WMW url to googles removal tool after the site owner changed the robots.txt file, so they are out for at least 180 days.
     
    markhutch, Nov 24, 2005 IP
  17. cyclinder

    cyclinder Peon

    Messages:
    29
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #57
    well google simply decided webmasterworld is of little value and i tend to agree with google on this ;)
     
    cyclinder, Nov 25, 2005 IP
  18. Nintendo

    Nintendo ♬ King of da Wackos ♬

    Messages:
    12,890
    Likes Received:
    1,064
    Best Answers:
    0
    Trophy Points:
    430
    #58
    er, did you read the page linked to in the first post??!!! They took the site out themselves using robot.txt and the form Google provides.
     
    Nintendo, Nov 25, 2005 IP
  19. Immorta

    Immorta Peon

    Messages:
    379
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    0
    #59
    I would imagine he is blocking bots to find the ones that ignore the robot.txt and then ban them through an IP ban which will solve a lot of the bandwidth problems.

    Seems like a sensible idea although he is going to lose a lot of traffic doing it.
     
    Immorta, Nov 25, 2005 IP
  20. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #60
    Really? It sounds like a really dumb idea to me. If this is the only way he could think of to figure out who are the "bad bots" so they could be banned, I'd be recommending that he hire someone from this forum to do a bit of consulting.
     
    minstrel, Nov 25, 2005 IP
    Crazy_Rob likes this.