1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Stupid SpamBots Getting to me!

Discussion in 'Security' started by nkiegrea, Feb 18, 2007.

  1. #1
    Hello, I own blitzgamer.com and currently have been getting spam bots coming to my site through google (I think) and writing reviews on my site that are just plain spam about various things.

    What I don't understand is that you have to be a member to write a review, the reviews in the admin panel shows the review to not have a member associated with it. SO they are not a member..

    I am pretty confused on the whole thing.

    If anyone can help much thanks and rep will be awarded!

    The site is being ran with a custom php script that the person I bought the site from created.

    Regards,
    Zack B
    BlitzGamer.com
     
    nkiegrea, Feb 18, 2007 IP
  2. Pat Gael

    Pat Gael Banned

    Messages:
    1,331
    Likes Received:
    68
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I got this robots.txt from another site having the same problem, just drop it into your root directory.
    
    User-agent: BotRightHere 
    User-agent: larbin 
    User-agent: b2w/0.1 
    User-agent: Copernic 
    User-agent: psbot 
    User-agent: Python-urllib 
    User-agent: NetMechanic 
    User-agent: URL_Spider_Pro 
    User-agent: CherryPicker 
    User-agent: EmailCollector 
    User-agent: EmailSiphon 
    User-agent: WebBandit 
    User-agent: EmailWolf 
    User-agent: ExtractorPro 
    User-agent: CopyRightCheck 
    User-agent: Crescent 
    User-agent: SiteSnagger 
    User-agent: ProWebWalker 
    User-agent: CheeseBot 
    User-agent: LNSpiderguy 
    User-agent: Alexibot 
    User-agent: Teleport 
    User-agent: TeleportPro 
    User-agent: MIIxpc 
    User-agent: Telesoft 
    User-agent: Website Quester 
    User-agent: WebZip 
    User-agent: moget/2.1 
    User-agent: WebZip/4.0 
    User-agent: WebStripper 
    User-agent: WebSauger 
    User-agent: WebCopier 
    User-agent: NetAnts 
    User-agent: Mister PiX 
    User-agent: WebAuto 
    User-agent: TheNomad 
    User-agent: WWW-Collector-E 
    User-agent: RMA 
    User-agent: libWeb/clsHTTP 
    User-agent: asterias 
    User-agent: httplib 
    User-agent: turingos 
    User-agent: spanner 
    User-agent: InfoNaviRobot 
    User-agent: Harvest/1.5 
    User-agent: Bullseye/1.0 
    User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95) 
    User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 
    User-agent: CherryPickerSE/1.0 
    User-agent: CherryPickerElite/1.0 
    User-agent: WebBandit/3.50 
    User-agent: NICErsPRO 
    User-agent: DittoSpyder 
    User-agent: Foobot 
    User-agent: SpankBot 
    User-agent: BotALot 
    User-agent: lwp-trivial/1.34 
    User-agent: lwp-trivial 
    User-agent: BunnySlippers 
    User-agent: URLy Warning 
    User-agent: Wget/1.6 
    User-agent: Wget/1.5.3 
    User-agent: Wget 
    User-agent: LinkWalker 
    User-agent: cosmos 
    User-agent: moget 
    User-agent: hloader 
    User-agent: humanlinks 
    User-agent: LinkextractorPro 
    User-agent: Offline Explorer 
    User-agent: Mata Hari 
    User-agent: LexiBot 
    User-agent: Web Image Collector 
    User-agent: The Intraformant 
    User-agent: True_Robot/1.0 
    User-agent: True_Robot 
    User-agent: BlowFish/1.0 
    User-agent: JennyBot 
    User-agent: MIIxpc/4.2 
    User-agent: BuiltBotTough 
    User-agent: ProPowerBot/2.14 
    User-agent: BackDoorBot/1.0 
    User-agent: toCrawl/UrlDispatcher 
    User-agent: suzuran 
    User-agent: TightTwatBot 
    User-agent: VCI WebViewer VCI WebViewer Win32 
    User-agent: VCI 
    User-agent: Szukacz/1.4 
    User-agent: Openfind data gatherer 
    User-agent: Openfind 
    User-agent: Xenu's Link Sleuth 1.1c 
    User-agent: Xenu's 
    User-agent: Zeus 
    User-agent: RepoMonkey Bait & Tackle/v1.01 
    User-agent: RepoMonkey 
    User-agent: Openbot 
    User-agent: URL Control 
    User-agent: Zeus Link Scout 
    User-agent: Zeus 32297 Webster Pro V2.9 Win32 
    User-agent: Webster Pro 
    User-agent: EroCrawler 
    User-agent: LinkScan/8.1a Unix 
    User-agent: Keyword Density/0.9 
    User-agent: Kenjin Spider 
    User-agent: Iron33/1.0.2 
    User-agent: Bookmark search tool 
    User-agent: GetRight/4.2 
    User-agent: FairAd Client 
    User-agent: Gaisbot 
    User-agent: Aqua_Products 
    User-agent: Radiation Retriever 1.1 
    User-agent: Flaming AttackBot 
    User-agent: Curl 
    User-agent: Web Reaper
    User-agent: Firefox
    User-agent: Opera
    User-agent: Netscape
    User-agent: WebVulnCrawl
    User-agent: WebVulnScan
    Disallow: /
    
    Code (markup):
    The last two in the list are well know for crawling your site in the quest of vulnerabilities.
     
    Pat Gael, Feb 18, 2007 IP
  3. nkiegrea

    nkiegrea Peon

    Messages:
    355
    Likes Received:
    43
    Best Answers:
    0
    Trophy Points:
    0
    #3
    So let me see if I understand what you are suggesting.

    Basically that robots.txt will replace mine which allows all robots..

    The list that you provied only allows those robots and no others? While my robots.txt file allows all bots which is leading to the spam..

    Is there any disadvantage of limiting to just those bots? Like what if a new search engine comes out or a smaller search engine that isn't listed in the robots.txt file?

    Anyone else have any ideas opinions on the problem and the robots.txt file?

    Regards,
    Zack B

     
    nkiegrea, Feb 18, 2007 IP
  4. Pat Gael

    Pat Gael Banned

    Messages:
    1,331
    Likes Received:
    68
    Best Answers:
    0
    Trophy Points:
    0
    #4
    That list avoid the above robots crawl your site.

    If you want to allow other mix this with your current robot.txt
     
    Pat Gael, Feb 18, 2007 IP
    nkiegrea likes this.
  5. nkiegrea

    nkiegrea Peon

    Messages:
    355
    Likes Received:
    43
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Ah.... so the long list is too block those bots... and all others will be allowed?

    Regards,
    Zack
     
    nkiegrea, Feb 19, 2007 IP
  6. nkiegrea

    nkiegrea Peon

    Messages:
    355
    Likes Received:
    43
    Best Answers:
    0
    Trophy Points:
    0
    #6
    I put the robots.txt on my server.. but still getting spam... =(
     
    nkiegrea, Feb 19, 2007 IP
  7. techie007

    techie007 Peon

    Messages:
    261
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I don't know much about it.. but it seems adding captcha in the registration page and review pages, should help you.

    I was getting a lot of spam from the feedback page, i just added captcha and now it is good for me. Let's see if that works for you as well.

    For ASP:
    http://sourceforge.net/projects/asp-captcha

    For PHP:
    http://www.webhosting.net/forum/showthread.php?p=154

    Have a great day!
     
    techie007, Feb 19, 2007 IP
    nkiegrea likes this.
  8. oziman

    oziman Active Member

    Messages:
    199
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    58
    #8
    If it's a custom made script then it has a hole somewhere.

    The bots are smart enough to search out and post..

    I would suggest both adding a captcha and tightening your code.

    O
     
    oziman, Feb 20, 2007 IP
  9. Mxhub

    Mxhub Active Member

    Messages:
    474
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    55
    #9
    switch to captcha-based contact form.
     
    Mxhub, Feb 21, 2007 IP
  10. sundaybrew

    sundaybrew Numerati

    Messages:
    7,294
    Likes Received:
    1,260
    Best Answers:
    0
    Trophy Points:
    560
    #10
    Check you forms...75% of all coders leave loose forms,not by fault but they just aren't aware of the security that is required


    I have bought so many scripts in the past, and the forms on the site were so wide that I could stick an elephant through it

    Have a PROFESSIONAL coder look at your forms, and tighten them all up

    I used to have this issue as well, and then I hired some really good people that new php real well and they locked my shit up tight
     
    sundaybrew, Feb 21, 2007 IP
    nkiegrea likes this.
  11. Smithers

    Smithers Banned

    Messages:
    1,442
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    0
    #11
    Yea I did the same thing I owned TheeForum and I had bots there all the time.
     
    Smithers, Feb 21, 2007 IP
  12. ChicagoMusicPromotions

    ChicagoMusicPromotions Peon

    Messages:
    10
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #12
    i had a simple solution to my forums i killed them First

    1. Update your php version on your forums - Especially PHPBB - mine got Owned Several Times
    make sure they are up to date

    2. administrator approval on all messages or blog entrys

    3. Or just kill the forums from your website - takes the fun out thou....

    :)
     
    ChicagoMusicPromotions, Feb 22, 2007 IP
  13. evera

    evera Peon

    Messages:
    283
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #13
    Hey I have a question to that list you posted.

    Will that block all users having these browsers or do I missunderstand something?
     
    evera, Feb 23, 2007 IP
  14. plumsauce

    plumsauce Peon

    Messages:
    310
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    0
    #14
    robots.txt will not help against spammers at all because they ignore the directives. spambots must be fought using technology not polite suggestions.
     
    plumsauce, Feb 23, 2007 IP
  15. TommyD

    TommyD Peon

    Messages:
    1,397
    Likes Received:
    76
    Best Answers:
    0
    Trophy Points:
    0
    #15
    Do spambots really care what rules you set in robots.txt?

    tom
     
    TommyD, Feb 23, 2007 IP
  16. nkiegrea

    nkiegrea Peon

    Messages:
    355
    Likes Received:
    43
    Best Answers:
    0
    Trophy Points:
    0
    #16
    lol, yes, the robots.txt def didn't stop the spamming.. :(

    does anyone know of a good php coder who has knowledge of putting in image verification for posting comments...?

    Thanks
     
    nkiegrea, Feb 24, 2007 IP
  17. kapengbarako

    kapengbarako Peon

    Messages:
    914
    Likes Received:
    28
    Best Answers:
    0
    Trophy Points:
    0
    #17
    kapengbarako, Feb 25, 2007 IP
  18. nkiegrea

    nkiegrea Peon

    Messages:
    355
    Likes Received:
    43
    Best Answers:
    0
    Trophy Points:
    0
    #18
    Is that only for phpbb forums?

    I am looking for just a form submit image verification system. Not for a forum.
     
    nkiegrea, Feb 25, 2007 IP
  19. nddb

    nddb Peon

    Messages:
    803
    Likes Received:
    30
    Best Answers:
    0
    Trophy Points:
    0
    #19
    You need to go back to the person who wrote it and make them fix it. This has nothing to do with spam bots. There is a simple problem here : Members should be the only ones posting reviews and yet, non-members can post reviews.

    That is a bad script. There will ALWAYS be spammers, but you don't always have to have a bad script. Blocking spam bots by useragent name means nothing at all. I could crawl your site now with a useragent name from google. You could tell by IP that I was not google, but it would go right by that ridiculously long robots.txt. robots.txt is NOT for security.. AT ALL.
     
    nddb, Mar 2, 2007 IP
  20. WebGeek182

    WebGeek182 Active Member

    Messages:
    510
    Likes Received:
    28
    Best Answers:
    0
    Trophy Points:
    95
    #20
    WebGeek182, Mar 3, 2007 IP
    nkiegrea likes this.