Directory Owners - Problem with automated submissions

Discussion in 'Directories' started by CanadianEh, Jun 10, 2006.

  1. #1
    I just deleted 102 of 102 submissions for one of my niche directories. None of the submissions were close to the topic of the directory.

    I have also seen most of the same submissions in my other niche directories (totally different topics). These are obviously automated submissions that are beating the captcha in PHPLD versions 2 & 3.

    Any suggestions, other than to make the directories paid.
     
    CanadianEh, Jun 10, 2006 IP
  2. Rod_Purnell

    Rod_Purnell Peon

    Messages:
    382
    Likes Received:
    29
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I have a policy on my directories that no more than 8 submissions a day are allowed from the same IP. I get the same thing you are talking about, so when that happens, the IP gets a filter. Don't have to do it often, but it's happening more often than it did before.
     
    Rod_Purnell, Jun 10, 2006 IP
  3. CanadianEh

    CanadianEh Notable Member

    Messages:
    3,812
    Likes Received:
    380
    Best Answers:
    0
    Trophy Points:
    260
    #3
    I didn't notice any obvious IP paterns.
     
    CanadianEh, Jun 10, 2006 IP
  4. rehash

    rehash Well-Known Member

    Messages:
    1,502
    Likes Received:
    30
    Best Answers:
    0
    Trophy Points:
    150
    #4
    put some image recognition
     
    rehash, Jun 14, 2006 IP
  5. CanadianEh

    CanadianEh Notable Member

    Messages:
    3,812
    Likes Received:
    380
    Best Answers:
    0
    Trophy Points:
    260
    #5
    That's the problem. The PHPLD captcha is turned on.
     
    CanadianEh, Jun 15, 2006 IP
  6. Obelia

    Obelia Notable Member

    Messages:
    2,083
    Likes Received:
    171
    Best Answers:
    0
    Trophy Points:
    210
    #6
    Write your own captcha. It doesn't have to be a sophisticated one, just different from everyone else's. The problem with the PHPLD one is that everybody uses it, so it just has to be cracked once.

    If everyone had a different CAPTCHA system, there's no way anyone would have time to work out how to crack it for several hundred directories.
     
    Obelia, Jun 15, 2006 IP
  7. CanadianEh

    CanadianEh Notable Member

    Messages:
    3,812
    Likes Received:
    380
    Best Answers:
    0
    Trophy Points:
    260
    #7
    Thanks for the great advice.

    I had a hunch about the captcha being compromised.
     
    CanadianEh, Jun 15, 2006 IP
  8. brunozugay

    brunozugay Peon

    Messages:
    150
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #8
    It would be nice if PHP Link Directory would consider making changes in time for their next release.
     
    brunozugay, Jun 15, 2006 IP
  9. BBSWatch

    BBSWatch Guest

    Messages:
    3
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    I get this as well. I'm approving maybe 15-20% on my niche directory, the rest don't even glance at the rules. I'll have to look into captcha for my list.
     
    BBSWatch, Jun 15, 2006 IP
  10. Obelia

    Obelia Notable Member

    Messages:
    2,083
    Likes Received:
    171
    Best Answers:
    0
    Trophy Points:
    210
    #10
    Image-based captchas have been crackable for some time, but until now most automated submission software has not bothered with cracking them. Recently I've noticed that many many directories have been using the exact same image verification captcha. It must have crossed the threshold of being worth their while to break this captcha.


    There's a big flaw in this thinking: PHPLD is hugely popular. If they release an upgrade, then your auto-submitters will soon follow, and you're back to square one.

    The only way they could get round this is if they implemented a trivia-based captcha that directory owners could easily modify. If the questions and answers were unique for each directory, and couldn't be figured out by machine, then that would squash a lot of the spam.

    Unfortunately this is time-intensive, and PHPLD directory owners tend to prefer to plug something in and go. So I'm convinced it will never happen, and even if it did it will never work as well as it could.

    I don't believe there's any short cut to preventing automated submissions: you have to write your own code and filters, because anything produced for a massive user base will be exploited big-time.
     
    Obelia, Jun 15, 2006 IP
  11. CanadianEh

    CanadianEh Notable Member

    Messages:
    3,812
    Likes Received:
    380
    Best Answers:
    0
    Trophy Points:
    260
    #11
    I took a look at the existing captcha. Didn't look like something I could tackel in the middle of all my other projects.

    If anyone comes up with something that works please send me a PM. I would be willing to pay for it.
     
    CanadianEh, Jun 15, 2006 IP
  12. Meatball239

    Meatball239 Peon

    Messages:
    23
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #12
    Hi all, my directory does not really get any automated submissions that i notice however, i am in the process of implementing a different Captcha as i am dissatisfied with the PhpLD Captcha as its not that easy to see and will probably cause me issues later on anyway.
    So I’ll let you know how it goes in the next day or so...
     
    Meatball239, Jun 15, 2006 IP
  13. iowadawg

    iowadawg Prominent Member

    Messages:
    10,918
    Likes Received:
    811
    Best Answers:
    0
    Trophy Points:
    380
    #13
    I have one niche directory that is food related.
    Every single day for the past week (give or take a day), I have been bombarded with submissions that are not food related at all.
    Rather they are fan sites, fan sites and more fan sites.
    One submission was for a dixie chicks site that was submitted to southern cooking!
    So at least someone has some sense of humor.
     
    iowadawg, Jun 15, 2006 IP
  14. adacprogramming

    adacprogramming Well-Known Member

    Messages:
    1,615
    Likes Received:
    62
    Best Answers:
    0
    Trophy Points:
    125
    #14
    Problem with this is, I do SEO for quite a few websites, it's not out of the question for me to add 15 - 20 sites to a new directory. I submit by hand, only submitting when I find a catagory that fits. I assume that you are not interested in having SEO companies submit to your directory then.:confused:
     
    adacprogramming, Jun 16, 2006 IP
  15. john269

    john269 Notable Member

    Messages:
    6,229
    Likes Received:
    116
    Best Answers:
    0
    Trophy Points:
    235
    #15
    I don't accept any directory submissions anymore. I had stopped accepting late last year. The main reasons for this is because I found that I was receiving alot of foreign none english submissions, I was getting alot of subimissions submitted to the wrong categories, people were not linking back to my site or adding my site to a bad part of their site, etc.

    So I have found it best to stop accepting submissions and if I want to increase the sites listed in my directory then I will manually find quality sites myself.
     
    john269, Aug 9, 2006 IP
  16. dvduval

    dvduval Notable Member

    Messages:
    3,372
    Likes Received:
    356
    Best Answers:
    1
    Trophy Points:
    260
    #16
    We could release a more sphisticated captcha. Keep in mind also there are armies of submitters, and as your directory gets popular, you will get a lot of submissions. I have never heard the captcha was being beaten by a robot, but if anyone can provide evidence of this, let me know.
     
    dvduval, Aug 9, 2006 IP
  17. john269

    john269 Notable Member

    Messages:
    6,229
    Likes Received:
    116
    Best Answers:
    0
    Trophy Points:
    235
    #17
    Yeah, they can beat it and they do.

    I have recently changed my submission forms where I have stopped submissions for the fields of my old form. I now have new database tables and have written a better form with different names for each field.

    Since then I have had a massive drop in submissions.

    I have had something like a 50 - 75% drop in submissions.

    All of the submissions are coming directly through my site now until the robots start learning my new form structures again, then it will be back to normal where I get alot more submissions again.

    I use image captcha validation on my submission form. Not all people know how to bypass this validation though so it does help quite a bit.

     
    john269, Aug 9, 2006 IP
  18. DirectoryFreak

    DirectoryFreak Peon

    Messages:
    166
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #18
    I also use a unique validation in my submission form, and when my stats show me a picture of automatation from submitters, I change some small things, like form names and sourcefiles. This works great for me.

    On the other hand, Im tired of submission firms who submit by numbers, rather than quality. These companies are the root to evil in my eyes. They do not take the time to check if the directory is "right", rather just hope for a approval.
    I own some directories, both niche and general. -Bad experience with some submission firms, have made me ban them from all my directories. (A shame for their customers)
    My best advise to people in general, is to submit your sites by yourself, or use some real quality submission firm.. DONT go for the cheap 15 year old kid who promisses he can submit to x-number of sites for you with just 1 title, description and keywords.
     
    DirectoryFreak, Aug 9, 2006 IP
  19. john269

    john269 Notable Member

    Messages:
    6,229
    Likes Received:
    116
    Best Answers:
    0
    Trophy Points:
    235
    #19
    I have one site at the moment hitting my link exchange form at one of my site. They are sending about 5 - 20 new links every day that are all different links that all seem to have the word viagra somewhere in the url.

    When I click on the links it gets redirected to a certain site. All the different urls seem to redirect to this one site.

    I keep rejecting these without even bother looking at the submission listing details. It is just spam and if I had approved them then my site would likely get banned from doing it. Have something like 200 or so urls on my site that quickly redirects to one individual site. I really don't know what they think they are doing, but they must think some people are stupid or something and will approve their submissions.

    What a joke they are!!!
     
    john269, Aug 9, 2006 IP
  20. DirectoryFreak

    DirectoryFreak Peon

    Messages:
    166
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #20
    Problem is that this works for them, because of ie. directory owners not being critic enough.
    When I get guys like this, I would like to remove them automatic so they dont waste even more of my time, either they leave the same IP or the same country/region based on IP. And combined with a banned list that grows daily it actually works sweet.

    I know some "innocent" will be hurt by this, but I cannot satisfy all, and my methods improve all the time as I find better ways to lure them out.

    My biggest problems atm come from some regions in Canada and India
     
    DirectoryFreak, Aug 9, 2006 IP