phpBB • Blog • CAPTCHAs in phpBB

Blog

CAPTCHAs in phpBB

Posted by Kellanved in Development, Modifications with the tags Bots, CAPTCHA, MODs, Project History, SPAM, Visual Confirmation on August 28th, 2008

This post no longer reflects the state of the art. See here .

CAPTCHAs – “Completely Automated Public Turing test to tell Computers and Humans Apart”s – are known as the foremost means to stop registrations by SPAM programs, so-called “Bots”. In phpBB, a visual confirmation CAPTCHA is used.

The key here is the “Completely Automated” part, meaning that the software – phpBB – creates the question and the correct answer without interaction by a user. This has the drawback that computers are usually able to find the answer as well, given time to adjust. This is an active field in research. In the end it is an arms race. A new CAPTCHA will usually buy a few months of peace, before the major Bot vendors adjust their products.

This article is about presenting some reasons behind our CAPTCHAs; it is not intended to be a case for or against CAPTCHAs in general or particular. It is not about other means to combat SPAM, but only about CAPTCHAs.

The CAPTCHAs of phpBB

CAPTCHAs first became part of the phpBB package with version 2.0.10, when the need for some means to stop registration of Bot users became obvious. (Actually it was bundled from 2.0.5 onwards, but not installed) For that purpose, the well known CAPTCHA from the 2.1 branch (which later became 3.0) was backported to 2.0. We all know that the CAPTCHA was very simple to beat, but it did the job at the time.

For a while.

The original phpBB CAPTCHA

However, as the development of what was later to become 3.0 proceeded, it became clear that it would not do to for a future version and so Robert “Xore” Hetzler took the task of developing an unique new set of CAPTCHAs for the development version.

The concept was having a powerful plugin system to allow a vast array of different CAPTCHAs, with a large number of configuration options. However, as development picked up again for the beta versions, it was decided that the system was too ambitious for the 3.0 line and should be pushed back.

In its stead, a Free Type based 3rd party CAPTCHA was bundled with the Beta versions. However, NeoThermic soon found out that it was broken by default.

This was for the betas

… that was too easy.

As a result, the CAPTCHA code by Xore was resurrected and surveyed for CAPTCHA implementations that were compatible with subsilver2 and prosilver, while being unique enough to keep Bots away. Two such algorithms were developed to work with the downsized CAPTCHA code, one of which was finally selected to become the default CAPTCHA of Olympus.

This one was rejected

And this one was accepted

We selected it, because it was very unlike existing implementations; naturally we are and always were aware of ways to break it – but that’s a different story. To this day, the bundled implementation works reasonably well. We are always working on possible replacements, should that change.

Default CAPTCHAs

The major issue with a CAPTCHA bundles with a software like phpBB is that it has to satisfy a very restrictive set of requirements.
At the very least it has to

work properly out of the box
work regardless of language settings
be solvable for almost all humans
and few Bots

The first is very important. Any approach that requires users to register to a service like recaptcha or to set up questions and answers is not acceptable. Users can switch to such a solution later on, but for the initial setup that would be very bad indeed.
2. is often overlooked, but very important nonetheless: the CAPTCHA has to work regardless of language and cultural background.
3. is rather obvious, but can be surprisingly difficult. Not everyone is able to see 3d effects in 2d images; some people are unable to identify emotions, some can’t tell red and green apart and so on. Finally, there is 4.; meaning that almost any existing solution won’t work.

Custom CAPTCHAs

A CAPTCHA just created for your site is a different story altogether. Here you can use tricks specific to your audience, like questions or images specific to the topic of you page. In fact, even very simple approaches will work as long as they are unique. Most custom CAPTCHAs would be broken in a matter of hours, should they be included in a package like phpBB. But on your unique page, a custom CAPTCHA can be an excellent defense.

CAPTCHA Services

At the moment, CAPTCHA services like reCaptcha or MSR’s Asirra are very fashionable. Conceptually the problem is that such solutions don’t work out of the box; they require the administrator to register for the service. Moreover, they usually rely on javascript. There also is the lingering problem that breaking such services would open up all pages using them. This incentive makes it likely that spammers will invest the money required to break services, leading once more to the same situation we are in already. If you are looking into readily available solutions to stop bots from registering, then services like the above are very good ideas indeed.

Q&A, Kitten Auth et al

The most frequently asked questions regarding CAPTCHAs are “Why don’t you use a Q&A approach?” and “What about kitten auth?”. In fact those are excellent questions. “Question & Answer” subsumes approaches where a somewhat not-trivial question is asked upon registration, with the user having to provide an answer within certain parameters. The problems with this approach are manifold. For starters, google will answer many questions, so that many good questions are not effective. Then, the program has to be able to verify that the entered solution was correct. What about typos? What about writing “two” instead of “2″? What about internationalisation? What to do once a bot has a correct answer? When doing multiple choice – how to prevent brute forcing? The list goes on. The main issue certainly is: the approach is not a CAPTCHA at all, but merely a challenge where the questions and answers are very finite. Any given set of questions and answers bundled with the software or downloadable will be known by bots instantly.

An attempt to sidestep the language issue is kitten auth. Kitten auth presents the user with a set of images, some of which satisfy a question (show kittens). This has the drawback that – like with Q&A – the number of questions is very finite. Any freely available set of images bundled can be hashed by bots, meaning that only unique picture sets will provide any barrier at all. Simple distortions are not even remotely enough to throw image identification algorithms off.
So the answer to all questions about kitten-auth and Q&A has to be: go for it if it works for you; it wouldn’t if we were to use it.

Audio CAPTCHAs

Sometimes the question arises why there are so few audio based CAPTCHAs. Generally the answer is threefold:

Internationalisation is almost impossible; letters are pronounced differently.
Many users do not have sound enabled
The noise needed to fool audio recognition programs also makes it impossible for humans to solve the CAPTCHA

Animated CAPTCHAs

Sometimes there are attempts to use flash or java to display CAPTCHAs. This approach has an underlying problem: the java or flash program that displays the CAPTCHA runs on the client computer and thus needs to know the code it is supposed to display. Bots can just extract that seed and don’t bother with the flash/java shell around it. Also it requires the client to have the particular platform installed, which is not a good assumption either.

Limitations

The lessons about CAPTCHAs certainly include:

Any CAPTCHA bundled with a major application is doomed
There are always spammers using humans to pass the CAPTCHA
There are always humans incorrectly rejected by the CAPTCHA

A simple Q&A CAPTCHA for phpBB3

The current 3.0 CAPTCHA implementation is lacking plug ability, something that had to be done to get it working in finite time. However, even so it is possible to use CAPTCHA classes other than the ones bundled without having to adjust code outside the CAPTCHA class.

The following code shall provide an example for a simple Q&A CAPTCHA that can be installed simply by replacing the existing captcha_gd file. Then the user would have to provide a question by altering the language entries and a set of images satisfying the question and another set not doing so. The images have to be in the “correct” respectively “incorrect” directories.

The code is very simple and not fleshed out to the degree one would expect from a MOD or so; it is merely given as an example.

Code: Select all

Code: Select all: <?php /** * * @package VC * @version $Id: $ * @copyright (c) 2008 phpBB Group * @license http://opensource.org/licenses/gpl-license.php GNU Public License * */ /** * @ignore */ if (!defined('IN_PHPBB')) { exit; } class captcha { var $width = 360; var $height = 96; /** * Create the image containing $code with a seed of $seed */ function execute($code, $seed) { global $phpbb_root_path, $config; srand($seed); mt_srand($seed); // Create image $capt = new yes_no_captcha(); init($code, $this->width, $this->height, $phpbb_root_path . 'images/captcha/smilies'); $img = $capt->paint(); // Send image header('Content-Type: image/png'); header('Cache-control: no-cache, no-store'); imagepng($img); imagedestroy($img); } } /** * @package VC */ class yes_no_captcha { var $letter_img = array(); var $path; var $img; var $width = 0; var $height = 0; var $x_per_letter; /** * Init method * @param string $code The correct answer * @param int $width The number of pixels in the x direction * @param int $height The number of pixels in the y direction * @param string $path The path to load the images one. Needs to contain directories "correct" and "incorrect" with right respectively wrong pictures */ function init($code, $width, $height, $path) { global $phpbb_root_path, $phpEx; $this->width = $width; $this->height = $height; $this->path = $path; $this->img = imagecreatetruecolor($this->width, $this->height); if (!function_exists('filelist')) { include($phpbb_root_path . 'includes/functions_admin.' . $phpEx); } $images = filelist($phpbb_root_path . $path); $code_len = strlen($code); $anticode_len = mt_rand($code_len - 1, $code_len + 1); $rand_str = md5(mt_rand(0, 2000000000)); $rand_str = str_replace('0', 'Z', strtoupper(base_convert($rand_str, 16, 35))); $anticode = substr($rand_str, 0, $anticode_len); $codes = array( 'correct' => $code, 'incorrect' => $anticode, ); $letters_correct = 0; $letters_incorrect = 0; while($letters_correct < strlen($codes['correct']) || $letters_incorrect < strlen($codes['incorrect'])) { $which = ($letters_correct < strlen($codes['correct']) && $letters_incorrect < strlen($codes['incorrect']) ? mt_rand(0,1) : ($letters_correct letter_img[] = array( 'letter' => $codes[$name][$i], 'image' => $name . '/'. ($images[$name . '/'][mt_rand(0, count($images[$name . '/']) -1)]), ); } $this->x_per_letter = floor($this->width / count($this->letter_img)); } function paint() { $count = count($this->letter_img); for ($index = 0; $index write_letter($index); } return $this->img; } function write_letter($index) { $imgfile = $this->path . '/' . $this->letter_img[$index]['image']; $imagedata = getimagesize($imgfile); switch ($imagedata[2]) { case 1 : $img = imagecreatefromgif($imgfile); break; case 2 : $img = imagecreatefromjpeg($imgfile); break; case 3 : $img = imagecreatefrompng($imgfile); break; default : return false; } $black = imagecolorallocate($this->img, 0, 0, 0); $img_rot = imagerotate($img, mt_rand(0, 30) - 15, $black); $position_x = $this->x_per_letter * $index; $color = imagecolorallocate($img_rot, 0, 10, 245); imagecopyresized($this->img, $img_rot, $position_x, 0, 0, 0, $this->x_per_letter, $this->height / 2, imagesx($img_rot), imagesy($img_rot)); imagestring($this->img, 5, $position_x + mt_rand(5,10), $this->height / 2 + mt_rand(5,10), $this->letter_img[$index]['letter'], $color); imagedestroy($img); imagedestroy($img_rot); } } ?>

captcha_gd
As an example, the class used with “Enter the letters underneath smiling emoticons” (which is a very bad idea, so don’t do that at home) and the 3.0 smilies.

Which ones are smiling/smirking/laughing?

And for the future? You’ll see …

Update

An early prototype for the phpBB 3.1 CAPTCHA plugins can now be found in our SVN repository.

You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.

35 Responses to “CAPTCHAs in phpBB”

Posted by Elias on August 28th, 2008 at 4:04 pm:

Very nice article.
Good information.

Posted by Jwxie on August 28th, 2008 at 4:10 pm:

great article
it seems even with 10x and 10x is still sensible for human(very clear), but not for bot

Posted by Jeffrey Rosen on August 28th, 2008 at 5:36 pm:

You guys have got to get support for reCAPTCHA instead of rolling your own.

Posted by NeoThermic on August 28th, 2008 at 7:40 pm:

Well Jeffery, we covered why supporting reCAPTCHA is not only difficult, but problematic in this blog post (look for the heading “CAPTCHA Services”).

NeoThermic

Posted by Jeffrey Rosen on August 28th, 2008 at 8:00 pm:

I don’t buy it. Sure, you have to sign up for an API key, but that is part of the security of the service. Since when does phpbb care about being 100% functional completely out of the box with zero configuration?

Of course you need to install imagemagick, set up your smtp server, configure your database, etc. to unlock various features of the software, so why is it so hard to ask the user to enter an API key if they want to use recaptcha, like WordPress does for akismet? If they don’t, default back to the standard captcha.

reCaptcha is pretty much perfect. Once reCaptcha is “broken”, that doesn’t mean that the reCaptcha has been broken, it means that computers have equaled humans in terms of OCR capabilities and we will have to move on to cat analysis and stuff. The beauty of it is that it doesn’t use an algorithm to generate images, it uses text that the cutting edge OCR software failed to parse.

Basically, don’t waste time rolling your own captcha which can certainly be broken trivially, just add support for recaptcha.

Posted by Jeffrey Rosen on August 28th, 2008 at 8:04 pm:

Furthermore, there are obvious benefits to having one centralized captcha provider. For example, one IP can no longer hit 2000 servers at once. The centralized captcha server would deny access to it. With a phpbb generated captcha, a robot can try to break captchas on 2000 servers concurrently and the phpbb installation is none the wiser.

Posted by vern on August 28th, 2008 at 9:18 pm:

I understand the logic in not using something that doesn’t work out of the box like reCAPTCHA, but it would be great if phpBB could have official support for it as an option in case the administrator does decide he wants to go that route later.

Posted by Martin Truckenbrodt on August 28th, 2008 at 10:16 pm:

Hello,
CAPTCHAs are cracked and will be cracked in future, too. Or not?
So IMO it’S not really a good idea to use it.
IMO there are two points about CAPTCHA and phpBB:
1. Guest Posting: Generally it’s not a good idea to use it.
2. Account activation: My personal tipp: Create a required custom profile and give this field a crazy or random internal name. E.g. I’m using the first name for this. SPAM account bots don’t know what’s the name of this field or whether it exists at all. 🙂 On idea for core pbpBB: Perhaps all profile fields should use random generated internal names?
All is only my personal opinion!
Bye Martin

Posted by Dog Cow on August 29th, 2008 at 12:15 am:

But there are also obvious drawbacks to having one centralized captcha provider. (Sorry to sound so contradictory! 🙁 )

– well, what if that stops working?
– what if they have to shut down for some reason?
– and as mentioned in the entry, what if the bots break reCaptcha?

It’s already been my personal experience that in some cases, typing just one word is sufficient for reCaptcha, since they even say that “one word, we know the answer to, the other, we don’t”

So it can really only check you on one word.

Posted by John Winter on August 29th, 2008 at 4:58 am:

I’d _really_ like to use Mollom (http://mollom.com). They do content analysis like Akismet does, but also serve image and audio CAPTCHAs.

Posted by Kellanved on August 29th, 2008 at 9:17 am:

A plugin for reCaptcha is naturally something that will turn up, but we can’t require users to subscribe to a centralized service. Many users do not want to use such services, others can’t. The default CAPTCHA thus has to be something else.

Moreover, reCaptcha does not equate computers equalling humans. The logic works the other way round: if computers can OCR like humans, then they can solve reCaptcha. But they can solve it without that step too.
The thing is: you do not need 100%; 20% correct guesses is fine for a bot. Recaptcha itself is not particularly difficult, no matter how cool the idea looks on paper. That’s why they introduced more distortion and the line to make separation harder.

Use what works for you, services are a good idea, but can’t be default.

Posted by Kellanved on August 29th, 2008 at 10:42 am:

Also, I recommend having discussions about blog posts to the forums. Comments are fine for short responses, but constructive discussion is something the forums would benefit from.

And yah, of course there will be reCaptcha in 3.2

Posted by Techie-Micheal on August 29th, 2008 at 5:02 pm:

Perhaps setting up a dedicated topic in phpBB Discussion for each blog post so that people can discuss, share ideas, and so on? It seems a bit redundant to have both the bulletin board and the blog for commenting.

That said, I’ll comment later as I have some things to share. Maybe I’ll just spam you with trackback requests again. 😛

Posted by suitlocal on August 29th, 2008 at 5:23 pm:

@Jeffrey Rosen – phpBB doesn’t require imagemagick be installed and by default, phpBB uses php’s mail() function, which alievates the need to confgigure an SMTP server.

The only thing you need to configure is the database and that, right there, already disuades users from installing phpBB, no doubt. By that I mean… lots of users still opt for flat file message boards because they don’t want to deal with the hassle of a MySQL database.

And plus, if you do it for reCaptcha, what’s next? Maybe Akismet! Maybe something else! Maybe in order to install phpBB, you’ll have to click Next 200 times because that’s how many services phpBB is asking you to enter API keys for! Why spam your little service when phpBB will do it for you?

Posted by Eric Martindale on August 29th, 2008 at 6:52 pm:

I’ve had best luck with preventing spam registrations by creating a question that users must answer to register.

Example: “Are you a human or a bot?”

Naturally, a bot can’t answer that question, but a human can. Since I implemented this on a few forums I run, I’ve never once had any trouble with spam.

On one of my larger forums, I’ve just tweaked the default phpBB3 CAPTCHA settings with good success – but remember, no matter how good a visual CAPTCHA is, there’ll eventually be an OCR system that can solve it with a reasonable degree of accuracy.

And then of course there’s the Mechanical Turk solution, which completely destroys any and all CAPTCHAs. 😀

Posted by Martin Truckenbrodt on August 30th, 2008 at 10:04 am:

Hello,
why to use a blog to publish information about a discussion board? 😉

Bye Martin

Posted by Thierry S. on August 30th, 2008 at 7:58 pm:

Good article!

The Mechanical Turk solution isn’t even necessary. Just create a porn site and ask the visitors to resolve the CAPTCHA you found in order to view the image… Guaranted mostly accurate & fast answer in a minimum time 🙁

Check out Luis Von Ahn presentation about human computation available on google video. Enjoy it 😉

Posted by Micaella Isparano on September 1st, 2008 at 2:32 am:

Some sort of an audio captcha would be very good, if it were very easy to use.

Posted by Kellanved on September 1st, 2008 at 10:17 am:

Yes, easy for spammers, hard for humans.

Posted by xiriox on September 3rd, 2008 at 11:23 pm:

This information will be helpful for when I get my forum running again but I see somethings I want to comment on. There seems to be audio captchas that people can get through using bots but why don’t we use hybrids. Some people have forums that use image captchas and if someone can’t understand then they can “listen to the letters” or get the audio captcha. this gives bots 2 methods of getting into your forum to spam but what if you took the questions and answers and twist them up. Image captchas ask you to identify letters from obeservation, audio captchas ask for the same but through listening. What I’ve only seen from one other person is a private custom captcha that doesn’t allow bots to register or seperates them. Some admins can learn when there is a bot only after they’ve registered, admins have the option of grouping them up and deleting them or taking away their permissions. This only goes for some of us and only with MySQL (I think). If we stick to captchas however, then when giving a captcha the question should be one unrelated to the captcha and with a random answer count. This way brutes can’t get through and bots can’t go through your database for an answer. I know from sight that this method can and has been done but I don’t know how to manually do it. The best way to protect yourself is look at yourself as a bot and see what you can do to penetrate yourself and take the right measures to become less helpful to the bot. You won’t ever be completely safe but you will get close. Recaptcha seems like a good option but can be very annoying so rather than using it you can take its scripts and apply them to your own cutom made captcha. There is always the illusion where you make it so one would have to use logic to get through which a bot can’t use. Put up a captcha which will not need an answer but will ask you for one. Bots always try to get around or will try filling the blanks so they can’t complete the registration. They will be led to an infinite random answer loop. There are few ther methods fellow c++ programmers have sought but I myself can’t justify all of them.

Posted by Nomi on September 3rd, 2008 at 11:43 pm:

hi all

CAPTCHA z a good utility but it still needs to be more powerful

Posted by Ger on September 4th, 2008 at 12:14 pm:

Very informative article, thank you for that.

By the way: on my forum, I made an extra custom profile field (which doesn’t show in the profile). It tells to enter a specified text (provided, so easy to copy-paste) and is required at registration. The field checks to be the length of the text provided, so when bots just random fill something in, the registration fails.

So far, for about 10 months, this works perfectly. No single bot has registerd yet.

Posted by ahsan on September 19th, 2008 at 7:14 am:

that is nice article that i was looking for
thanks phpbb

Posted by mrgtb on October 20th, 2008 at 5:56 pm:

Right now, there seems to be a big spike in SPAM BOT problems, vBulletin owners have been having some serious issues with the new spam bots of late. And I even noticed IPB are using “reCapture” on their own offical forum.

EDIT: Link removed – Highway
[ http://www.phpbb.com/rules/#rule6a ]

Posted by Mark S on November 2nd, 2008 at 4:58 pm:

You like captchas don’t you? Seems like the latest marketing garbage where many believe it improves your real human visitors experience with your site.

I found it pathetic to say the least, to bloat forms with crossed-out images and mysterious active scripts and other ajax and anti-ocr methods to identify if it’s a bot or a human who submits a form.

Forms content enrichment? no thanks, forms are complex enough.

In reality you only need some extra html with the forms and some extra form processing none of which is visible to your visitors and you can efficiently protect your form pages from automatic submission.

Posted by dams on November 6th, 2008 at 12:18 pm:

how about recapcha addon ?

Posted by Eyekhan on November 8th, 2008 at 4:09 pm:

Is it possible to deny the use of captcha altogether by initiating a registration completion by email method? Is this a viable option inside of phpbb?

Most of the spam I have been receiving on my forum seems to come from spoofed email addresses and bogus identities in Arabia. My Co-Admin spends an enormous amount of time proofing the user list for fakes.

Posted by gid on April 3rd, 2009 at 12:19 am:

I have a blind person trying to join the forum, and she cannot because of the captcha. An audio option would be great.

Posted by Darrell Shandrow on June 28th, 2009 at 1:45 am:

It appears CAPTCHA accessibility for blind and visually impaired users has not yet been addressed by PHPBB. What does PHPBB intend to do in order to stop locking us out? If PHPBB can’t or doesn’t want to roll out an audio CAPTCHA, there are now a number of options.

Posted by Stonewall on July 20th, 2009 at 1:18 pm:

I tried Captchas in their several permutations and have come to the irrevocable conclusion that this idea deserves to die.

Captchas are inconvenient and annoying to humans, but seem to present no barrier to spambots, which started flooding my phpBB 3.0 site from around the world.

I dropped the Captcha, substituted the very simplest of questions instead, thereby stumping spambots but allowing painless entry by any human.

Posted by Kellanved on July 22nd, 2009 at 1:21 pm:

The blog post is based on a past state and is as of 3.0.6 no longer reflecting the state of things in phoBB3.

http://www.phpbb.com/blog/2009/06/27/3-0-6-captcha-plugins-and-you/

Yes, simple questions are in there too.

Posted by gare on September 2nd, 2009 at 7:51 pm:

Can we use the captcha and then have the user answer a question? Is there an easy way to implement this for someone who can barely inderstand “html for idiots”?, ie me?
gare

Posted by Kellanved on September 3rd, 2009 at 10:44 pm:

Yes, 3.0.6 has an easy-to-use dialog to set up question without using html.

Posted by Stephanie Smith on November 27th, 2009 at 4:12 pm:

Congrats, I attempted to register to a phpbb hosted blog and was totally stumped by the captchas GD provided by your site. You succeeded in wasting a bunch of my time and locked me out. Not being a computer wizz I am unaware of the sevear problems that the spamming community is wrecking on the web. I would like to know more.

stephanie

locomotive engineer

Posted by Cranberry Muffinman on August 27th, 2010 at 1:48 am:

Is it possible to have a recapcha and a Q&A one at same time? i want both but it wont let me