Skip to content

CAPTCHA testing – Dedicated to Andy Glover

The reason that I started to write this blog post is because Andy Glover initiated the topic about CAPTCHA testing on Twitter. I had already written about captcha in one of my blog post here but, that was part of other title and very less was covered in it. That is why I thought of writing a dedicated blog post on CAPTCHA testing which is dedicated to Andy Glover (Don’t you know him? He is great cartoonist with respect to Software Testing. Check out his blog at http://cartoontester.blogspot.com/ – I love his cartoons and I am pretty sure that even you will). Below is a list of test ideas for testing CAPTCHA (Avoid spamming by bots while humans can still continue to spam even if there is CAPTCHA).

 

Functional / Usability / Accessibility

  1. When the CAPTCHA is not entered, there should be a client side validation which displays error message on submission of the form.
  2. CAPTCHA entry should be case sensitive.
  3. All the data of the form fields should be retained if error is occurred for CAPTCHA text field.
  4. There should be a link (AJAX) which reads like, “Refresh the text in CAPTCHA”. That would help end-users in refreshing the captcha characters because they were not comfortable in understanding some characters in the current image.
  5. There need to be audio support if the website is supported even for blind end-user or even partially blind end-users.
  6. Too much background noise might even spoil the comfort in listening to audio captcha. It is good to consider opinion of audio captcha from many people. Considering partially blind and blind people to listen to it might be an awesome idea.
  7. If web service is being used and is fetched from another server, it is important to see whether it is rendered smoothly in sync with the other form components and web page elements. In my experience, I have seen captcha being loaded after few seconds, after all page elements are displayed which gave me a feeling that there is nothing to be displayed and whole thing is completely loaded. Then, later suddenly it displays giving an odd feeling.
  8. Proper TAB indexing should be done even for captcha text field. I have experience where reCaptcha was used in registration form and TAB indexing was missed for it. Then I suggested to the developer to fix that as there is option by reCaptcha Google to provide tab indexing option.

 

Security

  1. CAPTCHA images should not reveal absolute path names. Usage of web services is a good idea, just like reCaptcha.
  2. Do not have cyclic fashion captcha images. Like 1 to 100 and then again 1 to 100. Easy to crack. It is good to have some algorithm which generates huge number of captcha images using image libraries.
  3. Usage of background noise in the image, different textures, and different angle of displaying the characters might be a good idea to make it difficult for some captcha cracking programs like http://free-ocr.com/ and few others.
  4. Audio to text converters – Use some of these software(s) and see whether they are able to crack the audio captcha or not.
  5. CAPTCHA should refresh on every wrong entry. Keeping it static might be vulnerable to brute force attack for captcha to bypass it.
  6. There needs to be server-side validation for CAPTCHA entry. Use Firebug to Inspect Captcha element and then just delete it from client-side. Then, just fill the form without captcha and submit it. If it gets submitted, then there is no server-side validation which is a high risk one. It’s equivalent to not having captcha.
  7.  CAPTCHA with question and answers in plain text and mathematical functions questions in plain text are not recommended in my opinion.
  8. Combinations of uppercase / lowercase alphabets, numerical, special characters could be used to increase the brute force combinations for CAPTCHA which would turn out to be very difficult to crack CAPTCHA quickly. Hackers usually do not employ brute force for so many numbers of combinations; rather they would hire a human to bypass the captcha manually. Well, yes. There are CAPTCHA breaking services.
  9. Saving list of questionnaire for CAPTCHA in JS file is easily vulnerable as all the questions could be retrieved easily and assertions could be easily added using some automation tool like Selenium and bypass CAPTCHA. I had seen this vulnerability in check-in service web application Gowalla or Foursquare – I do not really remember which one exactly.

 

Some experiences

  1. I had hacked the captcha by getting absolute path names. The images were named like 1.PNG, 2.PNG, 3.PNG…200.PNG. I used DownThemAll and gave range to download all images. Then I just prepared text equivalent of those captcha’s within 1 hour of time and had all 200 captcha equivalent text for those. Then added assertions like, when 1.JPG, enter the corresponding text. Then I was successful in bypassing any captcha.
  2. This was for Gowalla or Foursquare (Again I do not remember exactly), I was able to remove the CAPTCHA component from client side using Firebug and then submit the registration form without captcha. Guess what? I was able to register successfully. So, server side validation is a MUST.
  3. In Mozilla Firefox quality.mozilla.org, I saw a Turing test which always had a same question which means same answer. It was surprising to me. I reported it to them and now it is fixed.
  4. One of the government websites of Indiahad a CAPTCHA which was easily cracked by using http://free-ocr.com/ — I cannot reveal the website name because it has not yet been fixed.

 

I think I will stop here. I would add any other test ideas that I get in future in the comments section.

 

Resources

http://en.wikipedia.org/wiki/CAPTCHA

http://www.captcha.net/

http://caca.zoy.org/wiki/PWNtcha

 

Do you have any other specific questions or test ideas? Then comment section is always there. I hope bots do not want to ask questions or say some test ideas. I have got no CAPTCHA. I would not implement it till I see some great work in housekeeping *Giggles*.

 

I hope you all had a good read. Feel free to share this on Facebook, Twitter or LinkedIn. I do not charge for it. It’s FREE FREE FREE!!! Thanks in advance for sharing. Love you all.

Follow Me

SanthoshTuppad

Software Tester, Product Innovator, Security Enthusiast at Test Insane Software Testing Services
I have been as a software tester for over 5 years. I am a hands-on tester and I've been winning bug battles & testing competitions across the world. I am a testing enthusiast, who conducts free workshops on security testing across India (Covered locations: Bengaluru, Pune, Hyderabad & Chennai. Invite him to come to your location), and monthly meets for testers in Bengaluru. I am also an avid testing blogger.

My interests include traveling, driving my SUV, health & fitness and many others. I mentor budding entrepreneurs, testers, teams in any profession.
Follow Me

Latest posts by SanthoshTuppad (see all)

Share/Bookmark

16 Comments

  1. Andy Glover wrote:

    Thanks for the testing ideas, they will certainly help me at my work as we test the CAPTCHA application/s, and thank you for dedicating the post to me! It’s always good to hear people like the cartoons :)

    Wednesday, June 27, 2012 at 12:35 am | Permalink
  2. I S Patil wrote:

    Great blog Santosh,

    Wednesday, June 27, 2012 at 12:47 am | Permalink
  3. narain wrote:

    Well written, Its Great Article!

    Wednesday, June 27, 2012 at 4:16 am | Permalink
  4. Gaurav wrote:

    Very exhaustive coverage.

    Wednesday, June 27, 2012 at 9:44 am | Permalink
  5. Thanks Santosh for such a good post. I used all the suggestions and test ideas in the real time as I was testing a Registration Form.

    Friday, June 29, 2012 at 2:23 am | Permalink
  6. Deepak Malladad wrote:

    Very useful test ideas by santhosh.

    And in one of application which i was testing recently there the old format of captcha was used and the same captcha code was displaying when i checked with source code using Fire bug. Then i recorded whole scenario using Selenium IDE and right clicked on captcha image and stored the same by using variable and retrieved the stored Captcha code content via stored variable in captcha field when i run the recorded code it worked, The same captcha code was entering in the field what it was displayed in the image every time.

    Correct me if i am wrong.

    Friday, June 29, 2012 at 5:39 am | Permalink
  7. That’s a good one :) Anywhere in the client-side source code there should not be characters which match with the currently loaded CAPTCHA. That could be easily automated. Even the CAPTCHA patterns which are consistent should not exist.

    Friday, June 29, 2012 at 10:48 pm | Permalink
  8. I am glad :) Thanks!

    Friday, June 29, 2012 at 10:48 pm | Permalink
  9. Deepak Malladad wrote:

    Thanks Santhosh..

    Friday, June 29, 2012 at 11:35 pm | Permalink
  10. Excellent blog post Santhosh! I wouldn’t have thought about the path names in the images unless I’d been debugging it with firebug/dev tools, it’s fantastic to see that you caught someone out with this, great idea!

    Thanks for sharing.

    Monday, July 2, 2012 at 12:06 am | Permalink
  11. I S Patil wrote:

    Hello Santosh
    Need a help from you.
    While testing the application, when the Login is created the Password is stored in Encrypted format in DB, is it possible to decrypt the password.

    fec5000e12be980015664747e59ecf48

    this is the encrypted password

    Thursday, July 5, 2012 at 3:36 am | Permalink
  12. Yes, it is possible. You need to understand that, there is no fool-proof implementation. However, storing it in hashes will make it difficult and builds a better layer than just keeping it in without hashes format. You could Google for “RainbowCrack” where you get 8GB of hashes DVD and with GPU powerful machines, it might be possible to crack it however, cannot make sure if it could be really cracked. But, possibilities exist.

    Saturday, July 7, 2012 at 1:48 am | Permalink
  13. Thanks for the testing ideas, they will certainly help me at my work as we test the CAPTCHA applicationsshopping Softwares

    Monday, July 23, 2012 at 2:31 am | Permalink
  14. iphone mobileman wrote:

    I have to admit, I hate Captcha. I can never see some of those images to figure out what to type. It would help if it was bigger, and definitely they need to refresh with a wrong entry.

    Saturday, August 4, 2012 at 7:26 pm | Permalink
  15. Agreed, it’s more of usability for humans they got to do. People need to understand that, CAPTCHA is to avoid bots rather than humans :D

    Monday, August 6, 2012 at 1:26 am | Permalink
  16. Jari Laakso wrote:

    Hi,

    I’ll shortly comment a few comments.

    #11 – Not using “salt” is like cooking without salt. It just doesn’t work.

    #14 – I dislike CAPTCHA also for many reasons. Do you have ideas how to replace it?

    Have a fantastic testing!

    Best regards,
    Jari

    Saturday, August 11, 2012 at 3:21 am | Permalink

Post a Comment

Your email is never published nor shared. Required fields are marked *
*
*