My boss wants to recruit testers based on their testing efficiency (number of bugs identified). So, he’s shortlisted 5 people and I need to give them an app full of bugs and see how they fare in reporting obvious bugs, and hidden bugs. I know…. it kind of sounds weird. I guess, this is just like the coding world, where you hire a programmer by assessing his/her programming ability (which is a little easier). Once hired, these testers would be testing a java swing app, so their familiarity of testing frameworks/tools is not really required.
So, my question here is – How do I go about finding buggy apps (web/non-web), preferably java ones, that I can have the shortlisted testers have a go at? How would you go about this task if your boss asks you to do so? I am kind of clueless at this point – I googled a bit, thought about finding new apps on sourceforge with lots of bugs, but both approaches didn’t work for me.
1
I actually did this. I created a buggy calculator. It’s just HTML and JavaScript. I created it originally to help my father (a QA professional) practice testing. And because it was fun. I then used it when interviewing.
I don’t just look for # bugs found though. I look for:
- What bugs are found
- How methodical is the person in testing
- How clearly are the defects communicated
- What questions does the tester ask
8
If you want to do this, create your own buggy app. When I was hired into my current position, I was given an app to test and find bugs that was based on actual software the company worked on. Some of the bugs were pretty obvious, some were more subtle, but mostly here were plenty of bugs or arguably buggy behaviour. My two cents: while this can be fun, it doesn’t necessarily make for a good screen of one’s testing abilities. Finding obvious bugs in a simple and small application isn’t generally too challenging.
If you want to step things up a notch, both for yourselves and the applicants, find real apps and webpages and ask candidates to look for bugs within them. Not only will bugs be usually less obvious, but it may also lead to some enlightening conversations that give some insight into the applicant’s thought processes and evaluation skills. It may also force you and your colleagues to up your testing abilities and bug finding skills. But remember, even real-life apps have bugs and issues with them, so finding buggy apps shouldn’t be too tall a task.
This should be easy. You have one or more applications already, right? And you used source control, I hope. And you should have a record of historical bugs. And you really ought to be able to easily build from old versions in source control.
So see how well they can find the bugs you actually had. If that isn’t a realistic test I don’t know what is.