I have a PHP app which is very large. There are usually 2-3 developers working on it full time and we are getting to the point where we are making changes and creating bugs (cough features!). The software isn’t complex per say, just there is a lot going on (35~ controllers, about the same models, etc).
Even being careful it’s easy for a change in this view (tweaking an id on an element) to wreck an ajax query happening under some special condition (logged out while standing on one foot).
Unit tests are the first things which spring to mind, but we tried these on another app, and it’s so easy to forget them / or spend more time writing tests then doing tests. We do have a staging environment where code gets checked over before pushing live.
Maybe we need a part time Q/A person?
Anyone have any suggestions/thoughts.
1
Yes, you do need Q/A staff. Some of the many reasons include
- A dedicated tester costs money, but often less money than a developer, so the benefit of not using up your time is greater than the additional expense.
- A dedicated tester knows how to test things, particularly things that it’s not obvious how to automate. Driving automated tests for interacting with a system through a browser is a somewhat hairy but well-established discipline. If you get someone who already knows how to do it, you don’t have to spend even more time learning good tools and set-ups.
- A professional tester knows how to actually find defects. They are much more likely to think like a user of the application will think, and therefore exercise those states in the system that will actually come up in production, which means that those bugs that turn out to be highly visible will tend to be found earlier, saving you embarrassment and costs for ultra-urgent patches.
- Generalizing that, a tester does not think like a developer. It is hard to convey how much difference this makes if you haven’t experienced it. Consciously or not, a developer doesn’t want to find defects. They know how the system works and tend to avoid the typical nonsensical (to them) input or data that cause trouble in real life. If something works in an unexpected way, they know how to work around it and tend not to see this as a defect at all. They never have difficulty understanding what system responses mean, because they wrote them, even though this is a major cause of trouble in almost all real systems. In a nutshell: programmers tend to be bad at having the typical problems that users have, because they’re highly trained specialists. A tester has a much easier time performing the most relevant tests.
That said, nothing beats productive cooperation between a developer and a tester for driving system quality through the roof. A developer often notices symptoms that something is wrong way before the tester would. A developer can often advise a tester how to reproduce a problem much more efficiently and how to write a proper issue report, i.e. include that details that are actually necessary to figure out the problem. But all of that requires at least one tester you can work together with.
1
You most probably need more or better regression tests (not specificially unit tests). What kind of tests that should be you have to analyse yourself, but they should detect the bugs you are talking of. I suggest you start making a test plan and priorize those tests – and when you do this, initially don’t think too much about test automation.
Afterwards, ask yourself if you can automate some or most of the tests with reasonable effort. If the answer is yes, then you should program them. If the answer is “no”, and you think the “part time Q/A person” is cheaper, then it should be obviuos what you need. In most cases, it is a good idea to have both – a Q/A person for manual testing and inventing new tests, and a lot of automated regression tests, too.
3
Hire a professional QA
This should be done if you are developing a commercial project. Having a product ready without a robust testing strategy would cost you more with bug fixes. In addition, gaining new customers or retaining them will also depend on how good is your application tested.
Generally speaking unit testing should be applied to your code base, however integration testing and manual testing should not be discarded.
Unit testing is a really good idea, especially if your project is growing. If writing unit tests becomes a habit, it’ll ease your work a lot. There is a video on youtube about writing clean code, which is easier to maintain and test.
QA engineer is a must, too. A good QA tester will not only find bugs in functionality, but also will test if the app is user-friendly (which is almost impossible to test by yourselves). Here is a nice article explaining how QA team will save you time and money and help deliver better software.
15 controllers and models isn’t very large. It takes some time to make writing test a habit, kicking each other towards it (in a friendly way first) helps a lot.
There exist tools that can control the test coverage to some extend. Code Coverage Tools for PHP
1