Obviously, small usability fixes and bugfixes go directly into the stable product. What about small new features? Can you afford to just release them after internal testing, or do they have to be betatested by customers first?
Situation: This is a young commercial project, produced by a one-person company. It has an existing userbase and is at it’s second major version. Previous betatests have produced some results, however most feedback came from the stable product and not from beta versions.
3
If you’re code has good test coverage, including unit and integrations tests, and it’s not mission critical software, I personally don’t see the harm in skipping a beta test version. After all, a beta test is only useful if people actually use it to test it for issues, which it sounds like is not the case here.
Ideally, sure, you’d want actual users to have a chance to test the new functionality out. But thats not always feasable/practical, and the effort in doing so doesnt always pay off. I agree that beta testing doesnt always produce the information you need in such a situation. Whats often most helpful for small changes is to have a quick single round of ‘testing’ by beta testers to ensure there are no glaring problems.
If you are comfortable with the internal testing, what you can do is make the beta available for those who want it for a short period. On your website, or wherever is appropriate. You may get some feedback on it, maybe not (depends on the features and size/nature of the user base). After a short ‘open beta’, unless major problems are reported, send it out the door.
If your user base allows, perhaps build a small mailing list of users who’d be interested in trying beta builds. That can get a beta build into the hands of a few customers just to ensure there arent any show stoppers you might have missed. It doesnt have to be a formal beta process.
Another option here is to separate your development channels. If you have an auto-updater built into your software, make an option for the user to be notified of beta updates. Put a beta update online, and when you are satisfied with it, mark it as a release so that everyone else will get notified of the update. This way you do a ‘soft’ roll out, with a subset of the users getting the update, so if there is a problem, only 10% of your users are affected, not 100%.
Ideally, it should go as a trial feature.
If that feature is not requested by clients and did NOT have a UAT, it would be better to keep it as a Trial or Beta, without forcing clients to install it.
I don’t want to discourage beta-testing, but you might want to look into regression testing – to ensure that your “fix” doesn’t break anything else. I collect my test plans for each new feature, so that I can later retest that those features aren’t broken when new functionality is introduced. A basic plan that tests the most used and most important parts of the system, with links to detailed plans for specific parts of the system seems to work best, so that you can mix and match to focus your testing for a specific release.
If a full, written regression test is too much for you at this point, pretend to demonstrate the features of the software to an imaginary audience. I find more bugs that way…
1