I’m working in a software development team as software developer. I’ve been working on the same project for three years now. The software is a 32-bit desktop based C# application in .NET 4. Our target platform in Windows 7 (we had to support Windows XP till last year). The software communicates with various custom hardware for which custom drivers are written. The hardware manufacturing and driver software is written by our client. There is different driver for 32-bit and 64-bit Windows of course.
During our system testing phase, we execute all/most test cases in both 32-bit and 64-bit Windows 7. I can’t recall if we got any bug in our software that exist in only one flavor of Windows. Having this experience I’ve started to wonder, do we really need to test 32-bit software on 64-bit Windows?
What’s the industry standard?
2
Most of the bugs we encountered with running 32-bit software on 64-bit windows had to do with the location of the software (Program Files (x86)
instead of Program Files
), locations of registry keys (some were found in Wow6432Node). We had these problems mostly because we needed to communicate with other software (also 32-bit), and so we needed to test the software on both 32-bit and 64-bit…
When you didn’t have these problems, I believe it is quite safe not testing on both platforms when you explicitly compile in 32-bit mode. When compiled in 32-bit, the .NET runtime will run everything in 32-bit mode, and it should work the same as the 32-bit mode on 32-bit platforms.
According to 64-bit Applications (MSDN), 32-bit applications are run in Wow64 mode and Running 32-bit Applications (MSDN) explains this mode in more detail.
8
The hardware manufacturing and driver software is written by our client. There is different driver for 32 bit and 64 bit Windows of course.
So on 32 bit Windows, your software talks to one driver, and on 64 bit Windows, it talks to a different one? Let us assume there are new versions of these drivers from time-to-time. So when you only test your software on 32 bit Windows you cannot be sure there won’t be some differences in the 64 bit driver which will make the combination of your software + 64 bit driver fail. And from the viewpoint of your users, it does not matter who is to blame (you or the author of the driver), all they see is a non-working system. So even if your code is free of bugs, a test might reveal a bug in the 64 bit driver, and finding such a bug might help you take the right measures (like sending a bug report to the author of the driver).
Of course, when you have used those two drivers for years, and you are very confident that the behaviour is exactly the same, you might skip the tests for one platform, following the arguments in @DavidPerfors’s answer. As a compromise, you could run tests on 64 bit Windows only whenever a new driver version is available. Actually, this depends on the complexity of the drivers, your experience with and confidence in them.
Some additional things to consider:
- what OS type is your user base using most? 32 bit or 64 bit Windows? If you decide to test only on one platform, choose the one your users use most often.
- how severe is it when a new release of the software will not work on a less frequently used platform? For example, can your customers immediately step back and install the previous working release? Do they have only some inconvenience or real financial loss by this? If it is the former, testing on only one platform may be fine, if it is the latter, obviously not.
The default assumption in enlightened QA circles is “If you didn’t test it, then it doesn’t work”.
As a practical matter that’s usually an unreachable goal to strive for in much the same way application engineers might like to have unit-tests for everything; but they don’t believe they’ll ever reach it and release on schedule.
However your question can only be answered by, or in conjunction with, sales and marketing. You provide them with a cost to test and they provide an analysis of the market benefit. If estimations on both sides were sufficiently accurate, the answer would be simple
if B > C:
test_32bit_version()
In my experience, everyone’s cost estimates are inaccurate. As far as the other side of the equation, Dilbert once parodied decision making there with “I just asked my cat, Mittens”. To do much better, they’d need training in anthropological field methods.
1
Seeing as 99% of all Windows installations of Windows 7 and up, and a good portion of Vista as well, are 64 bit, why the hell would you even consider not testing for that platform?
It’s just a no-brainer, unless you’re making it specifically for a very limited group of users you KNOW are using 32 bit Windows and will continue to do so for the life of your product.
So yeah, test for 64 bit problems. In fact develop on 64 bit platforms, and probably supply a 64 bit version as standard with a 32 bit compiled version as an option for those few customers who’ve not upgraded to a new computer and OS in the last 6-8 years or so.
10
I would test any installer on as many different Windows setups as possible as from my experience the installers are most likely to fail on different systems.
Otherwise, as you know from your experience with the given software, bugs are most unlikely to just show up on 32-bit or 64-bit, and you can take some calculated risk.
Firstly, you should be having many test cycles with very little code changing between the later cycles as you get close to shipping. Any time you can save, you can use to create more test cases, and/or allow more (and hence smaller) cycles, so giving quicker feedback. (The risk of spending time testing X may be more than the risk of not testing Y, because you are testing X too much.)
Therefore
- Try to test on the “other bitness” to what you used for the proceeding test cycle.
- If you know the “bitness” your developer use, start with testing on other one.
- Divide the test cases up between the “bitness” to give come coverage on each
- But swap them between the “binnes” on each test cycle.
Nope. Likewise, when the FDA is done testing new medicines on mice and rats, they skip over testing on monkeys and just sell it for human consumption.
< /sarcasm>
Yes, yes yes yes yes. There is nothing but sadness in store for your software if you do not test every platform you possibly can. Things are always different, and the assumptions in the designer/coder’s head during the project usually only come passably close to modelling real life. So please, test your software. Please.