• Resources

  You need a test database to run the tests. The hardware to run the database is more expensive than using powermock. Releasing the resources used for unit testing against the database means the company doesn’t need to upgrade the server as soon.

• Reliability

  A test can fail because the database is down or in an inconsistent state. Guess what, your builds failed because the DBAs took the dev server down during the day to do some upgrades (and now the developers are trying to figure out what went wrong in the code – when its not the code).

• Additional maintenance

  Tests can be run in any order. Adding or removing a test should not cause the test suite to fail. Running against a database means there is some state somewhere (in the database). Typically, these tests require extra maintenance to ensure that the database maintains the proper state for the next test to run.

• Concurrency

  Two developers run the unit tests at the same time. Having a database means that the tests may collide on the database. The solution to this would be to clone the database for each test that is run, which is prohibitive on a real database. Which leads to another option to consider.

Consider also using HSQLDB with whatever dialect of database you are using. This way one can create an in memory database for each test. A unit test runs, constructs the necessary database, loads the data, the database connection from the code connects to HSQLDB and runs against the test data.

My questions are, what are the REAL world tangible benifits of proper unit testing I can use to push back?

Good unit tests are (per Clean Code, and elsewhere):

• Fast
• Independent
• Repeatable
• Self-Validating
• Timely

Not having true unit tests violates the first three (and usually the last one). This leads to some pretty significant problems:

1. Your tests are slow. Slow tests get run infrequently. Tests that aren’t run are almost useless.
2. Your tests can fail due to reasons unrelated to the code you’re testing. This makes it far harder to find out why tests have failed, wasting time and leading people to blame the environment rather than the code.
3. As the database changes, your results change. Getting some database in a known good consistent, maintained state for tests is tedious, annoying and error-prone.

In general, a lack of isolation in unit tests leads to worse tests which take longer to write, longer to investiage, and provide less confidence in your codebase. That in turn leads to people writing less tests or ignoring the tests more which is the downward spiral to chaos.

Unit tests are just a tool used to help a developer deliver reliable code. If you think like your boss thinks, you will be able to convince him if what you are proposing makes sense. However, if you present it as an evangelist on a band wagon, you won’t get the resources allocated. You will need to explain to him how he benefits by spending the time and resources retrofitting unit tests to you legacy application.

You mentioned legacy tests – that implies legacy code. Unfortunately, fitting unit tests to code that was not designed to be unit tested is a difficult, time consuming and expensive process. The business case get even harder if you have tests that provide useful results, and all you are going to achieve is (from the business case POV) alternate tests providing the same results. You will need to focus on the cost(time) savings…..

My guess is that you will not be able to present your boss with a robust business case because there is not one.

From what you described it sounds like you don’t like test framework A and want to switch to test framework B, both of which work. Suck it up and use what is existing and working, don’t create more work reinventing the wheel so you can use your preferred test framework.

It takes more than a want to use the latest flavor of the month framework to justify throwing out existing working code and spending huge amounts of time and money for essentially zero benefit to users.

Good unit tests provide localization. Functional tests might tell you “the add user function is broken”, but a good unit test will tell you that it’s broken because somebody changed the USER.LAST_NAME field in the database to be non-null. It’s also possible to test small changes directly, instead of needing a complex test environment (which may need to be re-initialized or purged for some tests).