So, I have an authentication module I wrote some time ago. Now I’m seeing the errors of my way and writing unit tests for it. While writing unit tests, I have a hard time coming up with good names and good areas to test. For instance, I have things like
- RequiresLogin_should_redirect_when_not_logged_in
- RequiresLogin_should_pass_through_when_logged_in
- Login_should_work_when_given_proper_credentials
Personally, I think it’s a bit ugly, even though it seems “proper”. I also have trouble differentiating between tests by just scanning over them (I have to read the method name at least twice to know what just failed)
So, I thought that maybe instead of writing tests that purely test functionality, maybe write a set of tests that cover scenarios.
For instance, this is a test stub I came up with:
public class Authentication_Bill
{
public void Bill_has_no_account()
{ //assert username "bill" not in UserStore
}
public void Bill_attempts_to_post_comment_but_is_redirected_to_login()
{ //Calls RequiredLogin and should redirect to login page
}
public void Bill_creates_account()
{ //pretend the login page doubled as registration and he made an account. Add the account here
}
public void Bill_logs_in_with_new_account()
{ //Login("bill", "password"). Assert not redirected to login page
}
public void Bill_can_now_post_comment()
{ //Calls RequiredLogin, but should not kill request or redirect to login page
}
}
Is this a heard of pattern? I’ve seen acceptance stories and such, but this is fundamentally different. The big difference is that I’m coming up with scenarios to “force” the tests out. Rather than manually trying to come up with possible interactions that I’ll need to test. Also, I know that this encourages unit tests that don’t test exactly one method and class. I think this is OK though. Also, I’m aware that this will cause problems for at least some testing frameworks, as they usually assume that tests are independent from each other and order doesn’t matter(where it would in this case).
Anyway, is this an advisable pattern at all? Or, would this be a perfect fit for integration tests of my API rather than as “unit” tests? This is just in a personal project, so I’m open to experiments that may or may not go well.
5
Yes, it is a good idea to give your tests names of the example scenarios you are testing. And using your unit testing tool for more than just unit tests maybe ok, too, lots of people do this with success (me too).
But no, it is definitely not a good idea to write your tests in a fashion where the order of execution of the tests matters. For example, NUnit allows the user to select interactively which test he/she wants to be executed, so this will not work the way intended any more.
You can avoid this easily here by separating the main testing part of each test (including the “assert”) from the parts which set your system in the correct initial state. Using your example above: write methods for creating an account, logging on and post a comment – without any assert. Then reuse those methods in different tests. You will also have to add some code to the [Setup] method of your test fixtures to make sure the system is in a properly defined initial state (for example, no accounts so far in the database, noone connected so far etc.).
EDIT: Of course, this seems to be against the “story” nature of your tests, but if you give your helper methods meaningful names, you find your stories within each test.
So, it should look like this:
[TestFixture]
public class Authentication_Bill
{
[Setup]
public void Init()
{ // bring the system in a predefined state, with noone logged in so far
}
[Test]
public void Test_if_Bill_can_create_account()
{
CreateAccountForBill();
// assert that the account was created properly
}
[Test]
public void Test_if_Bill_can_post_comment_after_login()
{
// here is the "story" now
CreateAccountForBill();
LoginWithBillsAccount();
AddCommentForBill();
// assert that the right things happened
}
private void CreateAccountForBill()
{
// ...
}
// ...
}
1
A problem with telling a story with unit tests is that it doesn’t make explicit that unit tests should be arranged and run entirely independently of each other.
A good unit test should be completely isolated from all other dependent code, it’s the smallest unit of code that can be tested.
This gives the benefit that as well as confirming the code works, if a test fails you get the diagnosis for exactly where the code is wrong for free. If a test isn’t isolated you have to look at what it depends on to find out exactly what’s gone wrong and miss out on a major benefit of unit testing. Having the order of execution matter can also raise a lot of false negatives, if a test fails it’s possible for the following tests to fail despite the code they test working perfectly fine.
A good article in more depth is the classic on dirty hybrid tests.
To make the classes, methods and results readable the great Art of Unit testing uses the naming convention
Test Class:
ClassUnderTestTests
Test Methods:
MethodUnderTest_Condition_ExpectedResult
To copy @Doc Brown’s example, rather than using [Setup] which runs before each test, I write helper methods to build isolated objects to test.
[TestFixture]
public class AuthenticationTests
{
private Authentication GetAuthenticationUnderTest()
{
// create an isolated Authentication object ready for test
}
[Test]
public void CreateAccount_WithValidCredentials_CreatesAccount()
{
//Arrange
Authentication codeUnderTest = GetAuthenticationUnderTest();
//Act
Account result = codeUnderTest.CreateAccount("some", "valid", "data");
//Assert
//some assert
}
[Test]
public void CreateAccount_WithInvalidCredentials_ThrowsException()
{
//Arrange
Authentication codeUnderTest = GetAuthenticationUnderTest();
Exception result;
//Act
try
{
codeUnderTest.CreateAccount("some", "invalid", "data");
}
catch(Exception e)
{
result = e;
}
//Assert
//some assert
}
}
So the failing tests have a meaningful name that gives you some narrative about exactly what method failed, the condition and the expected result.
That is how I’ve always written unit tests, but a friend has had a lot of success with Gerkin.
1
What you’re describing sounds more like Behavior Driven Design (BDD) than unit testing to me. Take a look at SpecFlow which is a .NET BDD tech that is based on the Gherkin DSL.
Powerful stuff that any human can read/write without knowing anything about coding. Our test team is enjoying great success leveraging it for our integration test suites.
Regarding the conventions for unit tests, @DocBrown’s answer seems solid.
1