Acceptance tests act at a very different level than, say, unit tests.

A unit test is very precise: it deals with one method, sometimes a part of a method. This makes it a perfect choice for regression testing. You make a change. A test fails while it passed during the previous commit. Great, you can easily pinpoint the source of the regression both in time (from commit N-1 to commit N) and space (this method of this class).

With acceptance tests, good luck if some start to fail. One bad change may cause one acceptance test to fail, or maybe ten tests, or a hundred. When you look at those hundred tests turning red without giving any hint about the location of the bug, the only thing which comes to mind is to revert to previous commit and start over.

Imagine acceptance tests as tests which consider the system as a black-box and which test a feature of the system. The feature may involve thousands of methods to be run, may rely on a database, a message queue service, a few dozen of other things. An acceptance test doesn’t care how much stuff is being involved, and don’t all the magic which happens behind the scenes. This also means that multiple acceptance tests may rely on the same method, which in turn means that a regression in a single method often leads to several failed acceptance tests. I had cases where a regression caused suddenly approximately fifty system and acceptance tests to fail, without giving any hint about the location of the bug.

Unit tests consider the system as a white-box. They are aware of the concrete implementations, and test a specific method, not a feature. By using mocks and stubs, unit tests achieve enough isolation to not being affected by the world: if the method works, the tests succeed. If the method has a regression, those tests fail. If another method somewhere in the code base doesn’t work, those unit tests still pass.

Imagine the following diagram which represents methods calling other methods:

Unit testing can be represented this way:

For example, unit tests U3 and U3′ are not affected by bugs in method 6, because it is replaced by a stub. Nor is it affected by the regressions in method 2, because the stub 5 doesn’t rely on method 2. In the same way, U8″ doesn’t care about method 7, because 9 is a stub and doesn’t rely on 11 which in turn uses 7.

Imagine you make a commit and your CI informs you that U8′ now fails. Where would you search for the problem?

• In U8′. Maybe the code it tests is correct, but the test is not. This happens, for instance, when requirements change: you change the code but forget to reflect the change in the tests.

• In method 8. There could be a regression.

• In stub 9. Maybe you implemented a change in code and in tests, but forgot to change the stubs.

On the other hand, you are sure the problem is not with method 11 or method 7.

Now this is how acceptance or system tests would look like:

Imagine you make a commit and your CI informs you that A/S2 failed. Guess where is the problem? Much more difficult, isn’t it.

Obviously, another aspect is that, as I described above, a small change can cause many acceptance tests to fail. For instance, a regression in method 7 may lead to A/S1 and A/S2 to fail. With unit tests, a regression in method 11 may cause U11 to fail, but will not affect, for instance, U8.

This leads to a huge benefit: the time you spend locating regressions. I’ve seen programmers who, when a regression is found, simply revert the source to the latest working commit and start over, because the code is a mess, and they don’t enjoy spending hours debugging, hoping to find the origin of the problem. This is unfortunate, especially when commits are done not as frequently as they can be.

With unit tests, you don’t waste all this time. They simply tell you that you have an issue with a given method in a given class, so you can focus your attention to the concerned method right after you discover the regression.

• Unit tests ensure things at a lower level are correct: Who cares? if the business is happy with what they’ve received, then what difference does it make? […]

Have you worked with really bad projects where you can’t make a change without breaking at least ten things in random locations?

Unit tests don’t magically solve this problem. However, you should care that things are correct at lower level. Quick hacks have a substantial maintenance costs. On the other hand, if the only thing programmers have are acceptance tests (and tight deadlines; and no insensitive to do their job correctly), when an acceptance test fails because somewhere, the font should be 12px, but appears to be 10px, they may eventually just end up writing:

this.font = 12;

and wait until testers get back to them telling that now, the font is 12px in situations where it should be 14px.

• Unit tests speed up long-term development time: Is it really worth the trade-off? Where there is code there are bugs, tests are not exempt from this. […]

Tests won’t magically make bugs go away. However, practice shows that writing a test, checking that it fails, then implementing the feature and checking that the test passes appeared to be a practice which generally leads to less bugs.

Similarly, why would anyone do code reviews? Reviewers are not exempt from lack of attention, and there are plenty of cases where a bug is missed by several reviewers and maintainers. This being said, you get more bugs without code reviews than with them.

Is it worth it? For your personal small app, not really. For business-critical code, yes, indeed.

• Unit tests ensure good architecture: No they don’t. […]

They do, somehow. By forcing to test methods in isolation, you force programmers to rethink coupling. Unit testing usually leads to short methods, classes which do one and one only thing (Single responsibility principle), dependency injection, etc.

A 4000 LOC method which relies on a few hundred other methods and requires access to the database is impossible to unit test (while it is perfectly normal to add acceptance tests to such method).

In Going TDD in the middle of the project, I describe specifically this sort of projects. Bad architecture led to the case where it was practically impossible to add unit tests later. If unit tests were considered from the beginning, it would reduce the disaster. The code would still be bad (since written by the same programmers who were fine writing a 400 LOC spaghetti method), but not that bad.

• Integration tests ensure the overall performance is accurate and optimal […]

Don’t know. Never heard of that. Performance should be checked by tests corresponding to the performance non-functional requirements.

• Functional tests ensure that output is as expected […]

Don’t know.

• TDD ensures that when changes are implemented, nothing else breaks in the process: Acceptance tests will ensure this too since everything’s based on business requirements/features. […]

No, this is the role of regression testing. TDD’s purpose is mainly to avoid writing tests which are not testing anything. This is why in TDD, the test should fail before you implement a feature: otherwise, either the test is wrong, or you actually don’t need the feature.

There are several testing-related questions:

• Which parts to test? Units (unit testing), whole system (functional), several systems together (integration)
• Which aspects to test? Functionality, performance, reliability, security, etc.
• When to do testing? Before (TDD/BDD) or after (non-TDD/BDD). Or never =)
• How deep to test? Or, how to view the system under test? “Black box”, “grey box”, “white box” approaches.

Also, TDD and BDD are the ways to design software and test it. Two stones at one bird!

Examples:

• I have a REST API. I write tests using Cucumber, viewing system as a black box, and I have a full spec which guides my testing process. This way I’m doing acceptance testing, but not BDD, since I do not design through testing. Design is in the spec.
• I have some rocket science software, I need to test a bunch of functions already written in Haskell. I write unit tests, but not TDD. That could be TDD if I’d write tests upfront.
• I want to write yet another DOM manipulation framework in JS. I use Jasmine framework to write tests. I test the framework as a black box. In this case I do BDD. And also it is functional testing.

Also I have some notes and clarifications about what you state:

• Unit tests ensure things at a lower level are correct

  Kinda, but not really. They just show the lack/presence of known bugs. If you’re interested in correctness, you may want to look at formal verification.

• Unit tests speed up long-term development time

  Not always, because poorly-designed systems can be hard or impossible to test. In such cases I believe it is not worth it. That’s why TDD was invented.

  If system is testable, it does worth a lot. Errors cost much less when caught early. They may cost too much then they are discovered in production phase.

• Unit tests ensure good architecture: No they don’t.

  Indeed they don’t. TDD and BDD do.

• Integration tests ensure the overall performance is accurate and optimal

  Not only. Functionality and reliability too. Two systems may pass their “local” tests and yet fail to work together. Integration testing shows how well parts work together.

• TDD ensures that when changes are implemented, nothing else breaks in the process

  Secretly, all types of automated testing do that. Showing “regression bugs” that is: module (or function, or system) A has tests over it, then you change module B, then A breaks, then you see it through the tests on module A and fix module A.

BDD actually started at the class level. JBehave was originally intended to be a replacement for JUnit.

The only meaningful difference between JBehave and JUnit back in 2004 was the removal of the word “test”, and the use of “should” to drive out different aspects of behaviour of the class and encourage questioning of those aspects of behaviour (“should it?”).

Conversations between Dan North (who created BDD) and Chris Matts (at the time, an analyst who was learning more about code and particularly mocks) revealed that the same patterns could be used at a system level, and the idea of using “Given, When, Then” to automate reusable steps was born.

Rather than thinking of them as “tests”, Dan encouraged developers to think of them as examples of how a class behaved, or how a system behaved. We called the system ones “scenarios” mostly to differentiate from the class-level “examples”, but both are synonyms.

The steps were created according to the context in which the class or system worked (the Given) and how those contexts changed the outcome (the Then). This pattern let people question whether the contexts needed to be considered, or whether an outcome was desired or not.

The different levels are appropriate for different audiences. Conversations about class behaviour typically take place between developers, while conversations about system behaviour involve the “three amigos” of tester, developer and business expert.

It’s often the case that developers are aware of more requirements than the business, from stakeholders who aren’t always present. For instance, devs will normally consider maintainability, performance, the APIs of third-party or legacy systems, themselves coming back to the codebase in 3 years time or new joiners coming on board, etc.

All of these are parts of the requirements which the business often don’t care about, as long as it works. Talking through the behaviour at a technical level can give the devs a chance to question whether what they’re doing is appropriate.

Some small projects may not see much benefit from this, particularly if the functionality of the project is simple and predictable. Large projects benefit from the “Test Pyramid”: few full-stack system tests, more integration tests, and even more class-level tests. See Lisa Crispin and Janet Gregory’s “Agile Testing” for more information.

As an example, this is a class-level test which uses “Given, When, Then” in comments to illustrate the waiter’s responsibilities and behaviour (C# but readable).

I find it helps if I don’t think of them as tests, though. I think of them as examples which illustrate how the system behaves. The idea isn’t to catch bugs; it’s to prevent bugs from scenarios which haven’t been considered from being there in the first place. Any regression bug is usually a symptom of poor design, and benefits from class-level refactoring and testing instead of shoving another system-level scenario into the mix.