According to Why is Global State so Evil?, I know one of the disadvantages of “global state” is that it makes code “harder to test”. I do not disagree with this, but what I don’t understand is, why would it be “harder to test”?
I know there are some answers talking about “harder to test”, for example:
-
https://softwareengineering.stackexchange.com/a/148109 : Unable to run tests with starting state, so the next test may start from altered global state and hence the value to assert is unknown
-
https://softwareengineering.stackexchange.com/a/148188 : After running a test, the global state changes, the next test would run using the altered state instead of initial state
But the premise of those answer is: after I test a method with a global state, the state is changed and the next test must use the altered state instead of the original state. My question is, why do the answers assume that? If I need to test a function with a global state, doesn’t “setting the global state at the beginning of each test” already solve the problem? eg:
public void testCalculateTotalPrice(){
UserData.instance=new UserData(); //reset the global state
//assert(this.calculateTotalPrice(10 //price of the product,3 //quantity),1200); // calculate would use UserData.isMember : member have discount
}
And I know there are other answers about global state which say we cannnot easily switch an implementation:
https://softwareengineering.stackexchange.com/a/148154 : unable to switch to test database from other database
However, I think the “unable to switch” problem is not caused by global state: I may create a switchable global state, eg:
public class UserData{
IDatabase instance;
}
public interface IDatabase{
}
public testMethod(){
UserData.instance=new TestDatabase();
this.method(); //method that would use UserData.instance
}
So “global state” doesn’t mean “unable to switch”.
While “global state” may have tons of disadvantages, “harder to test” seems not to be among it, because I can still test a method depending global state with its initial state by reverting the global state to the initial state at each time testing. Also I can setup a replaceable global state to switch implementation. Why would we say “global state” is “harder to test”? Do I have a misunderstanding about testing methods with global states? Or do I have some misunderstanding about testing?
Note: I guess there would some people argue that when I use global state, besides the initial state, I need to test other possible states, eg:
public class UserData{
public boolean isMember;
public boolean isVIP;
public static UserData instance;
}
public void testCalculatePrice(){
UserData.instance.isMember=false;
UserData.instance.isVIP=false;
assert(this.calculatePrice(10,3),1200);
}
So you may complain that this test only tests
UserData.instance.isMember=false;
UserData.instance.isVIP=false;
but not other cases such as
UserData.instance.isMember=true;
UserData.instance.isVIP=false;
But if you eliminate the global state with “parameter version”:
public void testCalculatePrice(){
assert(this.calculatePrice(10,3,false,false),1200);
}
I guess you would not complain why I don’t have:
assert(this.calculatePrice(10,3,false,true),1200);
assert(this.calculatePrice(10,3,true,true),1000);
assert(this.calculatePrice(11,3,true,true),1100);
right? And even if you complain, I believe the source of problem is not because it is using global state, but you missed some other important test cases: the problem of “need to test all possible combinations of global states” is equivalent to “need to test all possible combinations of values of parameters”.
Yes, resetting global state would ensure the method works with specific global state only, not other global state , but “parameter version” also ensure the methods work with specific parameter values only, why would we challenge the ability of covering test case in “global state” versions, but not in “parameter” version?
So the question is, why does using global state require us to test all possible states, but not in “parameter version”?
6
How do you know where all the global state is in your application?
When you have a function which does not use global state, all the information required for test state is in the parameters. When a function does use global state, there is no easy way (at least not in any language I’ve seen) to get a list of relevant global state so you can reset it. And in a large codebase which doesn’t have a “no global state” rule, the amount of global state may become very large.
Sometimes the global state simply isn’t resettable from the outside either, because it’s private.
2
If you reset the global state before each test, each test will execute in a known global state and will, therefore, be consistent. Trouble is, that’s not how it will be executed in the real application.
Now, you could make the argument that this is “OK” for Unit Testing.
When invoked in a known (single) state, the code works. All the real application stuff falls under “Integration” Testing, and that’s the Testers’ job, right?
I would disagree.
Instead, you could consider the code a “fixed point” and that you need Unit Tests that “cope” with how that global state changes. That way, you could be confident that each, individual invocation would work and your “Integration” Testing piece would just be “bolting together” these individual cases. Everybody wins.
The “trick”, though, is that you don’t know how or when that global state is going to change. Being manipulated by code outside of your control it could, potentially, even change during the execution of any given test! Talk about building on shifting sands.
7
My question is, why would the answers assume that?
Because it is the default unless you change it.
If I need to test a function with a global state, isn’t just “restoring the global state at the beginning of each test” already solve the problem?
Yes… if you never run tests in parallel.
And if you are one of the people who say “why would I run them in parallel”, I can tell you that you have never actually written tests for a larger application. Because you can easily cut the time your tests take by 95% if you write them to run in parallel on todays processors.
You would not accept single-threaded applications from any other vendor, why write your own software that way?
7
In an application, you usually start with global state initialised from settings and preferences, and then you add more state as the application is running.
When you call a method in real code, it assumes that the global state is there. Some is required for the method to work, some influences what it does. (You may say that’s a bad idea, but if we assume you have global state then I assume you use it)
If you just reset global state, you have two problems: One, your method may not be working. Two, you only test how the method works with one specific global state. So to test the method properly, you’d have to set up each legal global state and call the method repeatedly with each state. So it’s a bit of a mess.
2
One thing i don’t see anyone else bringing up here is the set of unique bugs that global state makes more likely. Especially raceconditions become a lot more likely to occur in an application with global mutable state. Raceconditions are notoriously difficult all around, and writing a automatic test suite that reliably protects against them is somewhere between almost impossible and actually impossible. Now, testing to ensure you don’t have raceconditions is difficult in an application that doesn’t have global state too, but you are much less likely to run into them, and therefore don’t have the same need to test for them.
Basically, global states puts you in a place where you need to test for raceconditions, and that’s hard
3
I think you’re conflating two things, basically the words “any” and “all”.
Does there exist global state that can be tested in the way you describe?
Yes. In a suitable language you could define a class TestableGlobal, which holds a value with getter and setter, plus a “reset” operation. Also, on creation, instances register themselves in some kind of collection. Then, the test just has to call some static function clear_all_testable_globals, which iterates over every instance of TestableGlobal, and resets them all.
Then, the code under test can safely use global data, provided it is all managed using TestableGlobal.
If we press this TestableGlobal concept a bit further, we can probably get to 85% of the functionality of a dependency injection framework with 15% of the work. Or, we could do what the thread you’re reading already suggests, and use a dependency injection framework.
Can all global state be tested in the way you describe?
No, because in the massive variety of programming languages that we might be talking about, and features of those programming languages that the code under test might use, there are plenty of forms of global state that the test code cannot easily access in order to reset them, or which can only access given extremely detailed knowledge of the implementation of the code under test, which makes the tests very far from black box testing.
In the example you give, suppose that the code under test starts using a second global. Then, the test doesn’t work any more, but the code hasn’t changed its public interface in any way. This is a very bad state of affairs: we much prefer to write tests that don’t rely on the test code “knowing” the non-public implementation details of the code under test. So, by using globals, the code is unarguably “harder to test”. We have to do work to fix the tests when we change the code. Things that required more work are by definition “harder” than things that require less work!
For another example, suppose the system has an attached radio telescope, and the code under test moves the telescope from its “rest position” pointing due upwards, to point at the Andromeda galaxy. Then the test code needs to know how to rest the “global state” of the radio telescope: that is to say the test code needs to call a function to move the radio telescope back to point due upwards. At some point in this process, perhaps when your test run is keeping you awake due to the noise from the telescope motors, will you not think to yourself: “using an injected dependency for the radio telescope, instead of global state, so that most of my tests can be run using mocks, would be easier than this”? And that’s just when Andromeda is above the horizon.
1
Trusting a global is like eating off the floor.
why would it be “harder to test”?
Because a global is known, globally.
doesn’t “setting the global state at the beginning of each test” already solve the problem?
Sure. Right up until it doesn’t. Wash the floor all you want. Soon as you run some code lord knows who has walked all over your “clean” floor.
Whatever else a global is, it isn’t yours. You have no reassurance over the life it leads. Even if you trace everything that refers to it today who knows what knows about it tomorrow.
You have to depend on things to get things done. We’ve proven over and over that when you do that it’s best to make what you depend on yours. Get your own copy of it that nothing else knows about. Now you know where that floor tile has been.
There are already good answers, but one additional consideration for some cases:
Global state management (reset and access) likely will require global initialization, i.e. to first reset the global state you might have to initialize your full application – which could be a massive multi-component server that takes a minute to boot up (exaggerating…perhaps…) just to reset the global state. If you want to test a small piece of code that could run isolated when it wouldn’t rely on global state, no need to first setup the whole application.
I don’t think there is one universal understanding of what “global state” even is.
I suppose it is commonly used as a synonym for “global variables” – that is, a facility offered by certain programming languages where fields exist in a “global scope”, are allocated automatically at the start of runtime, and are accessible from anywhere in the program.
By definition, these global variables cannot be reallocated at run time, access cannot be intercepted or diverted at all, and access can originate from anywhere in the program without further ado and without any possible constraint.
This doesn’t make global state untestable, but the inability to easily intercept or divert the access makes it more difficult to apply certain testing methods which depend on presenting certain carefully-set testing data, or on immediately checking the correctness of what is submitted.
The unrestricted access also creates widespread and direct dependency on certain global fields,
This facilitates (though it does not inherently cause or require) the disorderly design of data flows that make it difficult to divide the program into parts and test them severally – the assumption being that it is easier to test parts than to test the whole.
The direct dependency on global fields (even if they are pointers that can be intercepted/diverted) also means these fields must be reproduced in a test environment – this can make it more difficult to set up a test environment as the linkage has to occur at compile time.
The alternative to unrestricted global access is to have some feature of the code that explicitly distributes access to certain fields.
Access can still ultimately be distributed globally (so it doesn’t automatically solve disordered design which is difficult to test in separate parts), but the important difference for testing purposes is that, when under test, something other than the normal thing can be distributed around instead – something like a specially prepared mock – and the linkage to this thing can be established at runtime rather than compile-time – so for example, a precompiled library can be put straight into a new test environment, without recompiling the library source code together with the test environment incorporated.
It’s important to remember that the difference is only about how difficult certain testing methods and styles end up being. The use of global variables alone is not the difference between some piece of code being completely and easily testable, and being completely untestable by any means.
18
Why is global state hard to test?
Global state is a subtle way of coupling every function that uses it. Setting the state before each test case is all fine and well, but guess what’s not happening in the running production code: the global state is typically not being neatly reset before every function call.
The order of operations of a running program can be hard to understand as it is, but on top of that, some of the functions to be called may be determined at runtime, by user actions. As a result, you’ll run into bugs that are due to global state being altered by one function in a way that some other function didn’t expect, messing up your code in ways you didn’t even think of. The problem with global state is that, when you’re implementing any particular function, too many things can go wrong with other (ostensibly unrelated) functions because of reasons you initially aren’t aware of, and soon you have this web of complexity that will make working in this codebase a miserable experience. It’s going to turn into the “if you touch anything at all, something might break” type of codebase. The problem is that if you’re not very careful, global state soon severely limits your ability to reason about the correctness of a function based on only the code in the function itself.
These bugs may be hard to replicate, and my only happen intermittently, after the user performs some very specific, obscure sequence of operations. These bugs are hard to test in a broader sense of the word: they are hard to reproduce, hard to debug and figure out, hard to fix or mitigate cleanly without accidentally breaking something else, and can also be hard to simulate/capture in unit tests.
Doesn’t setting the global state at the beginning of each test solve the problem?
While setting global state at the beginning of each test case will let you test your functions under ideal conditions, the above means that there’s a whole class of bugs that you’re not covering at all, and that you might not even be able to cover effectively because of how convoluted the circumstances that give rise to such bugs can be. Simply having some tests running is not the point – the point is to have a suite of tests that gives you confidence that (1) your code does what it is meant to do, and (2) still works correctly after changes are made to a part of the system.
I believe the source of problem is not because it is using global state, but you missed some other important test cases: the problem of “need to test all possible combinations of global states” is equivalent to “need to test all possible combinations of values of parameters”.
From a high level point of view, yes, global state can be seen as input to your function, but the devil is in the details. It’s not necessarily about a particular global state being invalid as input, it’s that there will be a bunch of unexpected flows of logic spanning several functions, resulting in sequences of states changing over time, that will lead to unexpected or unpredictable results.
A certain global state might be a perfectly valid input for a given function, and the function might correctly compute what it was supposed to compute with such input. Still, the unexpected thing might be that you arrived at that state in the first place, because that wasn’t supposed to happen in a given scenario. E.g., maybe your user can’t click on a menu item, because it’s disabled, when in fact it shouldn’t be. There’s nothing wrong with the function that is in charge of the menu. There’s something wrong somewhere else in the logic flow, and it may be a result of an “unrelated” bugfix you did three weeks ago. You’ll be banging your head trying to figure out what sequence of events lead to such an outcome, and the bug will be in some bizarre place, not in the menu code, and not in the bugfix code.
With a function (or an object) that avoids globals and has well-defined inputs and outputs, and well-defined behaviors, you can reason about its correctness locally (without worrying about other code), and you can write a more effective, more structured set of tests. Furthermore, going one level higher, when reasoning about the correctness of the code that calls such functions, you don’t have to go back one level down to look inside those functions in order to understand what is going on.1 Now apply this idea recursively all the way up. If done well, many of the aberrant flows of logic that can arise with global state will not be possible by design. If there is a bug, you’re way more likely to find it in a place that makes sense, and fix it without affecting anything else.
1 To some people, this may sound like a pipe dream, but consider that you do it all the time when calling library functions. Imagine that simply calling a library or a framework function had a potential to break your own code somewhere else, and that then you had to look at the freakin source of the library to figure out why. Generally, you don’t know or care how those library functions are implemented, you know what they do in the high level sense, and that’s enough for you to understand the workings of your own function that utilities them. Well defined inputs, well defined outputs, predictable behavior. And the library itself is encapsulated and has no direct access to your own state, it only interacts with your code in a very structured way. Well, write your own code with a similar philosophy, in a smaller scale. As a rule of thumb, strive towards being able to treat calling your own code as calling a library function that’s safe and predictable.
5