I am a bit lost here, I am totally new to BenchmarkDotNet and trying to create a working benchmark with nice rendered output.
I think I have the output part covered.
However I did hit a view walls and every time I thought I had found a solution another problem pops up.
But first things first.
I want to test is the performance of collections (SortedDictionary, ImmutableSortedDictionary, SortedList) for different scenarios.
For the beginning I did start with an ItemInsert tests.
I wanted to know how fast can I insert 10 randomly generated items into an existing collection of 0, 10, 100, 1000, 10_000, 100_000, 1_000_000, 10_000_000 and 100_000_000 items.
Problems I ran into (Related to BenchmarkDotNet not shortcommings of generic datatypes in general 😉 )
Benchmarks are not isolated.
I ran into this when trying to prefill the collections in the GlobalSetup.
After the first Benchmark the next benchmark gets the collection with the itemd added by the first benchmark. Thats not what I want.
So where is the place to prepare the collection to test? IterationSetup would be an option but there are warnings everywhere not to use it and also rebuilding the collection before each benchmark would be damn slow. And doing it in the Benchmark itself would mess with the measurement I guess.
So how do I write a benchmark that prefills a collection with items and uses exactly this prefilled list for all benchmarks without the need to regenerate the collection before each benchmark.