I have a RuleBasedStateMachine
which took about 40 seconds to run before. We then had a change that increased the amount of time taken in one of the steps and introduced a bug.
Because of the performance penalty, it would normally take 90 seconds to succeed but if you were unlucky it could take more than 5 minutes. If you then managed to trigger the bug, the test would shrink for more than an hour.
We managed to figure out what was wrong with @hypothesis.settings(verbosity=Verbosity.debug)
, but it took time to pinpoint which test was failing because our test runs would timeout after an hour.
Is there any way of setting a timelimit on the amount of time spent shrinking?