I have a locust-based project running multi-process with multiple users, and with a custom LoadTestShape.

Some of my tasks can throw exceptions (e.g. VeryBadException) which cannot really be recovered from, and I would like the entire locust run to terminate as soon as possible. However, I’m failing to come up with a clean way to achive this.

Simply letting the exception be raised out of the @task is insufficient – the exception is reported by locust.user.task and execution continues:

@task
def request(self):
   ...
   raise VeryBadException()

I tried also raising StopUser and other locust exceptions but to no avail.

I also tried adding a call to runner.quit() before raising the exception:

@task
def request(self):
   ...
   self.environment.runner.quit()
   raise VeryBadException()

This does work in single process mode, but not with --processes=2 or more – locust just continues running. For example with --processes=2 --users=2 and injecting the exception once in the first user to test the behaviour:

[2024-06-07 12:06:32,922] localhost/INFO/root: Waiting for workers to be ready, 0 of 2 connected
[2024-06-07 12:06:32,922] localhost/INFO/locust.runners: Worker localhost_9ddc0e0488ca4bd49437132a4f6754fe (index 0) reported as ready. 1 workers connected.
[2024-06-07 12:06:32,923] localhost/INFO/locust.runners: Worker localhost_214704789d5243d3b3cc10dfa083e27d (index 1) reported as ready. 2 workers connected.
[2024-06-07 12:06:32,923] localhost/INFO/locust.main: Starting Locust 2.28.0
[2024-06-07 12:06:32,923] localhost/INFO/locust.main: Starting Locust 2.28.0
...
[2024-06-07 12:06:37,016] localhost/INFO/locust.runners: All users spawned: {"PopulateUser": 0, "RunUser": 2, "SetupUser": 0} (2 total users)
Traceback (most recent call last):
  File "users.py", line 963, in do_run
    raise Exception("Test BadThing happened")
Exception: Test BadThing happened
[2024-06-07 12:06:37,218] localhost/INFO/locust.runners: Sending spawn jobs of 2 users at 2.00 spawn rate to 1 ready workers
[2024-06-07 12:06:37,218] localhost/INFO/locust.main: Shutting down (exit code 0)
[2024-06-07 12:06:38,219] localhost/INFO/locust.runners: Spawning is complete and report waittime is expired, but not all reports received from workers: {"PopulateUser": 0, "RunUser": 1, "SetupUser": 0} (1 total users)
[2024-06-07 12:06:38,219] localhost/INFO/locust.runners: Worker 'localhost_9ddc0e0488ca4bd49437132a4f6754fe' (index 0) quit. 0 workers ready.
Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
Search   /                                                                                209     0(0.00%) |     40      35     120     39 |   10.67        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                       209     0(0.00%) |     40      35     120     39 |   10.67        0.00
... continues running ...

For comparison without --processes (single-threaded) which works:

[2024-06-07 12:09:42,164] localhost/INFO/locust.runners: Shape test starting.
[2024-06-07 12:09:42,164] localhost/INFO/locust.runners: Shape worker starting
...
[2024-06-07 12:09:45,175] localhost/INFO/locust.runners: All users spawned: {"PopulateUser": 0, "RunUser": 2, "SetupUser": 0} (2 total users)
Traceback (most recent call last):
  File "users.py", line 963, in do_run
    raise Exception("Test BadThing happened")
Exception: Test BadThing happened
[2024-06-07 12:09:45,376] localhost/INFO/locust.main: --run-time limit reached, shutting down
[2024-06-07 12:09:45,376] localhost/ERROR/locust.runners: While stopping users, we encountered a user that didn't have proper args <Greenlet "Greenlet-3" at 0x14d82ea20: _run>
[2024-06-07 12:09:45,379] localhost/INFO/locust.main: Starting Locust 2.28.0
[2024-06-07 12:09:45,379] localhost/INFO/locust.main: Shutting down (exit code 0)
Type     Name                                                                          # reqs      # fails |    Avg     Min     Max    Med |   req/s  failures/s
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
Search   mnist                                                                              2     0(0.00%) |    102      59     145     60 |    0.62        0.00
--------|----------------------------------------------------------------------------|-------|-------------|-------|-------|-------|-------|--------|-----------
         Aggregated                                                                         2     0(0.00%) |    102      59     145     60 |    0.62        0.00

Response time percentiles (approximated)
Type     Name                                                                                  50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
Search   mnist                                                                                 150    150    150    150    150    150    150    150    150    150    150      2
--------|--------------------------------------------------------------------------------|--------|------|------|------|------|------|------|------|------|------|------|------
         Aggregated                                                                            150    150    150    150    150    150    150    150    150    150    150      2
<EXIT>

What’s the correct way to achieve an immediate shutdown of locust in this scenario?