You really want to be looking at separating the concerns here — have your web app make requests to a message queue, have the execution components process messages from the queue. This also makes security much less nightmarish because you can push the things manging SSH tasks well off the web tier.

The original answer [above] was a bit terse and didn’t answer a few questions.

• How will this avoid SSH connection threads exhausting the server? It will avoid it because you can easily throttle most message queues to avoid that. And you can easily ad back-end workers if you are exhausing your initial setup without material change in your application.

• How can I have shared state between requests? The same way you would normally — you send the request out to the queue, it completes and brings data back which can be passed to the next request.

• How would it appear to users? Could there be status updates? In most cases you can get the data out of the queuing system, or at least have status updates for task completion. Most message queues can easily give indication of failures or failures to start as well.

• Would I have to block the process when communicating between my web server in the message queue be blocking? Not unless you wrote it that way. You can easily setup an ajax request to tickle the queue every few seconds for status on a message to update the UI.

• Is this over-engineering? Not unless you decide to roll your own message queue.