I noticed this question is asked a lot but the solution proposed in other threads do not work for me.

I have created an EMR cluster on AWS and I am running a quite time intensive code. The notebook fails randomly after a while with the following error:

An error was encountered:
Invalid status code '400' from https://xxx.xx.x.xxx:18888/sessions/0/statements/20 with error payload: "requirement failed: Session isn't active."

Other threads suggest to add the configuration for livy session timeout during configuration of the cluster:

But I get the error anyway, any suggestions?

I have been working with pyspark and clusters for quite a long time but it seems to me that EMR is a step back to the old days of Cloudera where you had weird clusters errors like this.