I am currently working on a project that involves Java calling calling native code, a JNIEXPORT function that returns a boolean, this native code performs 0 JNI or JVMTI calls, it just returns a boolean and nothing more.
The way the call takes place is somewhat unique due to constraints in my project:
myFunctionA and myFunctionB are always in interpreted mode, and myFunctionC is native and calls a JNIEXPORT.
- Something in the Java program calls myFunctionA, passing in 4 Java objects.
- Before the interpreter executes myFunctionA, a register holding the Method* oop is swapped, to make the interpreter execute myFunctionB. myFunctionB takes 0 parameters. (myFunctionA never runs)
- myFunctionB calls native myFunctionC, this calls my native code, and returns a boolean.
- myFunctionC returns to myFunctionB, when myFunctionB returns, the stack should get restored and everything should be fine.
Since myFunctionA is in interpreted mode at all times, this means the parameters passed to it will always be placed on the stack, and the stack gets restored when a return takes place.
On Java8_51 this code crashes after about 120k calls, on Java8_241 it crashes after about 220k calls, I have ran multiple tests and these numbers are consistent.
The crashes have an error code of -1073741571 which means a stack overflow crash on windows.
Looking at task manager, there does not seem to be a memory leak.
I have tried increasing the stack size by changing the launch options, this had the effect of freezing the Java program for a around minute and then crashing when number of calls was reached, but did not affect the number of calls that could take place.
Adding sleep calls in various places was able to sometimes change the error code, I was once able to get -1073741819 (STATUS_ACCESS_VIOLATION, some instruction ? referenced memory at ?)
I have tried periodically calling system.gc() (every 5k, 10k, 20k calls) inside the JNIEXPORT function, this had no affect.
My guess is that the objects passed to myFunctionA do not get freed properly or fully, causing some type of buffer/space within the JVM to run out, but im not sure what exactly it could be.