For a hobby project, I’m writing an x86 GC and JIT. For the GC, I need to maintain information about the stack layout (it’s a precise GC), for which I need to be able to find out which method the IP currently is in (and the complete call chain of course). How can you do this?
The best solution so far was to keep a b-tree of the start addresses of all jitted methods, and use that to look up the current method. However, this looks like a lot of overhead. An alternative would be to use the BSP to find the return address, go back a few bytes and see what address was called. I could then put some data before the entry point. However, that has the issue that the callee may not be a jitted method (there will be native methods on the stack). In that case, the data before the method would be garbage or may not even be valid memory (in some extreme corner cases).
What is the usual mechanism of implementing this functionality?