Question

Hello there, this is one of my old exam questions. I am quite confused about part 3 (from the question given). Using the theory from Sutton and Barto I gave it a try.

Here is how I discussed it. Would there be any more points to add? Maybe some remarks or so?

Thanks in advance!!

Answer