I’m a NN newbie. I would like to emulate the following NN architecture, taken from this paper. The goal of the stochastic optimal control problem is to minimise a total cost C_T. In the diagram below, s_i, a_i and xi_i are the state of the system, the action, and the noise at step i respectively. The system evolves according to

 s_{i+1} = s_i + f(s_i, a_i) + xi_{i+1}

where f is some function.

I have been trying to think about this as a chain of neural networks: excluding the first and last time step, each network N_i takes as an input (s_{i-1}, a_{i-1}, xi_i}, where a_{i-1} is the output of the previous network and xi_i is a fresh input, and outputs a new action a_i. Am I thinking about this the right way? If so, how can go about setting this up in Keras?

• I have browsed the documentation, and although there are some nice examples of chaining together NNs, I haven’t seen one yet that passes a fresh input xi_i at each stage.
• A second problem is that since the problem has dynamics, I need to be able to perform operations between the layers according to the evolution equation above. How can I achieve this?

Would be grateful for any pseudocode / guidance on how to proceed!