I have the x and y values of solving some kind of differential equation. I am trying to train a neural network on TensorFlow, but the result is unstable and not accurate. What is the reason? It is important to note that the prediction occurs as follows: at the input x at the output x2 at the input x2 at the output x3 at the input x3 at the output x4, etc.

predict 1

predict 2

predict 3 is already better

Here is my model:

epochs = 200
activ = 'tanh'
neuro = 100
lay_count = 20
model = k.Sequential()
optimizer = k.optimizers.SGD(learning_rate=0.01, momentum=0.9, nesterov=True)
model.add(k.layers.Input(shape=(17,)))
for i in range(0, lay_count):
    model.add(k.layers.Dense(neuro, name='Inner_dense' + str(i), activ=activ))
model.add(k.layers.Dense(3, name='Outer_dense'))
model.compile(loss='mse', optimizer=optimizer)

I tried to play with the number of network layers, the number of neurons, and to use the Adam optimizer. Didn’t help much.