Implementing a NN for regression task with 16(features)+1(bias) inputs and 1 output, I’m only using numpy and vectorization, when I train it on the training set, the first sample of the input is the only one learned perfectly, the others are kinda learned but not well at all.
Is there something I did wrong in backpropagation’s operations?
The bias is implemented in the first layer by adding a feature with value 1 to the training samples.
I tried different learning rates and network dimensions, but nothing changes. this is the kind of output I get, where l is the label, y the predicted value and the first 5 rows are the loss progression:
loss: [8702.85226111]
loss: [6.46234854e-27]
loss: [1.61558713e-27]
loss: [4.03896783e-28]
loss: [0.]
l: 131.042274 y: [131.042274]
l: 64.0 y: [103.78313187]
l: 89.429199 y: [30.54333083]
l: 111.856492 y: [108.32052489]
l: 69.3899 y: [57.11792288]
This is my colab notebook for this task:
https://colab.research.google.com/drive/1SNEjgZQkmQW9LV8PSxE_Lx4VIQSjf1rP?usp=sharing