I coded the perceptron algorithm form scratch and compared the weights I obtained after training with the weights I obtained after training the sklearn perceptron model. I believe even the sklearn model initializes the weights and biases as a zero vector and I chose the learning rate eta0=1
to match my perceptron code. (Note: the bias in my code is the last term in the vector w_b
)
My Code:
def perceptron(X_train, y_train):
#initialize weights as 0
w = np.zeros(len(X_train.columns))
b = 0
w_b = np.append(w, b)
while True:
misclassifications = 0
for X , Y in zip(X_train.values, y_train.values):
X_i = np.append(X, 1)
if Y*(np.dot(X_i,w_b)) <= 0:
w_b = w_b + Y*X_i
misclassifications += 1
if misclassifications == 0:
break
return w_b
w_b = perceptron(X_train, y_train)
Result: [-3. 6.7 -1. ]
sklearn code:
perceptron = Perceptron(max_iter=1000, eta0=1,random_state=42)
perceptron.fit(X_train, y_train)
print("weights are",perceptron.coef_)
print("bias is",perceptron.intercept_)
Result: weights are [[-4.7 10.1]] bias is [-2.]
I expected the weights to be same but they aren’t. Any clue on why?
rohit is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.