How to modify a neuron in the fully connected layer individually and design it as a statistical indicator such as kurtosis or negative entropy, as shown in the figure.enter image description hereIs it just a simple calculation of the kurtosis value after the linear layer? like as:
class KurtosisNeuron(nn.Module):
def init(self):
super(KurtosisNeuron, self).init()
def forward(self, x):
mean = torch.mean(x, dim=1, keepdim=True)
second_moment = torch.mean((x - mean) ** 2, dim=1, keepdim=True)
fourth_moment = torch.mean((x - mean) ** 4, dim=1, keepdim=True)
kurtosis = fourth_moment / (second_moment ** 2) - 3
return kurtosis
class CustomFC(nn.Module):
def init(self, in_features, out_features):
super(CustomFC, self).init()
self.fc = nn.Linear(in_features, out_features)
self.kurtosis_neuron = KurtosisNeuron()
def forward(self, x):
fc_output = self.fc(x)
kurtosis = self.kurtosis_neuron(fc_output)
林家兴 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.