Do PyTorch Custom Activation Functions Require Custom Backprop Implementation?
I was thinking about messing around with activation functions that are a little less standard. I know of making the class, implementing the forward, etc. but do more complicated functions require a backdrop function?