Is Normalized Simple Index a Viable Alternative for Positional Encoding in Constant Length Inputs?
I have a question about using positional encoding in transformer models, particularly regarding the suitability of a normalized simple index in scenarios where the input length is consistently constant.