PySpark: Element-wise sum ALL DenseVectors in each cell of a dataframe
I need to element-wise sum ALL DenseVectors for each sentence. There can be multiple DenseVectors (per sentence) in each row’s cell, i.e. embeddings.
I’ll explain with the help of multiple lists in a cell. Take the following example: