I’m converting two columns of double type to a vector using VectorAssembler but I’m getting a struct with all the vector parameters as the output. This is the code for the vector transform

val assembler = new VectorAssembler()
      .setInputCols(Array("a", "b"))
      .setOutputCol("total")
      .setHandleInvalid("skip")
    
val output = assembler.transform(df)

I’m getting an output that looks like this

{type=1, size=null, indices=null, values=[8051.436057210056, 8327.770213419028]}

instead of getting just the values vector. the type is

struct<type:tinyint,size:int,indices:array<int>,values:array<double>>

Not sure what i’m doing wrong here. Got my vector assembler from How to convert VectorAssembler output which is Vector type to Array