In a dataframe like the table below, I need to subtract the sum of two rows from another row, all grouped by type of CORP and BRANCH (where the first row already includes the other two).

I have a pyspark DataFrame with two columns. One is a float and another one is an array.
I know that the length of the array in each row is the same length as the the number of rows.
I want to create a new column in the DataFrame that for each row the result will be the dot product of the array and the column.

How can I convert a table of columns products and sales per Quarter to a table of 3 columns having Products, sales and Quarters using Pyspark. No date filed is defined in the table. The new Quarter column must be derived using the column name in the source table.

i have a list of dictionarys in python that i want to add as new column in pyspark dataframe such as all any single item in the list will tranclete to a cell in the new column.
when i try for example to transform it to pandas add the column and transform it to py spark the data becoume corrupted and i can figure out the correct functions combo to add the data without using pandas i tried using lit but or add a structType to no success
thanks for the help.

I have two PySpark DataFrames, df1 and df2, containing user information over 12 months in 2023. df1 contains the user ID and composite score for each month, while df2 contains the user ID and their salary for each month.

We try to update our table if a specific condition is fulfilled from the external data load. The code runs into several errors if we try this with our solution.

I have a dataframe with a contry descriptor ds_pais in English. I want to use GoogleTranslator to add a column via .withColumn that translates from english to spanish that country descriptor.