I had problem when trying to concatenate two dataframes with a few diferents colummns. One of the columns appear with NaN.

At first I selected the columns, and after that I renamed the columns, before write the concatenate’s code by pandas.

Example

df1
# A     B    C   D
# 0.3   0.4  4   7.1
# 0.9   0.8  9   3.2
# 0.4   0.5  8   2.9

df2
# A     B    C0   E
# 0.3   0.4  4   9.1
# 0.9   0.8  9   6.2
# 0.4   0.5  8   5.9

This the code that I use.

# Select columns
df1 = df1[[‘A’, ‘B’, ‘C’, ‘D’]]
df2 = df2[[‘A’, ‘B’, ‘C0’, ‘E’]]

# Rename columns
df1.columns = [“A”, “B”, “C”, “D”]
df2.columns = [“A”, “B”, “C”, “E”]
    
# Concatenate df
data = pd.concat([df1, df2], axis=0)
data= data.drop_duplicates(subset=[column name]).reset_index(drop=True)
print(data.shape)
data

Thats what happend n

data
# A     B    C   D    E
# 0.3   0.4  4   7.1  NaN
# 0.9   0.8  9   3.2  NaN
# 0.4   0.5  8   2.9  NaN

That I want

 data

# A     B    C   D    E
# 0.3   0.4  4   7.1  9.1
# 0.9   0.8  9   3.2  6.2
# 0.4   0.5  8   2.9  5.9

Could you help to fix the appear the data of the columns againts NaN