I am trying to import csv files saved in an Azure data container into a spark dataframe using a for loop. I am running this code on Azure Databricks. The for loop runs without any error. However I am unable to access the dataframe by its name.
I am able to access the teams dataframe by using df_spark.show()
Please could you help me.
”’
for df_spark in ['athletes', 'coaches','entriesgender','medals','teams']:
df_spark=spark.read.csv('/mnt/data/raw-data/'+df_spark+'.csv',header=True)
df_spark.show()
athletes.show(n=5) # Error: It wouldn't recognise the spark dataframe by name.
”’
The code