I have stored a parquet file in my worker machines as i want to try to do this project without using the cloud. I am aware that i’ll encounter an issue when trying to seek these files as the code i am running in my scheduler machine has the path specific to that machine, i have tried solving the issue like this:

hostname = socket.gethostname()

if hostname == 'pc1':
    return 'path1'

elif hostname == 'pc2':
    return 'path2'

elif hostname == 'pc3':
    return 'path3'

elif hostname == 'pc4':
    return 'path4'

however this still gives me the file not found error…