I am using a service account created in Google cloud platform to upload csv files to gdrive using a python script. This script was working file till yesterday but now I am getting below error.
Script:
<code>scopes = ['https://www.googleapis.com/auth/drive']
gdrive = 'D:/common-modules/gdrive.json'
folder_id = "1IxxxxyYxxxx"
def upload_option_strike_data(save_location, file_name):
try:
creds = service_account.Credentials.from_service_account_file(gdrive, scopes=scopes)
service = build('drive', 'v3', credentials=creds, cache_discovery=False)
file_metadata = {
'name': file_name,
'parents': [folder_id]
}
file_path = f"{save_location}/{file_name}"
media = MediaFileUpload(file_path, mimetype='text/csv')
service.files().create(
body=file_metadata,
media_body=media
).execute()
print(f"Successfully uploaded historical data collected for to Gdrive")
except Exception as e:
print(
f"Failed to upload historical data collected for .Function - upload_option_strike_data.Exception: {str(e)}")
upload_option_strike_data("D:/daily-data/output", "FINNIFTY_OptionData_20240514.csv")
</code>
<code>scopes = ['https://www.googleapis.com/auth/drive']
gdrive = 'D:/common-modules/gdrive.json'
folder_id = "1IxxxxyYxxxx"
def upload_option_strike_data(save_location, file_name):
try:
creds = service_account.Credentials.from_service_account_file(gdrive, scopes=scopes)
service = build('drive', 'v3', credentials=creds, cache_discovery=False)
file_metadata = {
'name': file_name,
'parents': [folder_id]
}
file_path = f"{save_location}/{file_name}"
media = MediaFileUpload(file_path, mimetype='text/csv')
service.files().create(
body=file_metadata,
media_body=media
).execute()
print(f"Successfully uploaded historical data collected for to Gdrive")
except Exception as e:
print(
f"Failed to upload historical data collected for .Function - upload_option_strike_data.Exception: {str(e)}")
upload_option_strike_data("D:/daily-data/output", "FINNIFTY_OptionData_20240514.csv")
</code>
scopes = ['https://www.googleapis.com/auth/drive']
gdrive = 'D:/common-modules/gdrive.json'
folder_id = "1IxxxxyYxxxx"
def upload_option_strike_data(save_location, file_name):
try:
creds = service_account.Credentials.from_service_account_file(gdrive, scopes=scopes)
service = build('drive', 'v3', credentials=creds, cache_discovery=False)
file_metadata = {
'name': file_name,
'parents': [folder_id]
}
file_path = f"{save_location}/{file_name}"
media = MediaFileUpload(file_path, mimetype='text/csv')
service.files().create(
body=file_metadata,
media_body=media
).execute()
print(f"Successfully uploaded historical data collected for to Gdrive")
except Exception as e:
print(
f"Failed to upload historical data collected for .Function - upload_option_strike_data.Exception: {str(e)}")
upload_option_strike_data("D:/daily-data/output", "FINNIFTY_OptionData_20240514.csv")
Error Message:
<code>Failed to upload historical data collected for .Function - upload_option_strike_data.Exception: <HttpError 403 when requesting https://www.googleapis.com/upload/drive/v3/files?alt=json&uploadType=multipart returned "The user's Drive storage quota has been exceeded.". Details: "[{'message': "The user's Drive storage quota has been exceeded.", 'domain': 'usageLimits', 'reason': 'storageQuotaExceeded'}]">
</code>
<code>Failed to upload historical data collected for .Function - upload_option_strike_data.Exception: <HttpError 403 when requesting https://www.googleapis.com/upload/drive/v3/files?alt=json&uploadType=multipart returned "The user's Drive storage quota has been exceeded.". Details: "[{'message': "The user's Drive storage quota has been exceeded.", 'domain': 'usageLimits', 'reason': 'storageQuotaExceeded'}]">
</code>
Failed to upload historical data collected for .Function - upload_option_strike_data.Exception: <HttpError 403 when requesting https://www.googleapis.com/upload/drive/v3/files?alt=json&uploadType=multipart returned "The user's Drive storage quota has been exceeded.". Details: "[{'message': "The user's Drive storage quota has been exceeded.", 'domain': 'usageLimits', 'reason': 'storageQuotaExceeded'}]">
I am using 200GB storage plan and is currently 143GB used.
Each of my csv files will be around 7MB size. When I upload a small file with size around 100KB, files are getting uploaded without any issues. Error indicates about storageQuotaExceeded, but unable to identify this quota setting (dont see anything from GCP console)
How to resolve this issue ?