I have a AWS Lambda function that will “move” objects in my S3 bucket to another location in the same bucket (changes the key) after a certain amount of time. I have this location, the archive, set with a life cycle configuration to save cost. This hasn’t been an issue previously, but as more objects have been added, the ‘list_objects_v2’ call is taking more time. This is the structure of my bucket:

111    
└───folder1
│   │   file011.png
│   │   file012.png
│   
└───folder2
    │   file021.png
    │   file022.png
|
222    
└───folder3
│   │   file031.png
│   │   file032.png
│   
└───folder4
    │   file041.png
    │   file042.png
|
archive    
└───111
│   │   folder1  //Images contained in these folders
│   │   folder2
│   
└───222
    │   folder3
    │   folder4

It isn’t an issue now, but as the archive folder grows, the amount of time it will take to complete the function will increase. I’d like to know if there is a way I can exclude the archive folder when calling ‘list_objects_v2’ or if there is a better alternative?

For a little more context, I’m using Python 3.12 and boto3. Currently calling ‘list_objects_v2’ with a paginator to get all objects in the bucket.

paginator = s3.get_paginator('list_objects_v2')
pages = paginator.paginate(Bucket=bucket_name)