We’ve used azure ADL2 for some time, and the amount of memory it consumes is problematic, forcing us to use awkward and lower-performing techniques. In a discussion in this github issue, it was confirmed that the API does a ton of buffering, and suggested to me that we try the “sync” interface when it is available, and this azure page suggest that v12 of the Java SDK supports the sync API. But I can’t find out anything more about it, and the last response I got on the github issue was “thanks but we’re closing this”.
I’m trying to get two questions answered:
- Will the “sync” API allow us to reduce memory consumption for multi-part uploads to ADL2?
- If so, how do we use it? Where is the documentation or an example?
Thanks, any help appreciated.