before, I used SQL joins to get data, and 10.000 lines would take fraction of a second.
select name,
profession.expertise,
resonsible.name,
building.locker
from technicians
join profession on technicin.profession = profession.id
join responsible on techician.responsible = responsible.id
join building on technician.locker = buiding.id
with migration to the cloud, I no longer have access to this DB and need to go via rest. unfortunatelly, translating this into python REST requests, even just for 50 lines, results in 20s of stall, 10000 lines would just be hours!? Because I have to itterate through all the results and request the related relationship link data, right?
pseudo python
result_list = requests.get("https://sample.com/api/v1/technicians")
for technician in result_list:
professn = requests.get("https://sample.com/api/v1/technicians/technician_ID/profession")
resopnsible = requests.get("https://sample.com/api/v1/technicians/technician_ID/responsible")
building = requests.get("https://sample.com/api/v1/technicians/technician_ID/building")
question: am I doing something wrong regarding the requests I make?
4
unfortunatelly, translating this into python REST requests, even just for 50 lines, results in 20s of stall, 10000 lines would just be hours!?
YES
Because I have to itterate through all the results and request the related relationship link data, right?
YES
Explanation
You have a client app that is responsible of requesting the API. This is what your query earlier did:
- Get All The Stuff (< 1s)
- Parse them (probably quick, but it depends exactly what you do with the data you have received)
What you currently do:
- Get all the stuff
- For each element
- get profession of element
- get responsible of element
- get building of element
So, while earlier you had a single request to the API and it had a single request to the RDBMS, now you do the same full request as earlier, but for each element, you are sending 3 new requests.
So you will need to carefully check what exactly requests.get("https://sample.com/api/v1/technicians") is returning for you, whether the profession, the responsible and the building is already included into the result. If so, then do not request for them separately, but instead use the data you have received.
If the other data is not included into the result, then study the API and see whether there is a way to get the technicians along with the profession, the responsibility and the building and use that instead.
If there is no such feature of the API, then implement one if you can, or, if you are unable either due to lack of access, lack of privilege or other reasons, then reach out to the person who is responsible for providing the API and explain that you need a comprehensive getter where you can get all the technicians you need along with their profession, their responsibility and their building. Also, it would not hurt to explain that you may want to have a paging, so if you could specify in some manner how many records you want to get, starting from which index, it would be great, as well as support for sorting if needed as well as filtering for the case when you want to only get records that fulfill some criteria.
2