I have a web app that searches across 2 APIs right now. I have my own Restful web service that I call, and it does all the work on the backend to asynchronously call the 2 APIs and concatenate them into one result set for my web app to use.
I want to scale this out and add as many other APIs as I can (currently looking at about 10 more). But as I add APIs, the call to my service gets (potentially) slower and more complex. How do I handle one API not responding … and other issues that arise?
What would be the best way to approach this? Should I create a service call for each API, that way each one is independent and not coupled to all the other calls? Is there a way on the backend to handle the multiple API calls without all the extra complexity it adds?
If I go the route of a service call per API, now my client code gets more complex (and I have a lot of clients)? And it’s more work for the client, and since I have mobile apps, it will cost the client more data usage.
If I go one service call, is there a way to set up some sort of connection so I can return data as I get it, in case one service call hangs?
4
In systems like this, you always have to consider that as far as your code is powerful, at least a call will be delayed, and if your system is successful, that event will happen hundreds of times making your system slow, then maybe unstable.
I do not know the nature of the integration, maybe game providers, maybe social networks or anything else. But if your application is not academic, and it represents a real business requirements (in production) I think this is part of the things to do:
Before code
- if possible, all systems that are integrated, they must be aware of the timeouts that are demanded from you (anticipating this from a contractual point of view, it is a good idea mostly because no one guarantees 100% of up-time. It’s myth).
- if point 1 is not possible, be objective and consider what we are willing to make our customers wait in front of the monitor (45 seconds, 10, 5, 2?). Found the value, we know how to set up the rest.
- Should be considered a constant calls/sub-calls prifile and logging;
- Should be planned for the possibility of suspending one or more services from the configuration (and not from code);
Technique
As for the technique, I think it is convenient to create a semaphore.
N functions will be called, the callback of each sub-function will be considered only if time set for each integration are respected.
So,
- A call is made to the main function,
- This considers what services are enabled to be contacted,
- The callback functions are configured,
- The sub-functions are called,
- While making calls, and while the general timer has not expired, all callbacks are considered to be unified in the response of the main function.
You need to look into threads and/or coroutines (Sometimes called fibers, goroutines, gevents, in different languages).
When a request comes in, you spawn off 10 threads/coroutines, then wait for them all to respond, or for a timeout. To the “web server”, your request is simply sleeping for a few seconds. But other threads/coroutines are actively doing work. If the work isn’t done in N seconds, the threads/coroutines are killed off and only the results you have are returned.
Example in Go.
Threads are heavy weight (harder to scale), but generally better supported depending on the language. Coroutines are more efficient (less RAM, less CPU), but have problems on some languages/libraries.
How would you handle one of the API calls failing? If only some of the calls succeed is it still a valid response to the client?
Since websockets aren’t widely supported and you are talking mobile platforms I think you have limited choices.
Long running requests
If the expected response is such that it may be processed in chunks before all requests complete, you may be able to stream the results in a single connection.
Ajax patterns provides some good information and examples for reading from a response stream before it has completed.
Using this approach, you would make a single request and handle the response as each API request completes.
Along with this, you will want to timeout individual API requests aggressively so that a hung service doesn’t heavily impact your interface.
Yo don’t want the client to timeout before done.
You could go as far as skipping calls for an expanding amount of time for any services which timeout or fail.
If it’s an aggregated news feed or something this may make sense but you could poll and cache the feeds server side instead.
Long polling
Instead of a single request, you could make an initial request and then poll for additional data. This has the advantage of not holding the connection and appearance of being a push.
This would completely defeat your intent of limiting the number of connections.
In conclusion
With all this said, I still think you need to consider all the factors of your design.
- what is the upper limit of API calls that will be made behind this proxy? 10 may be ok but at 20 or 30 is this still the right design?
- do all clients need the data from all services? Are you saving a little at the expense of all?
- what is the quantifiable amount of bandwidth and processing time saved using the proxy for say 1000 requests? Is this worth it?
1