I have a Python API interfacing with a C# application. The C# component transmits an image to the API, which undergoes byte conversion followed by base64 encoding to facilitate transmission within a JSON request.

Subsequently, the API (python) decodes the base64 data to reconstruct the image for processing within a Convolutional Neural Network (CNN). The CNN outputs a binary response, indicating the presence or absence of a detected object.

However, the current end-to-end processing time is suboptimal, averaging approximately 1.5 seconds. I seek to achieve a significantly faster response time, ideally targeting a latency of around 200 milliseconds.

Are there alternative approaches to include the image in the request without prior conversion, such as transmitting a grayscale matrix or the object image directly?

Concerning the CNN model, I possess the H5 file. Are there alternative model formats conducive to faster inference? Attempts to mitigate latency by image resizing and reducing CNN hidden layers have proven inadequate.

My codes:

Python

@app.post('/overlap/')
async def overlap(request: Request):
    global cnn
    body = await request.json()
    image_np = np.frombuffer(base64.b64decode(body["image"]), dtype=np.uint8)
    image = cv2.imdecode(image_np, cv2.IMREAD_GRAYSCALE)
    image_resized = cv2.resize(image, (360, 300))
    gamma_corrected = tf.pow(image_resized / 255.0, 0.6) * 255.0
    image_batch = np.expand_dims(gamma_corrected, axis=0)
    result = cnn.predict(image_batch)
    if result[0][0] > result[0][1]:
        overlap_result = False
    else:
        overlap_result = True
    return [overlap_result, str(result[0][0]), str(result[0][1])]

C#

using System;
using System.Diagnostics;
using System.IO;
using System.Net;
using System.Text;
using System.Threading;

namespace comunica_api
{
    class Program
    {
        static void Main(string[] args)
        {
            string imagePath = @"C:my_photo.jpg";

            try
            {
                Stopwatch stopwatch1 = new Stopwatch();
                stopwatch1.Start();

                byte[] imageBytes = File.ReadAllBytes(imagePath);
                string base64String = Convert.ToBase64String(imageBytes);

                Console.WriteLine("Image read");

                string url = "http://localhost:8000/overlap/";
                string json = "{"image": "" + base64String + ""}";

                var request = (HttpWebRequest)WebRequest.Create(url);
                request.Method = "POST";
                request.ContentType = "application/json";
                request.ContentLength = json.Length;

                byte[] jsonBytes = Encoding.UTF8.GetBytes(json);

                using (Stream requestStream = request.GetRequestStream())
                using (StreamWriter writer = new StreamWriter(requestStream))
                {
                    writer.Write(json);
                }

                stopwatch1.Stop();

                Stopwatch stopwatch2 = new Stopwatch();
                stopwatch2.Start();

                using (var response = (HttpWebResponse)request.GetResponse())
                {
                    using (var streamReader = new StreamReader(response.GetResponseStream()))
                    {
                        string responseText = streamReader.ReadToEnd();
                        stopwatch2.Stop();
                        Console.WriteLine("API Response:");
                        Console.WriteLine(responseText);
                        long elapsedTimeMs1 = stopwatch1.ElapsedMilliseconds;
                        long elapsedTimeMs2 = stopwatch2.ElapsedMilliseconds;

                        Console.WriteLine($"Image conversion time: {elapsedTimeMs1} ms");
                        Console.WriteLine($"API response time: {elapsedTimeMs2} ms");
                        Thread.Sleep(3000);
                    }
                }
            }
            catch (FileNotFoundException)
            {
                Console.WriteLine("The specified image could not be found.");
                Thread.Sleep(3000);
            }
            catch (WebException ex)
            {
                Console.WriteLine("API communication error:");
                Console.WriteLine(ex.Message);
                Thread.Sleep(3000);
            }

            Console.ReadLine();
        }
    }
}

Any idea for this problem?