So, i was trying to downsize my ML app by making so that the pythorch package won’t be needed in the docker anymore, i saw that the torch.hub.load is using onnxruntime under the hood , so i imagined that i could instead use the onnxruntime directly.

onnx_model: onnxruntime.InferenceSession = onnxruntime.InferenceSession('model.onnx', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])

I’m loding the model like this, and after some debugging i can even see that the model is loaded correctly.

Now, here is the problem, i created the following function to make predictios:

def predict(image, onnx_model: onnxruntime.InferenceSession):
    initial_image = image 
    image = np.expand_dims(image, axis=0).astype('float32') / 255.
    image = np.transpose(image, [0, 3, 1, 2])
    outname = [i.name for i in onnx_model.get_outputs()] 
    outputs = onnx_model.run(outname, {'images': image})
    return out

The parameters are : the image that must be 640*640 , and the inferenceSession itself( its bad practice, but i only need to confirm the possibility of going this way prior to making significant changes in the codebase.

The model runs fine, without erros , but when i go to see my outputs, it comes out as a List with 1 item, that looks like this :

[0:1] : [array([[ 4.0616, 9.6953, 9.3508, …, 0.0059212, 0.0024232, 0.9… 0.18608, 0.59289]], dtype=float32)] dtype: dtype(‘float32’)

max: 638.80396

min: 0.0

shape: (1, 25200, 10)

size: 252000

Since i’ve been working with pythorch, the normal output would be something like [(xmin,ymin,width,height), class_id, confidence] how could i manipulte the onnxruntime results to match the torchs output?