So, i was trying to downsize my ML app by making so that the pythorch package won’t be needed in the docker anymore, i saw that the torch.hub.load is using onnxruntime under the hood , so i imagined that i could instead use the onnxruntime directly.
onnx_model: onnxruntime.InferenceSession = onnxruntime.InferenceSession('model.onnx', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
I’m loding the model like this, and after some debugging i can even see that the model is loaded correctly.
Now, here is the problem, i created the following function to make predictios:
def predict(image, onnx_model: onnxruntime.InferenceSession):
initial_image = image
image = np.expand_dims(image, axis=0).astype('float32') / 255.
image = np.transpose(image, [0, 3, 1, 2])
outname = [i.name for i in onnx_model.get_outputs()]
outputs = onnx_model.run(outname, {'images': image})
return out
The parameters are : the image that must be 640*640 , and the inferenceSession itself( its bad practice, but i only need to confirm the possibility of going this way prior to making significant changes in the codebase.
The model runs fine, without erros , but when i go to see my outputs, it comes out as a List with 1 item, that looks like this :
[0:1] : [array([[ 4.0616, 9.6953, 9.3508, …, 0.0059212, 0.0024232, 0.9… 0.18608, 0.59289]], dtype=float32)] dtype: dtype(‘float32’)
max: 638.80396
min: 0.0
shape: (1, 25200, 10)
size: 252000
Since i’ve been working with pythorch, the normal output would be something like [(xmin,ymin,width,height), class_id, confidence] how could i manipulte the onnxruntime results to match the torchs output?
Aleandro Matteoni is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.