[Bug] Python SDK Batch Inference not working?

[matthost]

Checklist

• I have searched related issues but cannot get the expected help.
• 2. I have read the FAQ documentation but cannot get the expected help.
• 3. The bug has not been fixed in the latest version.

Describe the bug

An mmdetection model was compiled with static batch size of 3. When using Python Detector API to perform inference on a batch of 3 images, it is failing with RuntimeError: continuous uint8 HWC array expected.

Reproduction

Convert an MMDet model to TensorRT using MMDeploy with a static batch size > 1. Try to run batch inference:

from mmdeploy_runtime import Detector
import cv2

img = cv2.imread('image.jpg')

# create a detector
detector = Detector(model_path='model_path', device_name='cuda', device_id=0)
# run the inference
 detector([img, img, img])

Error: RuntimeError: continuous uint8 HWC array expected

Environment

Latest MMDeploy (1.3), MMDet 3.3, Torch 2.3, Python 3.8, linux

Error traceback

`RuntimeError: continuous uint8 HWC array expected`

[matthost]

Perhaps I am doing something wrong? I tried numpy stacking the images which did not help.

Perhaps related to #2808

[matthost]

I found the Detector class has a batch method so gave that a try:

detector.batch([img, img, img])

But it is passing a single image to the model it seems, which fails due to wrong dims:

[2024-08-14 16:07:31.873] [mmdeploy] [error] [trt_net.cpp:28] TRTNet: 3: [executionContext.cpp::validateInputBindings::2083] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::validateInputBindings::2083, condition: profileMinDims.d[i] <= dimensions.d[i]. Supplied binding dimension [1,3,800,1344] for bindings[0] exceed min ~ max range at index 0, maximum dimension in profile is 3, minimum dimension in profile is 3, but supplied dimension is 1.

[matthost]

The from mmdeploy.apis import inference_model API seems to work if you pass a list to img