yolo_v8_detector not compatible with torch backend because no good option for ragged tensor

[svikolev]

Short Description
[ yolo_v8_detector requires ragged tensor for bounding boxes. the torch version of this is nested_tensor however .size() is not supported for torch nested tensor. nested_tensor

Existing Implementations

for tf backend this works:

 labels = {
     "boxes": tf.ragged.constant(y_b, dtype=tf.float32),
     "classes": tf.ragged.constant(y_c, dtype=tf.int64),
 }
 t_d = {"images": tf.cast(np.array(x), tf.float32), "bounding_boxes": labels}

but for torch backend the equivalent:

labels = {
    "boxes": torch.nested.nested_tensor(y_b, dtype=torch.float32),
    "classes": torch.nested.nested_tensor(y_c, dtype=torch.float32),
}

t_d = {"images": torch.tensor(x, dtype=torch.float32), "bounding_boxes": labels}

returns the following error when attempting training
File ~.conda\envs\pumps_pycro_torch_keras\lib\site-packages\torch_tensor.py:1062, in Tensor.array(self, dtype)
1060 return handle_torch_function(Tensor.array, (self,), self, dtype=dtype)
1061 if dtype is None:
-> 1062 return self.numpy()
1063 else:
1064 return self.numpy().astype(dtype, copy=False)

RuntimeError: Internal error: NestedTensorImpl doesn't support sizes. Please file an issue.

from torch doc:
Even though a NestedTensor does not support .size() (or .shape), it supports .size(i) if dimension i is regular.

Other Information

print(torch.__version__)
print(keras_cv.__version__)
print(keras.__version__)
print(torch.cuda.is_available())

2.2.1+cu121
0.8.2
3.0.5
True

[sachinprasadhs]

it is due to to the lack of support of Ragged tensors in torch.
If you're using Ragged tensor, the only option as of now is to use tensorflow backend.

[github-actions]

This issue is stale because it has been open for 14 days with no activity. It will be closed if no further activity occurs. Thank you.

[github-actions]

This issue was closed because it has been inactive for 28 days. Please reopen if you'd like to work on this further.