TensorFlow Lite model for pedestrian detection

[sayakpaul]

Goal

Come up with a TensorFlow Lite model (an optimized FlatBuffer format with the .tflite file extension) that is capable of detecting pedestrians from a scene.

Deliverables

• The TensorFlow Lite model.
• A Colab Notebook showing the model conversion process and how to make predictions with the TensorFlow Lite model on static images.

[around-star]

@sayakpaul Here's the update . I made the model conversion and inference functions. I used a pretrained tflite model and checked my inference and prediction functions with it. Got a fair output but can be done much better if I can use my custom non-max-suppression function on the predicted output of the model. But the tflite model which I used, does not provide the "raw_detections" (without non-max-suppression) . Whereas the other models from tensorflow hub does provide those "raw detections" and thus tested it out. It worked better than the previous tflite model with different threshold values of min_score and non_max_suppression. But I am facing issues converting those models to its tflite equivalent.
I think the problem is pertained to the hub models itself instead of the conversion function. Can you help me with that?

[sayakpaul]

Without looking into the results you are getting it's a bit difficult for me to comment on this.

Also, it's problematic to comment without actually seeing your code used for conversion. It would be helpful to also know which exact model variants you have tried.

Have you considered looking into the following resources that show how to convert certain object detection models to TensorFlow Lite? Not all the models are TFLite compatible and you should know about that.

• https://github.com/sayakpaul/E2E-Object-Detection-in-TFLite
• https://github.com/sayakpaul/Adventures-in-TensorFlow-Lite/blob/master/MobileDet_Conversion_TFLite.ipynb
• https://github.com/sayakpaul/Adventures-in-TensorFlow-Lite/blob/master/EAST_TFLite.ipynb

[around-star]

@sayakpaul Thanks for the reply.
Here is the code.
Also I'm now thinking of training a model cuz the pretrained ones predicts some fair amount of false pos and negs.

[sayakpaul]

This hub model won't get converted to TensorFlow Lite because it's not compatible. You can follow the conversion process of a similar model here - https://github.com/sayakpaul/E2E-Object-Detection-in-TFLite (specifically this notebook).

Also, now might be a good time to study/discover if there are existing issues that other people have reported that are similar to yours.

[around-star]

@sayakpaul , @rishiraj
This is my final code
I have tested a SSD MobileNet_V1 tflite model on static images. I also have made the code for "video predictions" but that can only be run locally.

[sayakpaul]

@around-star great start.

A couple of improvement pointers -

• Make the notebook end-to-end runnable on Colab. You are using Drive files that may not be accessed by someone else. Please that. You can always host a set of files under Releases of any public GitHub repository and retrieve them using curl or wget. So, please reflect this change in the notebook.
• One of the major takeaways of this task is to actually do the conversion and not use any pre-trained TFLite model. So, you will need to work that out.
• Please format your notebook in a proper manner. You wouldn't want to include code that's not being used and you would also want to include comment annotations, and so on. You can refer to any of the notebooks from this repository to get an idea of what I am saying.
• It's possible to run predictions on a video and even show that all inside of Colab. Please refer to this notebook for more details.

[rishiraj]

@around-star While I appreciate the efforts you have put, I feel you have deviated from the original goals of this issue. The goal of this issue was to convert to a TFLite model and making prediction with it on static images. So using a pre-trained model doesn't help. Please go through the deliverables carefully. You can collaborate with @piyush-cosmo to get the model for conversion.

[around-star]

@sayakpaul , @rishiraj This is my improved colab. The model code is taken from @piyush-cosmo (colab).

[sayakpaul]

Nice work!

I think you could organize the Colab Notebook a bit more by deleting unnecessary cells and adding comments/texts wherever necessary. Also, try adding all the major outputs. I see the output of the following is missing -