Coral part I.

intro

... sometimes in Q2/2019, I purchased a Google Coral Dev Board, a USB Accelerator, a Camera, and an Environmental Sensor Board (see product list here). When I received the package, unfortunatelly I haven't had too much time to work with it, so started to read some articles instead:

• Say "Hello" to Google Coral
• Hands on with the Coral Dev Board
• Object detection and image classification with Google Coral USB Accelerator
• Hands-on with the Google Coral USB Accelerator

Later on to experiment with the TPU and object detection, as I'm a huge fan of LEGO, I decided to create a LEGO figure detector. Built some structures to host the board and the camera in a LEGO landscape:

to train a model

I followed the retrain an object detection model tutorial, but instead of training it for detecting pets, decided to retrain it for detectin LEGO figures, and instead of using MobileNet SSD v1, I decided to use MobileNet SSD v2.

To start a trianing, a labeled imageset is required, so I took approximatelly 400 pictures of LEGO figures with my mobile and the Coral Boards camera. iPhone produced HEIC images those had to be converted to JPG:

# brew install imagemagick
# magick mogrify -monitor -format jpg *.HEIC

Thereafter I used LabelImg for labeling, and started a docker image based training on my Mac. It turned out using a docker image localy, without a GPU is not a good choice. So decided to move the training to a powerful machine with multiple GPUs.

When model training finished, since I was not exactly following the tutorial provided on the Coral page, I had to check out convert_checkpoint_to_edgetpu_tflite.sh for next steps:

• exporting frozen graph from checkpoint: object_detection/export_tflite_ssd_graph.py
• converting frozen graph to TF Lite file: tflite_convert

python object_detection/export_tflite_ssd_graph.py
--pipeline_config_path=ckpt/pipeline.config
--trained_checkpoint_prefix=train/model.ckpt-400
--output_directory=models/
--add_postprocessing_op=true

... and that tflite compatible frozen graph can be converted to TFLite, including quantizing:

#!/bin/sh
tflite_convert
--output_file=out2.tflite
--graph_def_file=tflite_graph.pb
--inference_type=QUANTIZED_UINT8
--input_arrays=normalized_input_image_tensor
--output_arrays=TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3
--mean_values=128
--std_dev_values=128
--input_shapes=1,300,300,3
--change_concat_input_ranges=false
--allow_nudging_weights_to_use_fast_gemm_kernel=true
--allow_custom_ops
--default_ranges_min=0
--default_ranges_max=255

To run a tflite model on the TPU, it must be compiled for the edge TPU with edgetpu_compiler. Earlier this was possible to do on the Coral Board, but today I would do it on my mac with using docker-edgetpu-compiler

Edge TPU Compiler version 2.0.258810407
Input: out2.tflite
Output: out2_edgetpu.tflite

Operator                       Count      Status

CUSTOM                         1          Operation is working on an unsupported data type
CONCATENATION                  2          Mapped to Edge TPU
CONV_2D                        34         Mapped to Edge TPU
DEPTHWISE_CONV_2D              13         Mapped to Edge TPU
RESHAPE                        13         Mapped to Edge TPU
LOGISTIC                       1          Mapped to Edge TPU)

conclusion

To experiment with the model, and display what I wanted, the way I wanted, I modified the Google edgetpu_detect_server code and the edgetpuvision code. Be aware that the 'Edge TPU Python API' is now deprecated.

Since I've built a new version of the Rig with custom 3d printed LEGO parts, started to use an external camera, spent some time figuring out how to improve picture quality, deployed more models, etc.

Next part of the Coral story will be about 3d printed LEGO parts