YOLOX support for OpenVINO Detector (#5285)

* Initial commit to enable Yolox models with OpenVINO in Frigate

* Fix ModelEnumtType import error in openvino.py

* Initial edit of the docs to include verbage about yolox

* Initial edit of the docs to include verbage about yolox

* Elaborate configuration and limitations in docs.

* Add capability to dynamically determine number of classes in yolox model

* Further refinements

* Removed unnecesarry comments, improved documentation, addressed PR items

* Fixed lint formatting issues
This commit is contained in:
Anil Ozyalcin
2023-02-03 17:36:37 -08:00
committed by GitHub
parent 7083a5c9b6
commit b33094207c
4 changed files with 120 additions and 21 deletions

View File

@@ -101,7 +101,7 @@ The OpenVINO device to be used is specified using the `"device"` attribute accor
OpenVINO is supported on 6th Gen Intel platforms (Skylake) and newer. A supported Intel platform is required to use the `GPU` device with OpenVINO. The `MYRIAD` device may be run on any platform, including Arm devices. For detailed system requirements, see [OpenVINO System Requirements](https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/system-requirements.html)
An OpenVINO model is provided in the container at `/openvino-model/ssdlite_mobilenet_v2.xml` and is used by this detector type by default. The model comes from Intel's Open Model Zoo [SSDLite MobileNet V2](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/ssdlite_mobilenet_v2) and is converted to an FP16 precision IR model. Use the model configuration shown below when using the OpenVINO detector.
An OpenVINO model is provided in the container at `/openvino-model/ssdlite_mobilenet_v2.xml` and is used by this detector type by default. The model comes from Intel's Open Model Zoo [SSDLite MobileNet V2](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/ssdlite_mobilenet_v2) and is converted to an FP16 precision IR model. Use the model configuration shown below when using the OpenVINO detector with the default model.
```yaml
detectors:
@@ -119,6 +119,25 @@ model:
labelmap_path: /openvino-model/coco_91cl_bkgr.txt
```
This detector also supports YOLOx models, and has been verified to work with the [yolox_tiny](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/yolox-tiny) model from Intel's Open Model Zoo. Frigate does not come with `yolox_tiny` model, you will need to follow [OpenVINO documentation](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/yolox-tiny) to provide your own model to Frigate. There is currently no support for other types of YOLO models (YOLOv3, YOLOv4, etc...). Below is an example of how `yolox_tiny` and other yolox variants can be used in Frigate:
```yaml
detectors:
ov:
type: openvino
device: AUTO
model:
path: /path/to/yolox_tiny.xml
model:
width: 416
height: 416
input_tensor: nchw
input_pixel_format: bgr
model_type: yolox
labelmap_path: /path/to/coco_80cl.txt
```
### Intel NCS2 VPU and Myriad X Setup
Intel produces a neural net inference accelleration chip called Myriad X. This chip was sold in their Neural Compute Stick 2 (NCS2) which has been discontinued. If intending to use the MYRIAD device for accelleration, additional setup is required to pass through the USB device. The host needs a udev rule installed to handle the NCS2 device.

View File

@@ -105,6 +105,9 @@ model:
# Optional: Object detection model input tensor format
# Valid values are nhwc or nchw (default: shown below)
input_tensor: nhwc
# Optional: Object detection model type, currently only used with the OpenVINO detector
# Valid values are ssd or yolox (default: shown below)
model_type: ssd
# Optional: Label name modifications. These are merged into the standard labelmap.
labelmap:
2: vehicle