forked from Github/frigate
Initial support for Hailo-8L (#12431)
* Initial support for Hailo-8L Added file for Hailo-8L detector including dockerfile, h8l.mk, h8l.hcl, hailo8l.py, ci.yml and ssd_mobilenat_v1.hef as the inference network. Added files to help with the installation of Hailo-8L dependences like generate_wheel_conf.py, requirements-wheel-h8l.txt and modified setup.py to try and work with any hardware. Updated docs to reflect Initial Hailo-8L support including oject_detectors.md, hardware.md and installation.md. * Update .github/workflows/ci.yml typo h8l not arm64 Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com> * Update docs/docs/configuration/object_detectors.md Clarity for the end user and correct uses of words Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com> * Update docs/docs/frigate/installation.md typo Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com> * update Installation.md to clarify Hailo-8L installation process. * Update docs/docs/frigate/hardware.md Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> * Update hardware.md add Inference time. * Oops no new line at the end of the file. * Update docs/docs/frigate/hardware.md typo Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com> * Update dockerfile to download the ssd_modilenet_v1 model instead of having it in the repo. * Updated dockerfile so it dose not download the model file. add function to download it at runtime. update model path. * fix formatting according to ruff and removed unnecessary functions. --------- Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com> Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
This commit is contained in:
spanner3003
committed by
Nicolas Mowen
Nicolas Mowen
parent
e7fabce4e0
commit
4a35573210
@@ -5,7 +5,7 @@ title: Object Detectors
|
||||
|
||||
# Officially Supported Detectors
|
||||
|
||||
Frigate provides the following builtin detector types: `cpu`, `edgetpu`, `openvino`, `tensorrt`, and `rknn`. By default, Frigate will use a single CPU detector. Other detectors may require additional configuration as described below. When using multiple detectors they will run in dedicated processes, but pull from a common queue of detection requests from across all cameras.
|
||||
Frigate provides the following builtin detector types: `cpu`, `edgetpu`, `openvino`, `tensorrt`, `rknn`, and `hailo8l`. By default, Frigate will use a single CPU detector. Other detectors may require additional configuration as described below. When using multiple detectors they will run in dedicated processes, but pull from a common queue of detection requests from across all cameras.
|
||||
|
||||
## CPU Detector (not recommended)
|
||||
|
||||
@@ -386,3 +386,25 @@ $ cat /sys/kernel/debug/rknpu/load
|
||||
|
||||
- All models are automatically downloaded and stored in the folder `config/model_cache/rknn_cache`. After upgrading Frigate, you should remove older models to free up space.
|
||||
- You can also provide your own `.rknn` model. You should not save your own models in the `rknn_cache` folder, store them directly in the `model_cache` folder or another subfolder. To convert a model to `.rknn` format see the `rknn-toolkit2` (requires a x86 machine). Note, that there is only post-processing for the supported models.
|
||||
|
||||
## Hailo-8l
|
||||
|
||||
This detector is available if you are using the Raspberry Pi 5 with Hailo-8L AI Kit. This has not been tested using the Hailo-8L with other hardware.
|
||||
|
||||
### Configuration
|
||||
|
||||
```yaml
|
||||
detectors:
|
||||
hailo8l:
|
||||
type: hailo8l
|
||||
device: PCIe
|
||||
model:
|
||||
path: /config/model_cache/h8l_cache/ssd_mobilenet_v1.hef
|
||||
|
||||
model:
|
||||
width: 300
|
||||
height: 300
|
||||
input_tensor: nhwc
|
||||
input_pixel_format: bgr
|
||||
model_type: ssd
|
||||
```
|
||||
|
||||
@@ -107,6 +107,12 @@ Frigate supports hardware video processing on all Rockchip boards. However, hard
|
||||
|
||||
The inference time of a rk3588 with all 3 cores enabled is typically 25-30 ms for yolo-nas s.
|
||||
|
||||
#### Hailo-8l PCIe
|
||||
|
||||
Frigate supports the Hailo-8l M.2 card on any hardware but currently it is only tested on the Raspberry Pi5 PCIe hat from the AI kit.
|
||||
|
||||
The inference time for the Hailo-8L chip at time of writing is around 17-21 ms for the SSD MobileNet Version 1 model.
|
||||
|
||||
## What does Frigate use the CPU for and what does it use a detector for? (ELI5 Version)
|
||||
|
||||
This is taken from a [user question on reddit](https://www.reddit.com/r/homeassistant/comments/q8mgau/comment/hgqbxh5/?utm_source=share&utm_medium=web2x&context=3). Modified slightly for clarity.
|
||||
|
||||
@@ -100,6 +100,38 @@ By default, the Raspberry Pi limits the amount of memory available to the GPU. I
|
||||
|
||||
Additionally, the USB Coral draws a considerable amount of power. If using any other USB devices such as an SSD, you will experience instability due to the Pi not providing enough power to USB devices. You will need to purchase an external USB hub with it's own power supply. Some have reported success with <a href="https://amzn.to/3a2mH0P" target="_blank" rel="nofollow noopener sponsored">this</a> (affiliate link).
|
||||
|
||||
### Hailo-8L
|
||||
|
||||
The Hailo-8L is an M.2 card typically connected to a carrier board for PCIe, which then connects to the Raspberry Pi 5 as part of the AI Kit. However, it can also be used on other boards equipped with an M.2 M key edge connector.
|
||||
|
||||
#### Installation
|
||||
|
||||
For Raspberry Pi 5 users with the AI Kit, installation is straightforward. Simply follow this [guide](https://www.raspberrypi.com/documentation/accessories/ai-kit.html#ai-kit-installation) to install the driver and software.
|
||||
|
||||
For other boards, follow these steps for installation:
|
||||
|
||||
1. Install the driver from the [Hailo GitHub repository](https://github.com/hailo-ai/hailort-drivers). A convenient script for Linux is available to clone the repository, build the driver, and install it.
|
||||
2. Copy or download [this script](https://gist.github.com/spanner3003/4b85751d671d4ac55f926e564f1abc3e#file-install_hailo8l_driver-sh).
|
||||
3. Ensure it has execution permissions with `sudo chmod +x install_hailo8l_driver.sh`
|
||||
4. Run the script with `./install_hailo8l_driver.sh`
|
||||
|
||||
#### Setup
|
||||
|
||||
To set up Frigate, follow the default installation instructions, but use a Docker image with the `-h8l` suffix, for example: `ghcr.io/blakeblackshear/frigate:stable-h8l`
|
||||
|
||||
Next, grant Docker permissions to access your hardware by adding the following lines to your `docker-compose.yml` file:
|
||||
|
||||
```yaml
|
||||
devices:
|
||||
- /dev/hailo0
|
||||
```
|
||||
|
||||
If you are using `docker run`, add this option to your command `--device /dev/hailo0`
|
||||
|
||||
#### Configuration
|
||||
|
||||
Finally, configure [hardware object detection](/configuration/object_detectors#hailo-8l) to complete the setup.
|
||||
|
||||
### Rockchip platform
|
||||
|
||||
Make sure that you use a linux distribution that comes with the rockchip BSP kernel 5.10 or 6.1 and necessary drivers (especially rkvdec2 and rknpu). To check, enter the following commands:
|
||||
@@ -222,6 +254,7 @@ The community supported docker image tags for the current stable version are:
|
||||
- `stable-rocm-gfx900` - AMD gfx900 driver only
|
||||
- `stable-rocm-gfx1030` - AMD gfx1030 driver only
|
||||
- `stable-rocm-gfx1100` - AMD gfx1100 driver only
|
||||
- `stable-h8l` - Frigate build for the Hailo-8L M.2 PICe Raspberry Pi 5 hat
|
||||
|
||||
## Home Assistant Addon
|
||||
|
||||
|
||||
Reference in New Issue
Block a user