Unable to use Tensor detector with Nvidia GPU via Docker Compose #7815

diggity801 · 2023-09-15T05:28:39Z

diggity801
Sep 15, 2023

Hello,

I need help running a Tensor detector using my NVidia GPU. I receieve an error when I run it with docker compose, the detector fails to load and states that it cannot find the files. Did I map them wrong in Docker Compose? So confusing as the documentation doesn't cover this very important sub topic.

Thank you!

docker-compose.yml:

version: "3.7"
services:
frigate:
image: ghcr.io/blakeblackshear/frigate:stable-tensorrt
runtime: nvidia
restart: unless-stopped
ports:
- "80:5000"
- "8554:8554" # RTSP feed
- "8555:8555/tcp" # WebRTC over TCP
- "8555:8555/udp" # WebRTC over UDP
devices:
- /dev/dri:/dev/dri
environment:
- FRIGATE_RTSP_PASSWORD=12345678
- NVIDIA_DRIVER_CAPABILITIES=compute,utility,video
- NVIDIA_VISIBLE_DEVICES=all
volumes:
- ./config:/config
- ./data:/media/frigate
- /home/nvr/frigate/trt-models:/trt-models

deploy:    # <------------- Add this section
  resources:
    reservations:
      devices:
        - driver: nvidia
          count: 0 # number of GPUs
          capabilities: [gpu]
shm_size: '2gb'
build:
  context: .
  shm_size: '2gb'

config.yml:

snapshots:
enabled: true
clean_copy: true
crop: true
detect:
fps: 5

detectors:
tensorrt:
type: tensorrt
device: 0 #This is the default, select the first GPU

model:
path: /trt-models/yolov7-tiny-416.trt
input_tensor: nchw
input_pixel_format: rgb
width: 416
height: 416
#detectors:

ov:

type: openvino

device: AUTO

model:

path: /openvino-model/ssdlite_mobilenet_v2.xml

#model:

width: 300

height: 300

input_tensor: nhwc

input_pixel_format: yuv

labelmap_path: /openvino-model/coco_91cl_bkgr.txt

mqtt:
enabled: false
objects:
track:
- person
- bicycle
- cat
- dog
timestamp_style:
position: "tl"
format: "%m/%d/%Y %H:%M:%S"
color:
red: 255
green: 255
blue: 255
thickness: 2
cameras:
doorbell_camera:
enabled: true
ffmpeg:
hwaccel_args: preset-nvidia-h264
output_args:
record: preset-record-generic-audio-aac
inputs:
- path: rtsp://admin:[email protected]:554/h264Preview_01_main
roles:
- record
- path: rtsp://admin:[email protected]:554/h264Preview_01_sub
roles:
- detect
detect:
fps: 5
entrance_camera:
enabled: true
ffmpeg:
hwaccel_args: preset-nvidia-h264
input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -timeout 30000000 -use_wallclock_as_timestamps 1
output_args:
detect: -dn -an -f rawvideo -pix_fmt yuv420p
inputs:
- path: rtsp://10.0.0.2:8554/entrance_camera
roles:
- record
- detect
detect:
fps: 5
garage_camera:
enabled: true
ffmpeg:
hwaccel_args: preset-nvidia-h264
input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -timeout 30000000 -use_wallclock_as_timestamps 1 #-rtsp_transport udp
output_args:
detect: -dn -an -f rawvideo -pix_fmt yuv420p
inputs:
- path: rtsp://10.0.0.2:8554/garage_camera
roles:
- record
- detect
entryway_window_camera:
enabled: true
ffmpeg:
hwaccel_args: preset-nvidia-h264
output_args:
record: preset-record-generic-audio-aac
inputs:
- path: rtsp://10.0.0.2:8554/north_standard_camera
roles:
- record
- detect
detect:
fps: 5
living_room_camera:
enabled: true
ffmpeg:
hwaccel_args: preset-nvidia-h264
input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -timeout 30000000 -use_wallclock_as_timestamps 1 #-rtsp_transport udp
output_args:
detect: -dn -an -f rawvideo -pix_fmt yuv420p
inputs:
- path: rtsp://10.0.0.2:8554/unused_camera_10
roles:
- record
- detect
gym_camera:
enabled: true
ffmpeg:
hwaccel_args: preset-nvidia-h264
input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -timeout 30000000 -use_wallclock_as_timestamps 1
output_args:
detect: -dn -an -f rawvideo -pix_fmt yuv420p
inputs:
- path: rtsp://10.0.0.2:8554/gym_camera
roles:
- record
- detect
entryway_camera:
enabled: true
ffmpeg:
hwaccel_args: preset-nvidia-h264
input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -timeout 30000000 -use_wallclock_as_timestamps 1 #-rtsp_transport udp
output_args:
detect: -dn -an -f rawvideo -pix_fmt yuv420p
inputs:
- path: rtsp://10.0.0.2:8554/entryway_camera
roles:
- record
- detect
unused_camera_4:
enabled: true
ffmpeg:
hwaccel_args: preset-nvidia-h264
input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -timeout 30000000 -use_wallclock_as_timestamps 1 #-rtsp_transport udp
output_args:
detect: -dn -an -f rawvideo -pix_fmt yuv420p
inputs:
- path: rtsp://10.0.0.2:8554/third_floor_landing
roles:
- record
- detect
record:
enabled: true
retain:
days: 30
mode: all
events:
retain:
default: 30
mode: all
birdseye:
enabled: true
mode: continuous
quality: 1
#height: 1080
#width: 1920

live:
quality: 1
ui:
timezone: America/Los_Angeles
live_mode: webrtc
use_experimental: false
go2rtc:
streams:
doorbell_camera:
- rtsp://admin:[email protected]:554/h264Preview_01_main
- "ffmpeg:doorbell_camera#audio=opus"
entrance_camera:
- rtsp://10.0.0.2:8554/entrance_camera
garage_camera:
- rtsp://10.0.0.2:8554/garage_camera
entryway_window_camera:
- rtsp://10.0.0.2:8554/north_standard_camera
living_room_camera:
- rtsp://10.0.0.2:8554/unused_camera_10
gym_camera:
- rtsp://10.0.0.2:8554/gym_camera
unused_camera_4:
- rtsp://10.0.0.2:8554/third_floor_landing
entryway_camera:
- rtsp://10.0.0.2:8554/entryway_camera
webrtc:
candidates:
- 10.0.0.176:8555
- stun:8555

**log**
2023-09-15 05:26:57.309214429  [INFO] Starting Frigate...

2023-09-15 05:26:57.916499761 [2023-09-15 05:26:57] frigate.app INFO : Starting Frigate (0.12.1-367d724)
2023-09-15 05:26:57.948403291 [2023-09-15 05:26:57] peewee_migrate INFO : Starting migrations
2023-09-15 05:26:57.950771328 [2023-09-15 05:26:57] peewee_migrate INFO : There is nothing to migrate
2023-09-15 05:26:57.959814668 [2023-09-15 05:26:57] frigate.app INFO : Output process started: 721
2023-09-15 05:26:57.960240578 Process detector:tensorrt:
2023-09-15 05:26:57.960274066 [2023-09-15 05:26:57] detector.tensorrt INFO : Starting detection process: 720
2023-09-15 05:26:57.960728424 Traceback (most recent call last):
2023-09-15 05:26:57.960741164 File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-09-15 05:26:57.960742293 self.run()
2023-09-15 05:26:57.960746300 File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-09-15 05:26:57.960748015 self._target(*self._args, **self._kwargs)
2023-09-15 05:26:57.960752518 File "/opt/frigate/frigate/object_detection.py", line 98, in run_detector
2023-09-15 05:26:57.960753430 object_detector = LocalObjectDetector(detector_config=detector_config)
2023-09-15 05:26:57.960754284 File "/opt/frigate/frigate/object_detection.py", line 52, in init
2023-09-15 05:26:57.960755087 self.detect_api = create_detector(detector_config)
2023-09-15 05:26:57.960756140 File "/opt/frigate/frigate/detectors/init.py", line 24, in create_detector
2023-09-15 05:26:57.960770126 return api(detector_config)
2023-09-15 05:26:57.960771106 File "/opt/frigate/frigate/detectors/plugins/tensorrt.py", line 198, in init
2023-09-15 05:26:57.960772098 assert (
2023-09-15 05:26:57.960776335 AssertionError: TensorRT libraries not found, tensorrt detector not present
2023-09-15 05:26:57.960784823 Exception ignored in: <function TensorRtDetector.del at 0x7fe8081b0550>
2023-09-15 05:26:57.960785624 Traceback (most recent call last):
2023-09-15 05:26:57.960788930 File "/opt/frigate/frigate/detectors/plugins/tensorrt.py", line 238, in del
2023-09-15 05:26:57.960865121 if self.outputs is not None:
2023-09-15 05:26:57.960869032 AttributeError: 'TensorRtDetector' object has no attribute 'outputs'
2023-09-15 05:26:57.963180425 [2023-09-15 05:26:57] frigate.app INFO : Camera processor started for doorbell_camera: 725
2023-09-15 05:26:57.965371717 [2023-09-15 05:26:57] frigate.app INFO : Camera processor started for entrance_camera: 727
2023-09-15 05:26:57.970427518 [2023-09-15 05:26:57] frigate.app INFO : Camera processor started for garage_camera: 729
2023-09-15 05:26:57.970540245 [2023-09-15 05:26:57] frigate.app INFO : Camera processor started for entryway_window_camera: 730

NickM-27 · 2023-09-15T12:08:18Z

NickM-27
Sep 15, 2023
Collaborator Sponsor

Why is there openvino config there?

I think a proper issue would be easier to work though, but it's also worth noting that the 0.13 beta makes this process easier since the models are built automatically and held within the config folder.

0 replies

diggity801 · 2023-09-15T14:24:45Z

diggity801
Sep 15, 2023
Author

Oh interesting. I'll try that beta to generate the models. I commented out OpenVINO detector to try the Tensor detector, which is failing as you know. I appreciate the reply and your time! I'll try that and post back tonight.

…

On Fri, Sep 15, 2023, 5:08 AM Nicolas Mowen ***@***.***> wrote: Why is there openvino config there? I think a proper issue would be easier to work though, but it's also worth noting that the 0.13 beta makes this process easier since the models are built automatically and held within the config folder. — Reply to this email directly, view it on GitHub <#7815 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA76IIC2UWVRROBMEO6PRU3X2RALZANCNFSM6AAAAAA4ZISNOA> . You are receiving this because you authored the thread.Message ID: ***@***.*** com>

1 reply

NickM-27 Sep 15, 2023
Collaborator Sponsor

Oh interesting. I'll try that beta to generate the models.

be sure to read the release notes and the new docs -- things have changed and there are new requirements

diggity801 · 2023-09-16T19:31:40Z

diggity801
Sep 16, 2023
Author

Got it working! Couldn't have done it without you Nick!

1 reply

NickM-27 Sep 16, 2023
Collaborator Sponsor

You're welcome, I notice you have a camera running detect @ 20 fps. I'd suggest turning that down in the detect settings, this does not need to match the camera fps. The default is 5 and in my experience there is no need to go above 10 for fast moving objects

BobHasNoSoul · 2025-06-21T19:29:58Z

BobHasNoSoul
Jun 21, 2025

i have the same issue running the latest beta

any chance someone can talk me through why it cannot find the model?

1 reply

hawkeye217 Jun 21, 2025
Collaborator

See the official documentation: https://deploy-preview-16390--frigate-docs.netlify.app/configuration/object_detectors#generate-models

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Unable to use Tensor detector with Nvidia GPU via Docker Compose #7815

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 4 comments 3 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

Unable to use Tensor detector with Nvidia GPU via Docker Compose #7815

Uh oh!

Uh oh!

diggity801 Sep 15, 2023

ov:

type: openvino

device: AUTO

model:

path: /openvino-model/ssdlite_mobilenet_v2.xml

width: 300

height: 300

input_tensor: nhwc

input_pixel_format: yuv

labelmap_path: /openvino-model/coco_91cl_bkgr.txt

Replies: 4 comments · 3 replies

Uh oh!

NickM-27 Sep 15, 2023 Collaborator Sponsor

Uh oh!

diggity801 Sep 15, 2023 Author

Uh oh!

NickM-27 Sep 15, 2023 Collaborator Sponsor

Uh oh!

diggity801 Sep 16, 2023 Author

Uh oh!

NickM-27 Sep 16, 2023 Collaborator Sponsor

Uh oh!

BobHasNoSoul Jun 21, 2025

Uh oh!

hawkeye217 Jun 21, 2025 Collaborator

diggity801
Sep 15, 2023

Replies: 4 comments 3 replies

NickM-27
Sep 15, 2023
Collaborator Sponsor

diggity801
Sep 15, 2023
Author

NickM-27 Sep 15, 2023
Collaborator Sponsor

diggity801
Sep 16, 2023
Author

NickM-27 Sep 16, 2023
Collaborator Sponsor

BobHasNoSoul
Jun 21, 2025

hawkeye217 Jun 21, 2025
Collaborator