[Feature Request] automatic clip backup

[somejavadev]

Describe what you are trying to accomplish and why in non technical terms
I want to be able to able to record to multiple locations to prevent losing storage should the Frigate NVR be stolen/taken during a robbery or the primary disk location dies such as in a hard drive failure.

Describe the solution you'd like
When it comes to specifying the storage location for recordings providing multiple locations should be possible such as NFS, SMB or FTP locations as well as the local disk location. Recordings should then be saved to both/all locations specified.

Describe alternatives you've considered
Alternatives would be to set this up at file level using OS file synchronisation (rsync, syncthing, nightly backups etc.) and have a job manually run to sync the locations although this is not ideal and the possibility for the redundant location to be out of sync by a couple of minutes/hours is quite large.

Additional context
Having redundant storage locations would be ideal to cover scenarios of theft of the actual NVR or a sudden failure of a mini PC's SSD. Additionally one could potentially configure this at a per camera level. This feature is available on NVR's such as hikvision where you can add a NFS mount and configure the camera's to store on both the local disk and remote location.

[NickM-27]

Recording segments to multiple locations doesn't really help in the scenario you're describing because frigate is needed to playback the recording segments effectively. It would likely make more sense to have something that automatically exported full clips of events only instead.

[somejavadev]

@NickM-27 thanks that does make sense as the clips needs to still be playable at the end of the day. I wonder if one cannot hook into the OS file system to determine if a new completed clip has been created and then fire a rsync command for the clip as a work around.

[NickM-27]

I think it would make more sense it have a feature built in to frigate to send these clips if a user enables it to a separate volume.

[cwmchenry]

FWIW: this would be incredibly useful for me as well (long-time Frigate user/fan).

Already have a sync solution in place such that my server automatically uploads files added to a specified path to remote storage in basically real-time. Using it currently with the snapshot "clips" just to have some form of safety net in the event of a burglary. However, being able to specify a secondary storage path for Frigate to automatically export clips of detected events would be ideal for this type of safety net. Doesn't even need to have any fancy built-in cloud storage support--just let the user specify a path via a Docker volume mount and take care of the remote-storage logistics on their end.

Thanks for all your work! Amazing software.

[distinctjuggle]

Recording segments to multiple locations doesn't really help in the scenario you're describing because frigate is needed to playback the recording segments effectively. It would likely make more sense to have something that automatically exported full clips of events only instead.

Why does Frigate record 10 second clips as opposed to continuously writing to the disk with larger (configurable) file sizes for situations of 24/7 recording? All that I can see it accomplishing for a 24/7 recording setup is increased overhead (both in I/O and technically file sizes), reduced compatibility, and added complexity for inducing failure.

I understand the underlying motivation for this is that some users only record when motion is detected, thus the thought process is that this reduces unnecessary writes. But why does this carry a consequence into 24/7 recording setups as well? The inability of easy playback outside of Frigate is a huge issue - I see zero reason why it shouldn't be possible for a user to directly copy a file out of their storage directory and play it back as desired. Sure, the exporting tool usually handles this for us, but why is that necessary by design? Why can't Frigate have a separate mode for continuous 24/7 recording in which a user defined file runtime exists (5 minute intervals, 15 minute intervals, etc)?

Is this too much of a technical problem given the existing Frigate design, or is there some other reason why it isn't already implemented? Personally, I see the saving of all possible footage as the highest priority of an NVR. It's the explicit goal of a video recorder.

[NickM-27]

The 10 second segments is optimal for a number of reasons:

1. The segments are loaded partially via HLS, 10 seconds is optimal for HLS
2. MP4 files are corrupt if they're ended abruptly. The 10 second segments limit how much recording can be lost if a camera stream crashes.
3. It's also less IO in many cases because the recordings are written in RAM and only moved to storage if it matches the retention config

Even if you record continuously you often want to keep recordings of objects longer and this allows that.

It also makes exporting much simpler

[distinctjuggle]

The 10 second segments is optimal for a number of reasons:

1. The segments are loaded partially via HLS, 10 seconds is optimal for HLS

2. MP4 files are corrupt if they're ended abruptly. The 10 second segments limit how much recording can be lost if a camera stream crashes.

MP4 containers aren't a requirement for recording, no? Mkv surely would cover anything.

More relevant to your point, MP4 files can be written with headers at the start of the file to avoid that issue. It's just an option. Or, .ts files can be used.

Even beyond that, all encoding for the same camera will be identical, so the headers can be borrowed. Recordings can be easily recovered using a tool like: https://github.com/anthwlock/untrunc

I had to do this a few weeks ago with a different NVR program, and instead of losing those potential seconds, I recovered all of the final data, easy as cake. That's far better than having it discarded. This could even be automated within Frigate if it was really that important - there'd be limited reason to ever discard the 10 second clips, as is currently the case on my system with one camera which is acting up. The program was super easy to use, one very straightforward command and it was working.

3. It's also less IO in many cases because the recordings are written in RAM and only moved to storage if it matches the retention config

Even if you record continuously you often want to keep recordings of objects longer and this allows that.

I don't see why it can't do both. Other systems do.

It also makes exporting much simpler

How? It's splicing together several videos as opposed to splicing together fewer. It's just ffmpeg either way. And regardless, you could directly download the longer form video with the other method. Exporting is a neat tool, don't get me wrong, but it wouldn't even be necessary in many cases otherwise.

---

The main thing I take issue with is the added overhead, and the loss of the final seconds of a dying stream.

[NickM-27]

The 10 second segments for HLS are optimal for fast loading and the way that frigate frontend works

How? It's splicing together several videos as opposed to splicing together fewer. It's just ffmpeg either way. And regardless, you could directly download the longer form video with the other method. Exporting is a neat tool, don't get me wrong, but it wouldn't even be necessary in many cases otherwise.

Yes but it makes the ffmpeg command much simpler to generate dynamically as you don't need to calculate times to seek into specific segments for the start and end.

The current implementation works very well for the many different config options frigate offers. Implementing a larger mp4 segment would just be extra work and maintenance, requiring implementing a separate player in the UI to accomplish the same thing for the end users perspective. Frigate offers many APIs for interacting recordings and users shouldn't be interacting with the segments directly anyway.