Commit Graph

57 Commits

Author SHA1 Message Date
Nicolas Mowen
2da99c2308 Improve periodic sync reliability and make it optional (#8647)
* Improve recordings sync reliability

* Cleanup

* Formatting

* Make logs consistent

* Make syncing optional
2023-11-18 06:06:00 -06:00
Nicolas Mowen
ca84732574 Ensure storage cleanup doesn't fail (#8531) 2023-11-07 23:21:43 +00:00
Nicolas Mowen
cc5357a31a Remove iterator from recordings events (#8522) 2023-11-07 07:02:02 -07:00
Nicolas Mowen
92906a500a Fix event only datetime check (#8500) 2023-11-07 11:32:31 +00:00
Nicolas Mowen
257bd89733 Reduce recording info timeout (#8506)
* Decrease timeout

* Use const for queue timeout
2023-11-07 11:32:11 +00:00
Nicolas Mowen
89dd114da1 Write recording segments to cache with timezone info (#8468)
* Store recording segments with timezone info

* Don't use _

* Use different separator due to timezone
2023-11-05 14:30:29 -06:00
Nicolas Mowen
14c89c9b63 Fix periodic sync check (#8472)
* Fix periodic sync

* Formatting
2023-11-05 09:28:40 -07:00
Nicolas Mowen
63233a5830 Periodically sync for stale recordings (#8433)
* Periodically cleanup recordings files / DB

* Make automatic sync limited ot last 36 hours
2023-11-04 02:21:29 +00:00
Nicolas Mowen
e89dafa82e Handle recording checks in utc (#8379)
* Handle recording checks in utc

* Formatting
2023-10-30 20:25:21 -04:00
Nicolas Mowen
3d70d29672 Delete export if it fails (#8381)
* Delete export if it fails

* Fix import
2023-10-30 20:24:11 -04:00
Josh Hawkins
16dc9f4bf7 update debug message 2023-10-26 17:32:58 -06:00
Josh Hawkins
52b47a3414 empty assumption for events 2023-10-26 17:32:58 -06:00
Josh Hawkins
139664e598 assumption on empty 2023-10-26 17:32:58 -06:00
Josh Hawkins
441c605312 make sure entire segment is accounted for 2023-10-26 17:32:58 -06:00
Josh Hawkins
def889e3a8 start_time is a datetime obj 2023-10-26 17:32:58 -06:00
Josh Hawkins
613f1f6bd6 check frame time for segment 2023-10-26 17:32:58 -06:00
Josh Hawkins
e173377859 change warning to debug 2023-10-26 17:32:58 -06:00
Nicolas Mowen
1a9e00ee49 Add count of audio labels to active count (#8310)
* Add count of audio labels to active count

* Formatting
2023-10-24 19:26:46 -04:00
Blake Blackshear
9fc93c72a0 more consistent use of iterators in select queries (#8258) 2023-10-21 10:53:33 -05:00
Sergey Krashevich
8f349a6365 use sum() instead of len() to count only enabled cameras (#8232) 2023-10-19 17:14:06 -05:00
Nicolas Mowen
98200b7dda Fix recording segment management (#8220)
* Fix timing error

* Downgrade logs
2023-10-18 18:18:22 -05:00
winstona
cd35481e92 Fix recording events intermittently missing (#8162)
* fix queues not emptying fully by changing gets to a blocking call with short timeout

* add extra error/warning messages when there's a possibility of missing recording segments
2023-10-18 06:52:48 -05:00
Nicolas Mowen
4c7ea01137 Don't print ffprobe stdout logs (#8153) 2023-10-13 17:04:38 -05:00
Nicolas Mowen
7f5fba08b7 Fix export name (#8090) 2023-10-07 08:40:20 -06:00
Nicolas Mowen
7d0216b8fb Improve default timelapse args and make timelapse customizable (#7840)
* Add args to ignore audio and only process keyframes

* Add timelapse args to config

* Update docs

* Formatting

* Fix spacing

* Fix formatting

* add example of math for pts
2023-09-21 06:20:05 -06:00
Nicolas Mowen
111933d3b4 Refactor Exports To Better Handle Recording Configs (#7846)
* Refactor export logic

* Fix param

* Ensure float is used

* Fix variable assignment

* Fix range

* Formatting
2023-09-21 05:24:49 -05:00
Nicolas Mowen
3797340efa Set export sub process to be lower priority (#7862) 2023-09-21 05:22:35 -05:00
Nicolas Mowen
a2206bb177 Reduce database queries to necessary fields (#7751)
* Reduce database queries to necessary labels

* Set columns for other queries

* skip creating model instances

---------

Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>
2023-09-11 17:07:04 -05:00
Andrew Reiter
e941012d48 Disambiguate 12h export filename timestamp (#7340)
* Add AM/PM to exported video filename

* Switch to 24h time
2023-08-01 06:25:43 -05:00
Martin Weinelt
d0b38f0a47 Reuse CACHE_DIR constant (#7306) 2023-07-29 13:19:21 -05:00
Nicolas Mowen
f8c0f74ab8 Batch delete recordings for removed cameras as well (#7302) 2023-07-28 05:49:52 -05:00
Nicolas Mowen
761daf46ea Refactor Recordings (#7275)
* Run ffmpeg sub process & video_properties as async

* Run recording cleanup in the main process

* More cleanup

* Use inter process communication to write recordings into the DB

* Formatting
2023-07-26 05:55:08 -05:00
Nicolas Mowen
bfa7a5cc60 db improvements (#7227)
* Store camera labels in dict and other optimizations

* Add max on timeout so it is at least 60

* Ensure db timeout is at least 60

* Update list once a day to ensure new labels are cleaned up

* Formatting

* Insert recordings as bulk instead of individually.

* Fix

* Refactor event and timeline cleanup

* Remove unused
2023-07-21 07:29:50 -05:00
Nicolas Mowen
f2ff55dba2 Set timeout based on number of cameras (#7219) 2023-07-19 07:42:32 -05:00
Nicolas Mowen
dacf45cd88 Add tests for recordings retention and fix bug (#7183)
* Add tests for segment info

* Fix logic
2023-07-16 13:07:15 -05:00
Nicolas Mowen
662025a961 Remove faster_fifo (#7181)
* Remove faster_fifo

* Remove const
2023-07-16 07:42:56 -05:00
Nicolas Mowen
8e584cf844 Add option for enabling sync recordings (#7169) 2023-07-15 08:38:21 -05:00
Nicolas Mowen
d0873631cc Fix logic (#7165) 2023-07-14 20:46:01 -05:00
Nicolas Mowen
00016b7499 Save average dBFS and retain segment with dBFS in motion mode (#7158)
* Hold audio info queue for recordings

* Add dBFS to db

* Cleanup

* Formatting

* Fix check
2023-07-14 19:05:14 -05:00
Nicolas Mowen
baf671b764 Cleanup and organize utils (#7033)
* Force birdseye cameras into standard aspect ratios

* Organize utils

* Update tests

* Formatting

* Isort

* Fix tests

* Cleanup

* isort
2023-07-06 09:28:50 -05:00
Sergey Krashevich
c38c981cd0 Performance: multiprocessing improvement: step 2 (#6986)
* Refactored queues to use faster_fifo instead of mp.Queue

* Refactored LimitedQueue to include a counter for the number of items in the queue and updated put and get methods to use the counter

* Refactor app.py and util.py to use a custom Queue implementation called LQueue instead of the existing Queue

* Refactor put and get methods in LimitedQueue to handle queue size and blocking behavior more efficiently

* code format

* remove code from other branch (merging fuckup)
2023-07-06 07:56:38 -05:00
Nicolas Mowen
c3b313a70d Audio events (#6848)
* Initial audio classification model implementation

* fix mypy

* Keep audio labelmap local

* Cleanup

* Start adding config for audio

* Add the detector

* Add audio detection process keypoints

* Build out base config

* Load labelmap correctly

* Fix config bugs

* Start audio process

* Fix startup issues

* Try to cleanup restarting

* Add ffmpeg input args

* Get audio detection working

* Save event to db

* End events if not heard for 30 seconds

* Use not heard config

* Stop ffmpeg when shutting down

* Fixes

* End events correctly

* Use api instead of event queue to save audio events

* Get events working

* Close threads when stop event is sent

* remove unused

* Only start audio process if at least one camera is enabled

* Add const for float

* Cleanup labelmap

* Add audio icon in frontend

* Add ability to toggle audio with mqtt

* Set initial audio value

* Fix audio enabling

* Close logpipe

* Isort

* Formatting

* Fix web tests

* Fix web tests

* Handle cases where args are a string

* Remove log

* Cleanup process close

* Use correct field

* Simplify if statement

* Use var for localhost

* Add audio detectors docs

* Add restream docs to mention audio detection

* Add full config docs

* Fix links to other docs

---------

Co-authored-by: Jason Hunter <hunterjm@gmail.com>
2023-07-01 08:18:33 -05:00
Nicolas Mowen
9137f1594b Remove legacy recordings file cleanup (#6947)
* Remove legacy recordings file cleanup

* Remove unused
2023-06-30 07:13:00 -05:00
Nicolas Mowen
b66810247e Optimize stacked recordings (#6809)
* Make camera recordings mover asynchronous

* Formatting

* Move to using cv2 instead of external ffmpeg process

* Use ffprobe if cv2 failed

* Formatting

* Fix bad access

* Formatting

* Update frigate/record/maintainer.py

Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>

* Update name of caller

---------

Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>
2023-06-16 07:09:13 -06:00
Nicolas Mowen
ff90db30e6 Various Bug Fixes (#6768)
* Fix birdseye infinite loop

* Fix division by zero
2023-06-12 06:06:02 -04:00
Sergey Krashevich
dfd574beeb Refactor storage stats calculation to use powers of 2 for more accurate values (#6765)
* "Refactor storage stats calculation to use powers of 2 for more accurate values"

* replace 1000000 to 2^20

* Refactor storage unit size display to use binary prefixes

This commit updates the display of storage unit sizes in both the camera storage stats and the Storage component in the web UI to use binary prefixes (MiB and GiB) instead of decimal prefixes (MB and GB). This provides more accurate and consistent representation of storage sizes
2023-06-11 14:49:13 -05:00
Sergey Krashevich
5c27cb7e9b Optimization of Sync Records: Implementing Pagination and Temporary Table (#6585)
* Update pull_request.yml

* Add temporary table for deletion and use pagination to process recordings in chunks for deletion of recordings with missing files

* move RecordingsToDelete class to models.py

* recording cleanup: bugfixes

* Update cleanup.py

* improve log message in cleanup.py

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2023-06-11 08:01:50 -05:00
Nicolas Mowen
8bc76d19db DB Optimizations (#6712)
* Enable auto vacuums

* Enable auto vacuum

* Fix separator

* Fix separator and remove incorrect log

* Limit to 1 row since that is all that is used

* Add index on camera + segment_size

* Formatting

* Increase timeout and cache_size

* Set DB mode to NORMAL synchronous level

* Formatting

* Vacuum every 2 weeks

* Remove fstring

* Use string

* Use consts
2023-06-11 07:23:18 -05:00
Nicolas Mowen
435d152423 Fix bug introduced in new linter (#6754)
* Fix bug introduced in new linter

* Ignore this error

* Fix more

* Ignore boolean error too
2023-06-11 07:18:47 -05:00
Nicolas Mowen
d3949eebfa Add API and WebUI to export recordings (#6550)
* Add ability to export frigate clips

* Add http endpoint

* Add dir to nginx

* Add webUI

* Formatting

* Cleanup unused

* Optimize timelapse

* Fix pts

* Use JSON body for params

* Use hwaccel to encode when available

* Print ffmpeg command when fail

* Print ffmpeg command when fail

* Add separate ffmpeg preset for timelapse

* Add docs outlining the export directory

* Add export docs

* Use ''

* Fix playlist max time

* Lower max playlist time

* Add api docs for export

* isort fixes
2023-06-08 07:32:35 -04:00