Frigate+ 2025.0 Base Model Update #16132
Replies: 14 comments 67 replies
-
|
yea i was just wondering is moving off my coral to a nvidia card for 640x640 worth it due to the bigger resolution and increased sensitivity for smaller objects. whats the current recommended way |
Beta Was this translation helpful? Give feedback.
-
I have an i5-7400 so Intel HD 630. Not sure how it compares to the n100. Quick comparison over here to some benchmarks I should be a little better off than the n100. But it's not apples to apples at all. https://gist.github.com/ironicbadger/5da9b321acbe6b6b53070437023b844d?ref=blog.ktz.me |
Beta Was this translation helpful? Give feedback.
-
Oh really??? I'm sorry, I'm really behind then. Currently, I use all the same 4k cameras, the amcrest 4k. My detect is: For all cameras. The main streams are capable of 3840x2160. I go into the admin panel of the camera model, and the main stream says it is the 3840x2160. Do I just have to change the detect width and height to 3840x2160? Do I need to change anything else? |
Beta Was this translation helpful? Give feedback.
-
|
At 1080p, you should already be able to detect objects far away. How far away are you talking about? This might be a issue with motion detection sensitivity, not resolution. I would recommend using the debug view to first see if you get motion boxes for objects that far away. If you do, then see how big the region boxes are. Those show you exactly what is sent to the detector. If you have good sized region boxes around these far away objects and it's still not detecting, then the model isn't returning scores above the min_score. That could just mean you need a lower min_score or that you may actually benefit from using the yolonas model. The way frigate zooms in and crops down to the region typically compensates well for mobiledet not detecting small objects. Once it crops the frame, the small object isn't really small anymore. |
Beta Was this translation helpful? Give feedback.
-
|
It's about a 300 foot driveway. Generally don't get detection til about 50 feet or less, and by that time, it's not much of a heads up, They're already at my door. |
Beta Was this translation helpful? Give feedback.
-
|
It all depends on how large a person is relative to a 320x320 cropped area from your 1080p camera. You should easily be able to detect a person that is 1/4 of the height of your frame since they would be roughly 300px tall. This would be considered a "large" object, not a small object. I would consider a small object to be 100px tall or less, so something 1/10th of your frame height. |
Beta Was this translation helpful? Give feedback.
-
|
Is the config verb for waste bin waste_bin? |
Beta Was this translation helpful? Give feedback.
-
|
yes that is the label |
Beta Was this translation helpful? Give feedback.
-
|
I've got a pair of Corals and a P2200 for frigate/plex- might be time to transition away from the coral with this update. |
Beta Was this translation helpful? Give feedback.
-
|
@NickM-27 @blakeblackshear is it possible to do 640x640 with Corals in the future? |
Beta Was this translation helpful? Give feedback.
-
|
It's unlikely to make sense. The model architectures that could possibly work are still too slow to run in realtime in my opinion. I think people are vastly overestimating the benefit of a 640x640 model. Bigger is not necessarily better. Frigate was specifically designed to compensate for using a smaller 320x320 model. Using a 640x640 model can actually make performance worse because it limits how far Frigate will zoom in on the image when running detection. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the tempering of expectations @blakeblackshear I guess sometimes we like the new shiny too much 😆 |
Beta Was this translation helpful? Give feedback.
-
|
Very nice, awesome to see the constant progression. Very happy so far. The 640x640 from the 24.3 model has improved my detection accuracy by quite a bit. Although I'm not sure how an Nvidia card with the 640x640 compares to a coral device - my old coral system handles about 10 cameras without problems, when using my Nvidia RTX 4000 with onnx it did hit 100% CPU use on the detector after just 5 cameras (4K each). |
Beta Was this translation helpful? Give feedback.
-
Are you sure that's not just initially while it builds the model? |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks Blake! Do I need to regenerate my yolov7x-320 or yolov7x-640 to take use of these? I will say I did move from a coral tpu to RTX3060 and the difference in detection and accuracy is pretty crazy. Inference speed isn't much faster but it's more consistent and accurate. |
Beta Was this translation helpful? Give feedback.
-
|
The yolonas models are entirely separate, unrelated to your trt models |
Beta Was this translation helpful? Give feedback.
-
|
Blake, I assume suggestions are still by request? What is the timeline to have these implemented either on demand in the web UI or automatically on upload? |
Beta Was this translation helpful? Give feedback.
-
|
They are not by request. They run automatically when you upload as long as you are an active subscriber. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for your hard work! I've been using the coral mobiledet model, is there a way to try yolonas 640x640 using the same model generate request, without having to make 2 requests? |
Beta Was this translation helpful? Give feedback.
-
|
I’m interested in this too since I’m out of requests now. |
Beta Was this translation helpful? Give feedback.
-
|
There isn't. These are completely different training pipelines, so each one requires it's own GPU time. |
Beta Was this translation helpful? Give feedback.
-
|
Gotchya, thank you!! |
Beta Was this translation helpful? Give feedback.
-
|
Couple of quick questions. Reading the above comments, seems the larger resolution isn't a good idea for a coral? Is the model done per camera or collectively. I have a camera where I plan to rotate 30-40 degrees due to a tree growing in its FOV that is constantly detected as a waste bin. Does this undo the work so far and should I delete all images? Any way to delete an old camera name from frigate+ with no images? Is the Australia Post logo on the list for future? |
Beta Was this translation helpful? Give feedback.
-
I think you are mixing up different concepts, or I don't understand what you mean. The coral only has a 320x320 model, while yolonas has a 320x320 model. This is unrelated to the resolution of your camera or detect resolution.
it is one model for all images you have submitted and annotated.
adjusting cameras in general doesn't affect things, you can adjust without conflicting with already submitted images
You'll need to email [email protected] |
Beta Was this translation helpful? Give feedback.
-
|
Legend, thanks |
Beta Was this translation helpful? Give feedback.
-
|
Will these new labels be auto-detected on Frigate+ website in the future or now? I love this new feature and think it's pretty cool and hopeful even if I am not detecting these for my needs the community could benefit from image submissions that "accidentally" have these in them as well. |
Beta Was this translation helpful? Give feedback.
-
|
It says Amazon though. Not waste bin. You need to make it a false positive of amazon Oh wait, the top corner, I see it now. Weird. I have heaps of things not ticked and I never get birds or deer or whatever else suggested |
Beta Was this translation helpful? Give feedback.
-
And if I wanted to select waste_bin for the bin I have to go toggle on the annotation for that camera to do so. I marked the one with Amazon as I was trying to cycle through and select waste_bin but since it isn't turned on for annotation I can't. |
Beta Was this translation helpful? Give feedback.
-
|
There is already a request to select labels that aren't enabled for your camera. Probably some kind of box you can uncheck to toggle limiting the options. When you add new labels, that kicks off a suggestion job to scan all of the images for that camera for the new labels. That's why adding waste bin causes suggestions to start appearing. But if you don't actually care whether your model can detect those labels, there isn't a good reason to add them at all. |
Beta Was this translation helpful? Give feedback.
-
|
Semi-related - say someone does want to go through all of their old images to annotate Horse, but it's a relatively rare occurrence: I press Space to save-and-next a lot. But loading the next image is relatively slow. If I press back, and then Space again, it loads almost instantly presumably because the browser already has it cached. Could you force it to preload the next image or five, thereby making it much faster to quickly go through a backlog of old images for a single new label? |
Beta Was this translation helpful? Give feedback.
-
Yes. This is already on my radar. |
Beta Was this translation helpful? Give feedback.
-
|
I guess I haven't even considered my detect resolution after setting up everything initially. Considering testing openvino but was wondering if I should also up my sub stream resolutions to 1080 vs 720? CPU/GPU usage is almost nothing and the coral does just fine with 720p currently. I don't tend to notice detections going unnoticed but I guess I wouldn't know unless I watched consistently. |
Beta Was this translation helpful? Give feedback.
-
|
Can always try, the only difference will be CPU and CPU usage, the coral is not affected directly by resolution |
Beta Was this translation helpful? Give feedback.
-
|
Just curious what the Frigate+ criteria is for including infrared night shots (mostly for people and cars)? are you particular about motion blur, graininess, overexposure? should we only be including really clean shots? |
Beta Was this translation helpful? Give feedback.
-
I have examples in my training set similar to those lighting conditions. |
Beta Was this translation helpful? Give feedback.
-
|
@blakeblackshear has waste bins that the model identifies that are barely visible on a camera frame in darkness 😂 EDIT: well, that was silly. I just had to look up a few comments to see that Blake already posted those waste bins. |
Beta Was this translation helpful? Give feedback.
-
|
My model is surprisingly accurate on detecting cars at night. Like Blake said - it can recognize the proper configuration of just a couple lights, even at a distance. It did take a while to get to this point though. At first it missed everything. Then I had a bunch of false positives. Now it's fantastic. My rule of thumb is if a human could reasonably tell what the object is (even if it takes them a minute to study the image) with no additional context (being familiar with the scene, having seen prior clearer frames, etc.), then annotate it. |
Beta Was this translation helpful? Give feedback.
-
|
Am i better off not training using images like this @blakeblackshear 2 bins in each shot here. I feel I should delete these from the model and use ones with slightly better lighting. I'm also tempted to put my bins out on thw wrong day to get some shots with no other bins to train on. |
Beta Was this translation helpful? Give feedback.
-
|
I wouldn't label those in the base model training set. It depends on what the same area would look like in those lighting conditions when the bins aren't there. During training the model is constantly being tested to determine which variations are best. If it's starting to assume lots of things are waste bins incorrectly, it gets it's hand slapped during training. It will then be less confident on difficult labels appropriately. Adding examples of waste bins that are difficult isn't much of a problem. Leaving clearly visible waste bins unlabeled is what will really create confusion. |
Beta Was this translation helpful? Give feedback.
-
|
My front camera is 180degree FOV. Do you tag all these cars in the distance in order to not confuse the model. |
Beta Was this translation helpful? Give feedback.
-
|
Those are the suggestions. missing 20ish |
Beta Was this translation helpful? Give feedback.
-
|
Which is still much better than it used to be :). I'm not sure if suggestions are based on the current base model, or your own model, or what. Hopefully they continue to improve in the future. |
Beta Was this translation helpful? Give feedback.
-
|
Oh for sure. Even I'm struggling to distinguish between the cars at the end of the street on the left. But they are cars, so I'm trying to tage them. |
Beta Was this translation helpful? Give feedback.
-
|
The suggestions is a larger model trained specially for suggestions |
Beta Was this translation helpful? Give feedback.
-
|
Good to know |
Beta Was this translation helpful? Give feedback.
-
|
The new 2025.0 model offers a higher resolution for yolonas, but not mobiledet? Why not? I'm not understanding the explanation provided above. If, for example, I have 11x cameras (detection @ 10fps 1280x720) and my assigned coral only ever ticks over at a few percent of CPU utilization and 6ms inference speed. It seems like it should be able to handle significantly more load, no? And having the option for higher accuracy for "smaller objects" has strong appeal, as we have a lot of normal-sized, but "far field" objects and would like to see fewer false negatives (and positives). The further away the object (or the smaller it is) it seems like having 640x480 would enable more accurate object identification. Otherwise, if 640x480 doesn't offer an improvement for some use cases, then why even offer it for yolonas? Please explain |
Beta Was this translation helpful? Give feedback.
-
corals do not have utilization. They run a single model at a given speed, and can only run a single inference at a time. If you are referring to the CPU usage shown in the UI this refers to the CPU used to feed the coral with data to run inferences, not the utilization of the coral itself. |
Beta Was this translation helpful? Give feedback.
-
|
You can look at the coral docs for example to see how significantly model size impacts inference time https://coral.ai/models/object-detection/
|
Beta Was this translation helpful? Give feedback.
-
|
Beauty explanation thank you!! |
Beta Was this translation helpful? Give feedback.
-
|
Indeed, 6.5ms to 229.4ms would be quite the jump. Thank you for the deeper explanation. |
Beta Was this translation helpful? Give feedback.
-
|
I did try training a mobiledet model at a higher resolutions. It takes substantially longer because the batches of images that fit in memory are much smaller. I also saw worse accuracy at higher resolutions. The YOLO architectures are designed from the ground up to detect small and large objects in the same image accurately. Also, I don't think higher resolutions are necessarily better as I explained here. |
Beta Was this translation helpful? Give feedback.
-
|
Running Frigate on i5-12500 (UHD 770 iGPU), with 640x640 model ov time went from 20-30ms to 50-60ms, CPU load from 20% went to 60%, rolled back to 320x320, will try again when I add nvidia GPU to my frigate server. |
Beta Was this translation helpful? Give feedback.
-
|
This is consistent with what I would expect. |
Beta Was this translation helpful? Give feedback.
-
The base model in Frigate+ has now been updated to 2025.0. All model requests going forward will use this version. In order to get a new model, you will need to submit a model request.
This update was focused on incorporating additional training data from the new labels added in 2024.3 (motorcycle, bicycle, boat, usps, dhl, an post, purolator, postnl, nzpost, postnord, gls, dpd, horse, bird, raccoon, fox, bear, cow, squirrel, goat, rabbit, waste bin, bbq grill, robot lawnmower, umbrella). In addition, thousands of additional examples of reported false positives were incorporated.
Along with this update, you will see that yolonas model requests now generate two different sizes: 640x640 and 320x320. If you have sufficiently powerful hardware, the 640x640 model offers higher accuracy especially for smaller objects. When using the larger model, you can expect GPU/CPU usage for the detector to double and it will have 2x slower inference times. However, the detections per second should also decrease because it can scan larger parts of the frame at once. I would love to hear your feedback on the larger size as the training data isn't always representative of real world Frigate usage.
Beta Was this translation helpful? Give feedback.
All reactions