"Cutting out" short segments produces many events

[Draxter14]

Hello, Thank you so much for this amazing software!
Love it but would appreciate some help.

I switched from Auto-editor python script because I could never get the settings right for my case and DVR-scan seemed to work out of the box. I'm trying to cut out motionless parts of my drawing videos. I use scan only mode and subsequently process cmd output event data in a spreadsheet to get it into a video editor (Resolve). I've been trying to get the settings right but if I get an acceptable result, the number of motion events that gets output is really high which is cumbersome to work with through the spreadsheet. Many "cut positions" - end of previous to start of a new event - are adjacent to each-other (practically have the same time. So inside the editor I get many clips with continuous source frame count with a redundant cut in-between) or very near in the source footage and I would like to eliminate those and have a continuous event if a pause between motion is shorter than some small specified value. Having a bigger "time before" and "time after" event setting seems to lower the number of output events the most but that defeats my purpose of removing smaller (but not too small) motionless parts from the lengthier motion parts.
I realize that my case scenario is somewhat different from the intended use (e.g. lightning videos...) where you look for motion events in a mostly static video and I'm kinda inverting the logic, but still...
Can you recommend some settings that would "leave out" most of the motionless parts from a video but not produce many events?
Ideally there would be a setting where I could specify an amount of time below which the motionless part wouldn't be treated as such and would be ignored and the motion event before and after it would output as a single event. Similar to "minimum event length" but for no-motion. Something like: if no-motion duration is under x frames, ignore and don't end the event.
I hope this makes sense.

Maybe this has something to do with my issue, IDK.
I should mention that when I change the kernel size to different odd values in the cfg file, it always says default 3 on MOG2 in the cmd initialisation output (I'm downscaling with 2 a 1080i video). I don't know if that is just default info or are my kernel changes really not working. Other .cfg settings are propagating fine it seems.

Also I was reading some discussions about exporting to editors (I think issue #116 and #156 in PySceneDetect and another one I can't find).
If that is still relevant, maybe you can look into Auto-editor. They have the output for XML and FCPXML, that also works in Resolve, figured out. I had to modify one of their .py to make it work for my older Resolve 17 but the export is working well for me.

I'm not sure where to post this. Hope here is fine.

[Breakthrough]

Ideally there would be a setting where I could specify an amount of time below which the motionless part wouldn't be treated as such and would be ignored and the motion event before and after it would output as a single event.

This is the intended purpose for the -tb and -tp options, but as you pointed out there's a bug with -tb in that it does not cause merging right now. Are you able to share any sample footage of what you're using as an input?

Thanks for the diagram, it's definitely helpful, but I'm having trouble understanding why the parameters for time before/post events can't be tuned to achieve the desired result.

Many "cut positions" - end of previous to start of a new event - are adjacent to each-other (practically have the same time. So inside the editor I get many clips with continuous source frame count with a redundant cut in-between) or very near in the source footage and I would like to eliminate those and have a continuous event if a pause between motion is shorter than some small specified value

This is what the --time-post-event option is supposed to be used for, is this not working as intended? Any examples you can share would be very helpful. It's possible this is related to #72 like you mentioned, but I'm also not sure if that would achieve what you want here. If it does though, happy to prioritize that for an upcoming release.

---

Taking a step back, it seems like your issue is having too many events with really low values of pre/post event durations. Would this still be an issue if DVR-Scan produced the output in XML/FCPXML format?

P.S. Thanks for the kind words, it means a lot ❤️

[Draxter14]

Thank you so much for your quick response!
Indeed I was trying with a longer -tb than -tp. Maybe that is part of my issue. Although if tb isn't joining... it should join with tp part, right? I probably should study this some more.

I'm happy to share a sample, just don't know where/how, I'm pretty much a noob here. Probably even a 3 min test video would be too big to upload here, or not?
I can do it tomorrow, getting very late here.

If I understand correctly the pre and post times get added to the motion event from motionless parts. It's like padding. And if combined padding is bigger than the motionless gap, the motion events should join. Is that how it should work?
But I end up with continuous (regarding frames) video clips with cuts (not joined). If this is the bug then hopefully it gets resolved.
I'm editing these videos to make them shorter and and less boring to watch so I can't have the padding to big. Say somewhere up to a second.
But even with the defaults I get events in hundreds because they are not joined, while a few tens of them would be much easier to process and edit.

XML/FCPXML output would be much appreciated although it is version dependent and I had problems with that in the past. I currently opted for the EDL format that is ancient but still in use because of the simplicity I think. So much easier to construct (although it took me almost a week to learn everything to make it happen) and I'm guessing more version independent. And I need that because I'm working on an older machine.
But that would still be awesome!

P.s.: I edited my first post above with a kernel related issue just when I noticed your reply so you probably haven't seen that. A paragraph before the last one, if you'd be so kind to take a look.
Thank you! 🙏

[Draxter14]

Some additional thoughts... (Sorry for the length, I lack shorthand skills)

[1] With recent changes to PySceneDetect (specifically adding a merge strategy), it should be possible to further specify another window where, after the events have been expanded, they can also be considered for merging. For example, if there are two events detected at frames 10-20 and 50-60, and we set -tb=10 and -tp=15, the resulting events will be 0-35 and 40-75. You could then set another parameter to merge any motion events that are N frames close, e.g. --merge=5, that way you can also set an effective distance between events for them to be considered the same (rather than just relying on them overlapping). That way you can also ensure that for a given clip, you know there was no more motion detected for at least some time after the end of the event (otherwise they would be merged and the clip would have been longer).

^This from #68 is exactly how I wish it would work.

I think I see now what is happening with the kernel. The kernel input (.cfg) value takes the downscaling of video into account and gets devided by the downscaling factor and rounded to the nearest odd value. E.g. 11 on input becomes 5 on initialisation. I wasn't inputting high enough values to see the difference before.

If you include XML/FCPXML export, please keep the current CLI output HH:MM:SS.000, HH:MM:SS.000,... line with --scan-only at least as an option. For a fallback if we have version mismatches with our downstream pipelines.

[Breakthrough]

If I understand correctly the pre and post times get added to the motion event from motionless parts. It's like padding. And if combined padding is bigger than the motionless gap, the motion events should join. Is that how it should work?

Yes correct - that is how things should work, but does not yet as per #72. This is most likely what you're seeing unfortunately. I'll make sure to prioritize that for an upcoming release.

[...] it should be possible to further specify another window where, after the events have been expanded, they can also be considered for merging [...]

^This from #68 is exactly how I wish it would work.

Thanks for refreshing my memory on that. Could you file a feature request for this in the DVR-Scan repo? Would be happy to add a new option called "merge-window" or something to allow this to be specified.

If you include XML/FCPXML export, please keep the current CLI output HH:MM:SS.000, HH:MM:SS.000,... line with --scan-only at least as an option. For a fallback if we have version mismatches with our downstream pipelines.

Will do. I don't think there's an open issue for tracking XML support either - feel free to submit a new one for that as well.

---

Regarding the kernel size thing:

I should mention that when I change the kernel size to different odd values in the cfg file, it always says default 3 on MOG2 in the cmd initialisation output (I'm downscaling with 2 a 1080i video). I don't know if that is just default info or are my kernel changes really not working. Other .cfg settings are propagating fine it seems.

I'm unable to reproduce this on my end, my changes to kernel size seem to work okay. Could you share your config file and the DVR-Scan command you're using?

[Draxter14]

Firstly, thank you so much, your support is impeccable! Also, an innocent bystander reading all my whining might think that this software is problematic... but no no, let me tell you, DVR-scan is the best in what it does out of everything I've tried, in the known universe! :)

Could you file a feature request for this in the DVR-Scan repo? Would be happy to add a new option called "merge-window" or something to allow this to be specified.

Thanks, I'll be adding the feature requests, just need to learn how to do it right.

This is what I've been able to figure out about the kernel (commenting myself below):

The kernel input (.cfg) value takes the downscaling of video into account and gets devided by the downscaling factor and rounded to the nearest odd value. E.g. 11 on input becomes 5 on initialisation. I wasn't inputting high enough values to see the difference before.

Here is my command as per your request. I haven't installed anything, running a portable version from CLI (cmd) started from Explorer folder with the source video.
My command:
D:\BI\DVR-Scan-1.6.2-win64\dvr-scan.exe -c D:\BI\DVR-Scan-1.6.2-win64\dvr-scan.cfg -i video.mp4 -R regions.txt
I don't know if the order of parameters matters, haven't been able to find instructions on that.
I also didn't add DVR-scan to %PATH% or anything like that if I remember correctly because it just worked (because I'm calling it directly?) but maybe I should have. (Bit of a noob.)
Attaching my latest version of .cfg (zipped)
dvr-scan_cfg.zip
Since our discussion I went with smaller -tb and bigger -tp and it does seem to improve the output (less of redundant cuts, although I haven't been able to test extensively).

Let me know if you still need the video sample. I've cut out a 1 min clip but it's just over 50Mb. I probably can't share it here? I could try to compress it more, I suspect it probably wouldn't influence the results much.