Beats: Add editing controls and show downbeats on scrolling waveforms [vn+1]

[acolombier]

This PR is based on @fwcd 's PR #12343, itself based on @Holzhaus 's #4489 PR.

Kooha-2024-06-07-18-43-20.mp4

This attempt TL;DR; is: do not disrupt the way the beat grid currently works

This is how this attempt differs from the previous one:

• BPM doesn't get change when adding or removing a new marker
• Adjusting the BPM will have a consistent behaviour whether editing a specific marker or the end of track marker
• Faster/slower BPM will adjust the BPM by 0.01, and will auto repeat if the button stays pressed for more than half a second
• Faster/slower BPM will behave like above but with a 0.1 if the right click is used instead
• Translating the grid will only impact the current marker, and the button can stay pressed to repeat
• Beatgrid markers can be turn on and off as waveform options
• Beat colours support alpha

TODO:

• Fix Serato beat grid import
• Store the bar length using the beatmap
• Implement missing usecase
• Fix the "last beat" not displayed when the next marker grid uses a different beat length

Fixes #13308

Disclaimer:
I know there was a lot of discussion on these types of change in each of the above PRs as well as in the Zulip thread that goes with it. I'm sorry, but I only read the latest messages of each as there was a few hundreds of them, going over a few years of activity. Apologies if this proposal contains approaches that have been discussed and ruled out already.

[JoergAtGithub]

Just for completeness, #4489 was just a small part splitted of from this work: https://github.com/mixxxdj/mixxx/wiki/Measures-Downbeats-Bars-And-Phrases

[Swiftb0y]

haven't been able to look at the meat (beats.cpp) yet. In the meantime, can you take care of CI failures. The inline annotations produced make it hard to review (and they need to resolved anyways, so better sooner than later).

[Swiftb0y]

NIT: this is already done if its needed in the append afterwards. Is it feasible to do the reserve outside loop once?

[Swiftb0y]

NIT: lets DRY this a little.

Suggested change

	if (!m_beats.isEmpty()) {
	QPen pen(m_beatColor);
	pen.setWidthF(std::max(1.0, scaleFactor()));
	painter->setPen(pen);
	painter->drawLines(m_beats);
	}
	auto drawLines = [&](const auto& color, const auto& lines){
	if (!lines.isEmpty()) {
	QPen pen(color);
	pen.setWidthF(std::max(1.0, scaleFactor()));
	painter->setPen(pen);
	painter->drawLines(lines);
	}
	};
	drawLines(m_beatColor, m_beats);

[Swiftb0y]

I think this could be simplified if instead of going through each beat and then partitioning it into downbeats, marker beats and regular beats, you "just" replaced the m_beatShader by a mixxx::RGBShader and just supplied the correct color for each each vertex. Does that seem feasible?

[acolombier]

That's doable and sound like the way to go indeed.
This was just a quick hack to get something working to get some user testing first, before I make that better!

[acolombier]

Thanks for looking into this @Swiftb0y this quick!

With my latest commit, I should fix most of the CI issues + failing test, and also started cleaning up some of the mess. Before looking too much into the code, I'd been keen to hear whether or not we are okay with approach I've taken from a UX perspective, and perhaps also gets some user testing first. As I said in introduction, I know there was a lot of discussion on the matter and I'd like to make sure this doesn't end up abandoned as well.

[JoergAtGithub]

It crashes here, if I try to place a marker before the very first marker:

[acolombier]

With the last commit, I have introduce customizable beats per bar for each markers. To do so, I'm using a "double beat" in the beat map, meaning a beat with the same position that the previous one. This is used to infer the bar length.

Pro: no need for a proto change
Con: If you revert and try to load a track using a custom bar length, you will get an assert here. This will lead the grid being invalidated, same than if we were introducing a new proto version anyway.

@JoergAtGithub I tried many things but couldn't reproduce it. Could you please provide me with some steps and maybe a short recording? I'm probably missing a settings or something.

[JoergAtGithub]

This time I got antoher assert:
DEBUG ASSERT: "markerIt != markers.end()" in function class std::optional<class std::shared_ptr > __cdecl mixxx::Beats::tryRemoveMarker(class mixxx::audio::FramePos) const at D:\mixxx-main-worktree\src\track\beats.cpp:926

[JoergAtGithub]

And now I got the original again:

DEBUG ASSERT: "beatFraction > 0 && beatFraction < 1" in function bool __cdecl mixxx::Beats::findPrevNextBeats(class mixxx::audio::FramePos,class mixxx::audio::FramePos *,class mixxx::audio::FramePos *,bool) const at D:\mixxx-main-worktree\src\track\beats.cpp:460
Unhandled exception at 0x00007FFBFE405438 (Qt6Core.dll) in mixxx.exe: Fatal program exit requested.

[acolombier]

Right, my fault, DEBUG_ASSERTIONS_FATAL had gone OFF for some reason. I can reproduce both now.

The first one is related to the Fix the "last beat" in the description, the second one seem to be a wrong offset. I'll let you know when I get both sorted.

[JoergAtGithub]

Con: If you revert and try to load a track using a custom bar length, you will get an assert here. This will lead the grid being invalidated, same than if we were introducing a new proto version anyway.

The link "here" doesn't point to a particular line of code. But I wonder if it's possible to add some code to the 2.4.2 release that just prevents the grid invalidation. And than merge this PR with the new beatgrid feature into 2.6.

[acolombier]

Fixed

[acolombier]

@JoergAtGithub I believe fixed most of the asserts and added some tests to cover the new introduced use-cases. I would appreciate if you could give it another go 🙏

[acolombier]

I wonder if it's possible to add some code to the 2.4.2 release that just prevents the grid invalidation. And than merge this PR with the new beatgrid feature into 2.6.

It's worth to highlight the invalidation would only happen if one:

• Uses custom measure size in 2.6
• Reverts to 2.5

So while it feels quite an edge case, it may also feels fairly natural to need to regenerate the beatgrid to the user, so perhaps worth keeping as is.

[JoergAtGithub]

I tested again, and no asserts occured anymore!

It's worth to highlight the invalidation would only happen if one:

* Uses custom measure size in 2.6

* Reverts to 2.5

So while it feels quite an edge case, it may also feels fairly natural to need to regenerate the beatgrid to the user, so perhaps worth keeping as is.

The missing backward compatibility, was one of the main reasons, why the predecessor PRs got never merged. We need an commonly agreed strategy here.
Personally, I have no strong preferences here, as long as the final implementation is not compromised by the legacy.

[JoergAtGithub]

The markers in the waverforms are difficult to recognize. The visual representation in the original PR #2961 was more clear in my opinion: https://mixxx.zulipchat.com/#narrow/stream/109171-development/topic/beatgrid.20editing.20UI/near/206216183

[acolombier]

I tested again, and no asserts occured anymore!

Great to hear!

Personally, I have no strong preferences here, as long as the final implementation is not compromised by the legacy.

Make sense to me. What do you think of my proposition of backward compatibility, except if the multiple bar length feature is used?

The markers in the waverforms are difficult to recognize

Yeah it could be better I agree. The colour of the bar marker can be customised by the them. By default I wanted to have something not striking to much ( like the original red colour used before), but I guess we could find a middle ground.
Adding custom drawing (like the triangle in the previous version) might be too much and also not as straight forward to do as just a custom colour, than can easily be customised.

[acolombier]

Overall, are we happy with getting this PR in as is? Just wondering if I should start looking at making it production ready, and clean up the implementation/fix conflict or of there is still outstanding controversial aspects.

[Swiftb0y]

Well, the primary thing I'm confused about currently are the complications around lastBeatsPerBar, but I haven't read the corresponding code thoroughly enough yet.

[Swiftb0y]

One of the primary changes are that you changed the beatmarkers to store the beatlength instead of the total beatcount, right? Can you share some of the rationale behind that or link me to a previous discussion?

[acolombier]

the primary thing I'm confused about currently are the complications around lastBeatsPerBar

There is left over from the v1 solution, which will be removed/cleaned up as part of the making it production ready if I get consensus on this solution. This historically comes from the fact that the "last" marker state isn't stored as a BeatMarker object, but at the beat grid level.

Can you share some of the rationale behind that[..]?

Sure - the main reason for this change is, if you store the number of beats per marker, resizing marker (effectively splitting a marker in two, or merging two in one) will change the BPM.

Say you have a track at 128BPM, somewhere in the track, the beat grid shift by 1/3. If you set a new marker (=divide the single marker in two), you will end up with two different BPM (slightly less than 128BPM depending of the number of beats before than new marker, slightly more than 128 after)
Now, if you want to adjust the BPM of each of these marker, previously, the bpm would drastically change depending of the marker size. If the marker was 10 beats (at 128 BPM still), increasing the BPM of the active marker (adding a beat) will lead to the new BPM of 128BPM + 10%. Now for a marker of 2 beats, that would change the BPM by 50%.
Back to the usecase of wanting to shift the grid by a fraction of BPM, the only way to make it work was to set two markers: one at the beat, setting the BPM to 384 (128 / 1 / 3), then a second marker to "restore" the grid to 128 BPM. You can imagine what happen if you play the track in autosync - the other deck sync with 384 BPM for one beat (or one third beat of 128BPM)

(Note that the same usecase can be applied for tracks with a speed up or slow down, using variable BPM)

IMHO, using beat length is the only way to cover every usecase and maker the user experience of editing the beat grid smoother.

[Swiftb0y]

There is left over from the v1 solution, which will be removed/cleaned up as part of the making it production ready if I get consensus on this solution.

That would be great. yes.

Sure - the main reason for this change is, if you store the number of beats per marker, resizing marker (effectively splitting a marker in two, or merging two in one) will change the BPM.

Thank you, yes. That makes total sense.

I'm very pleased with the direction this is going in, but I'm not the person that was reviewing the previous PRs, so I'd really like to know what @daschuer thinks.

[daschuer]

I am very happy to see progress here. Thank you for picking this ball up. I have an unfinished branch here rotting aroud since some time.

The biggest issue is the pure size if this PR and the number of open topics we have discussed along these lines with valid but different concerns.

My current prio is to finish the stem topic so I am afraid it will take a while for looking in all details.

What we should do now is confirm the current and future data structures. In the meantime it would be nice to split out all uncritical and agreed changes so that we do not suffer from stucking again.

[acolombier]

The biggest issue is the pure size if this PR and the number of open topics we have discussed along these lines with valid but different concerns.

I could try to split it if you think there are part that can go already and don't have any concerns.

What we should do now is confirm the current and future data structures.

I managed not to change the existing data structure and built the improvement on top of BeatMap; BeatMarker were generated after parsing each beats contained in the map, my only change was to change this function but keep the serialisation as it is. I have already added unit test to ensure that new parser/generator is backward compatible with the previous one and build BeatMap in the exact same way.

It would be useful if everyone with concern could give me a quick sum up of where the concerns are, so I can see if this is something my solution can address, and otherwise close that PR if it is going to a dead end.

[ywwg]

would it be possible to split the rendering changes from the grid editing changes?

The first pass could be turned off by default and assume that the cue point is a downbeat (not always true...), but then we could make sure then rendering is right before we move on to the editing UX

[acolombier]

would it be possible to split the rendering changes from the grid editing changes?

I'm going to try that next. The plan will be to break that in two PRs, one with backend changes only without edition capability, and later a PR with UI and edit feature. How does that sound?

[acolombier]

I broke down the feature in two PR now (follow up here) and managed to reduce slighly the size.


Hopefully this is now easier to review and merge 🤞