Suggestion: Allow correction of left-right-balance data

[remifreiwald]

Recently, a colleague of mine and I were discussing the data Bipolar is generating regarding left-right-balance.

The numbers generated for left-right-balance should lie between 0 and 100. But there are values outside of this range. Probably not the fault of Bipolar; we think the data from V800 and its sensors is already wrong.

Our suggestion: Values outside of this range should be corrected to 50.

Further thoughts:

• We checked a file in Polar ProTrainer 5 to find out, if this is necessary. We found out that extreme values (over 100) are influencing the average, as long as there is power/force at the same time.
  In other words: When riding down a hill, the power is 0 Watt and PPT5 ignores those outlier values. But in every other case PPT5 is not correcting those outliers on its own.
• Other training software (or Excel users) may even use all balance values regardless of power, thus impairing the average even more.
• During long downhill segments, people tend to stand on one pedal. As written above, PPT5 ignores those parts in its calculations (because power is 0). But in the attached screenshot you will see, that those segments are still displayed and can be confusing or at least annoying.

Since this would alter the data, it could be practical to have a checkbox which allows to turn this feature on/off.

[remifreiwald]

Of course, one could argue that a value of 50 is just as wrong as a value of 130, if it should have been 90 at this point... Both are 40 points from the desired value...

Maybe it would be better to correct negative outliers to 0 and positive outliers to 100?

But that would not improve the teal-colored valleys in the screenshot.

[pcolby]

Thanks @remifreiwald, some good suggestions there. Bipolar internally does a few things to determine LR balance, and its quite possible that the code is not quite right. I'll definitely take a look.

Just to confirm, are we talking HRM? (I'm not seeing LR balance values in the GPX/TCX code, but I could just be missing it)

Thanks.

[remifreiwald]

Hey, yes, we only checked HRM files.

[pcolby]

Hi @remifreiwald, I've made a few tweaks to the way LR balance is calculated already... just to make the calculation more correct when data samples are missing (ie the sensor was temporarily offline). But this doesn't do anything (yet) to validate / correct for erroneous extreme values.

I'd be very interested in seeing how the data looks after the most recent code changes. Can you test using the current master? (takes some advanced knowledge, so not surprised if you can't) Otherwise, can you provide some test data set that exhibits this issue (maybe I already have some?) and explain to me how to get the view you included above so I can play with it locally?

Thanks.

[remifreiwald]

Hi! I will send you new data which contains different balancing values.

In the meantime you can work with the last data set which included power and laps. There are no extreme values outside of the normal range (0 to 100), but there are still segments with either 0 or 100.

To get the same view in Polar ProTrainer 5:

• Double-click on any day in the calendar
• Top-right corner: "Add New Exercise"
• Bottom-right corner: "Attach"
• Choose an HRM file
• Then double-click on the small preview of the colored graphs (bottom-right)
• Open the "Curves" menu (see attached screenshot)
• Now you can choose which curves you would like to see

[pcolby]

Unfortunately I don't access to a Window machine at the moment, so can't run PPT5 currently.

I you have a moment, you could have quick look at the test data pre/post this change by comparing, for example training-sessions-267510822.hrm@1bd0eda to
training-sessions-267510822.hrm@a6ef773.

Thanks.

[remifreiwald]

Unfortunately the exercise contains only very "limited" balance data. The values are either 50 or 0.

In the screenshots you can see, that all 0 values were converted to 50.
And we can say, that PT5 is able to open the files.
So, at least that works.

My colleague will send me a suitable exercise file (probably monday). And I will try to compile the code myself. Just never worked with Qt before, only Java and PHP stuff...

PRE

POST

[remifreiwald]

Hey, I got new data, which contains more realistic balance values. The only thing missing is values outside of the range 0-100.

Attached you will find the data and the CSV/TCX exports from Flow.

I couldn't compile Bipolar, yet, because the Qt installation fails due to a time-out, have to check this again...

balance-testdata.zip

[pcolby]

Thanks @remifreiwald, here's a copy the resulting HRM file before and after the changes I made above.

training-sessions-1988256040.zip

Looks like its all good now. But as you say, there's no extreme values outside the 0-100 range anyway. I think I'll keep this open until we get some more data that shows the problem (happy to fix it, just want to fix as close to the source of the bad data as possible 😄 )

Thanks again!

[remifreiwald]

I checked the files in ProTrainer 5 and found out they actually contain outliers:

No. 1
Time: 0:02:32
Power: 29 Watts
Balance: L255 & -155R

No. 2
Time: 0:02:37
Power: 0 Watts
Balance: L255 & -155R

No. 3
Time: 0:02:55
Power: 0 Watts
Balance: 176 & -76R

I don't know if this is coming from the raw V800 data or if this is interpretated by Bipolar, but I guess, if we can fix this, everything should be fine :)

[pcolby]

Ah, yep, can see the outliers in the data now. Will take a look :)

[pcolby]

BTW, Bipolar already enforces a technical limit (imposed by the HRM format itself) for the balance to be between 0 and 255 (inclusive):

stream << '\t' << qMax(qMin(leftBalance, 255), 0);

Which is why you don't see any negatives, but I've updated the code (just locally for now) to warn on negatives as well (was already warning on >100%), so now I get:

QWARN  : TestTrainingSession::toHRM(training-sessions-1988256040) leftBalance of 355 % is outside the range 0..100
QWARN  : TestTrainingSession::toHRM(training-sessions-1988256040) leftBalance of 335 % is outside the range 0..100
QWARN  : TestTrainingSession::toHRM(training-sessions-1988256040) leftBalance of -44 % is outside the range 0..100
QWARN  : TestTrainingSession::toHRM(training-sessions-1988256040) leftBalance of -492 % is outside the range 0..100
QWARN  : TestTrainingSession::toHRM(training-sessions-1988256040) leftBalance of -492 % is outside the range 0..100
QWARN  : TestTrainingSession::toHRM(training-sessions-1988256040) leftBalance of -176 % is outside the range 0..100
QWARN  : TestTrainingSession::toHRM(training-sessions-1988256040) leftBalance of 176 % is outside the range 0..100
QWARN  : TestTrainingSession::toHRM(training-sessions-1988256040) leftBalance of -387 % is outside the range 0..100
QWARN  : TestTrainingSession::toHRM(training-sessions-1988256040) leftBalance of -226 % is outside the range 0..100
QWARN  : TestTrainingSession::toHRM(training-sessions-1988256040) leftBalance of -226 % is outside the range 0..100

There you can see the three extreme positives you noted in your comment above, but there's actually ~7 negatives that were already being silently capped too.

Anyway, will look into the source of the errors next, and code around accordingly.

[pcolby]

Hi @remifreiwald, just checking, this isn't a fixie is it? (not expect it is). Just asking 'cause there's one sample where the left and right power samples are both negative, which is technically not possible (well, not this magnitude) on a coaster system. ie proves the data is wrong (at least for that one sample).

[pcolby]

Almost done fixing this one, but just a quick note to point out that @remifreiwald's most recent data set includes a enough negativity in one sample, that the total power for that sample in negative, which results in the generated TCX files being invalid, since Garmin ActivityExtesion schema requires power to be a positive integer.

So I'll fix that up as part of this change too :)

[pcolby]

Ok @remifreiwald, I've updated the code to filter out negative power samples, and restrict the LR balance to between 0 and 100 (c554552). This is in addition to the earlier change to use LR balance 50 when total power is 0, and both L and R match.

Have a look at the resulting HRM file, and let me know what you think: training-sessions-1988256040.hrm

Thanks.

[remifreiwald]

Hi @pcolby! Sorry, for the late answer...

The test gear used wasn't a fixie. It was a single gear, but freewheeling was possible.
We collected the data on a Wahoo Kickr (https://eu.wahoofitness.com/devices/bike-trainers) and took the power from Polar Kéo Power Pedals (https://support.polar.com/en/support/keo_power_bluetooth_smart).

My colleague, who was riding the bike, thinks it is possible, that he may have pulled on both pedals right at the moment of quickly standing up, to sprint for a couple of seconds.

I opened the HRM file in ProTrainer 5 and everything looks great!

I found only one point where Power is 0 and LR balance is: L100 - 0R
It's at 0:02:53, but this is a minor thing and ProTrainer 5 ignores it anyway.

I think it's much better now!

[pcolby]

Thanks @remifreiwald :)

I found only one point where Power is 0 and LR balance is: L100 - 0R

Good pickup. I can see the issue - the "equal to use 50" logic was comparing the pre-force-positive values... easy fix, coming up :)

[pcolby]

I just pulled the trigger on the v0.5.7 release... it'll take several hours before Travis CI releases the dmg and around 24 hours (!!) before AppVeyor releases the exe to the GitHub release page (if all goes well).

Edit: had to cancel those builds (it was going to try to release a non-existent OSX build).

New builds are now:

• OSX
• Win

[pcolby]

Builds finished. Latest installers available at https://github.com/pcolby/bipolar/releases/tag/v0.5.7 :)