[tactile_merger] Suggestion to publish also negative forces

[guihomework]

Some customers are interested in having the contact_states be published also when the tactile_states messages contains negative values, potentially producing a negative averaged force for a "group" of taxel.
reasons invoked are :

• seeing the vector of force displayed in rviz in all case positive and negative is a good way to find problems.
• negative forces can be measured by some tactile elements on sticky material, so should be displayed.

Currently it seems the design only accepts values higher than epsilon and positive https://github.com/ubi-agni/tactile_toolbox/blob/melodic-devel/tactile_merger/src/taxel_group.cpp#L111.

one open question is :

• should the feature of displaying negative forces be optional ? if yes, should it be default behaviour (non backward compatible) ?

[guihomework]

a PR is almost ready but will be provided when the decision is made on the optional character of the feature

[rhaschke]

I'm fine to have this behaviour the default and not optional.

[guihomework]

I'm fine to have this behaviour the default and not optional.

personally I am not decided. With negative force allowed in a test run, a lot of data was produced, and the arrows displayed in the rviz_tactile_plugin were totally intermixed, and difficult to see they changed from negative to positive.

An improvement of the display plugin, for instance negative forces in a different color, could solve the consequences of this PR.
But in some cases, it might be desirable to publish only contacts with positive forces, or maybe even above a user selected threshold (other than epsilon) to filter noisy data.

[rhaschke]

Publishing "negative" forces is just fine - actually (force) vectors don't have a sign.
The problem arises when computing the contact location (and normals) with averaging from negative weight contributions:

tactile_toolbox/tactile_merger/src/taxel_group.cpp

Line 108 in d8bcbff

pos += w * it->position;

In this case, you would move the contact into the wrong (opposite) direction. As an example, just consider two taxels being contacted, one with positive, one with negative normal force.

Thus, |weight| should be considered for position and normal averaging.

I like the idea of having a configurable threshold to skip contacts with too small sum |w|.

[guihomework]

To me, the arrow displayed by the rviz plugin has a direction, and to me, the averaged normal should always point outwards. So what was called "negative force" was an arrow along the normal but pointing inwards, making the arrow with a "negative" sum when multiplied by the normal vector here https://github.com/ubi-agni/tactile_toolbox/blob/melodic-devel/tactile_merger/src/taxel_group.cpp#L125 isn't it ?

It is a very good remark that weighted centers and norms will be wrong if some weights are negative, even if the sum is positive. So that is a bug on the current code, even without the suggested "allow negative force" feature.

the suggestion was only to allow sum to be negative.
Should one fix the position and normal averaging in a separate commit (as it is unrelated to allowing "negative sum") ?

[rhaschke]

A vector has a direction, but not a sign. Of course, here (as you pointed out) we are referring to a vector pointing inwards vs outwards w.r.t. a surface / normal direction. Hence, w.r.t. to the normal direction we can compute the sign as sgn(force * normal).

It is a very good remark that weighted centers and norms will be wrong if some weights are negative, even if the sum is positive. So that is a bug on the current code, even without the suggested "allow negative force" feature.

Yes.

Should one fix the position and normal averaging in a separate commit (as it is unrelated to allowing "negative sum") ?

No, this should be fixed here, because the sum as computed now (and before) has no physical meaning. Do you want to tackle this? Or may I do, because I already have a very clear picture in my mind, how the code should be changed.
You are very welcome to contribute the configurable threshold.

[guihomework]

You can tackle this issue, I have no chance to do what you have in your head, and then the threshold is considered in the same part, so not worth to split.

[guihomework]

Thinking a little (actually for a while) more, I have a problem with the existence of an "average point of contact and average normal" when in the forces applied to the group there are negative values.
If the amplitudes on the tactile_state of taxels of the group are all negative or all positive, I conceive the existence of meaningful average point/normal of "pushing" and "pulling" . before this PR we did pushing on our tactile tips (MID and glove), and the computation worked well and the display was realistic, nicely "interpolating" the true contact center of a group.

With one force pushing and one force pulling, with the same amplitude but one in opposite direction of the normal, with using |weight| in the computation of the average normal and center, the sum of the force would not be along the average normal at all. The average point is also not really meaningful any more to me.
I tried to imagine using sticky object on half of its surface, and then apply a rolling motion on the fingertip. the stick part would pull the non sticky part would press. What is the average contact center ? is it really in between the 2 taxel center as you suggest ?

I did some drawings for the force, the normal and the center

I would validate :
force = sum ( w * normals of individual taxels)
torque = sum (pos of individual taxels cross ( normal of individual taxels * w))

not sure what to do with normal and pos. they look wrong in both solutions when one taxel value is negative. Maybe the average pos and normal are not computed if one taxel value of a group is negative ?

I feel it is wrong to put the normal and position in an artificial position.

[rhaschke]

Position and normal move in weighted fashion as well with #11. Not sure why you kept them in the (unweighted) center of both contacts. It looks more like this (top row is your's, bottom row is corrected):

[guihomework]

I agree I made a copy paste mistake, indeed in #11 the center and normal are weighted, so move according to positive weights (upper line, right 2 drawings i initially transmitted had wrong center, and should look like what you draw right bottom.

but on the left column of your drawing, I don't agree that the more Force2 is negative the more the point goes to the right of the image, it gets closer to Force2 center. In fact the left most column is wrong too, with same weight one positive, one negative, the green point is infinite away, due to sum on the denominator being close to zero.
the normal was not divided by the "potentially close to zero" sum but normalized, hence I think the normals corresponding to the existing code were correct on the left column of my initial drawing

This should be the fixed version

this is why weighted average should not be computed for partially negative weights. For this reason you want to use absolute weighting, BUT if you look at the right column bottom case (of the correct drawing), isn't it wrong to have an average point and normal at all, they make no sense physically speaking. they are not even a kind of instantaneous rotation center of some sort. I know IRC are for velocities, but apply those forces F1 and F2 to a free floating object you will get a certain movement, somehow in the direction of the red arrow + a rotation due to torque, and maybe one could consider there exist a single point of application with a force and a torque that would produce the same motion, but I doubt it is the green point as of the right column. In dynamics mass is involved, and mass is always positive, while force can be pulling or pushing forces, here we artificially invent a mean point to just represent a average of pushing and pulling forces.

maybe we should calculate the sum of "negative" forces (and their weighted average), and sum of "positive" contributing forces (and their weighted average), and then draw them both , each at their meaningful point of application ?

[rhaschke]

I will only reply to the, in my view, important part of your comment, namely that you don't see a physical meaning of the averaged contact point and normal in case of positive and negative force contributions.
Formally, I agree, only force and torque have a physical meaning. Nevertheless, the computed contact point provides a coarse estimate of the contact position and it smoothly changes - as expected - within the convex shape of all individual taxel positions.
The normal direction is just a coarse estimate of the normal at this position.
This status (of coarse approximations) doesn't depend on the sign of the force contributions. These entities don't have a physical meaning for only positive or negative forces as well.

[guihomework]

the average center and normals have a physical meaning to me in case of only positive and negative forces as it is an interpolation of the center of application of pressure of a larger contact surface. We experienced it it looks fine. If we use the example I said with a cube, half sticky half not and rotate it at the around a central point between 2 taxels, I would expect the the average contact point to be split in 2. One half where I press on one taxel and one half where I pull on the second taxel.
An mid point is not a good interpolation anymore.

but our users should decide what they need. I will trigger them again to respond here

[GereonBuescher]

For the tactile sensors, which can only record normal forces (pushing and pulling), the estimated Force Point of Attack should stay within the covered area. At least for my point of view this restriction would reduce a possible error of fals estimated Force Attack Point positions. For sensors with shear forces measurement capability, the model would be changed depending on the speceivic mechanical design. For the standard case we have currently (normal force), the sub taxel positioning is really good working. For the case of positive and negative directions, it would make sense for me, to move the attac point more to the positive force, independen of with vector is longer. This, because the attac point should show the most possible position for the real contact. In case of a siticky rolling ball, this would be more in direction of the positive force.

[GereonBuescher]

Having both negative and positive directions next to each other could lead ot a vector sum of zero, but result in a momentum, wich might be insteresting for some applications. Nevertheless, we should alwas publish the state :)

[guihomework]

For the case of positive and negative directions, it would make sense for me, to move the attack point more to the positive force, independent of which vector is longer. This, because the attack point should show the most possible position for the real contact. In case of a sticky rolling ball, this would be more in direction of the positive force.

I tend to think this too. So maybe we should just consider positive values for the position and normal averaging (and not the |weight|), but keep the summing of all forces and torques with the weight (not with |weight| )?

[rhaschke]

The average center and normals have a physical meaning to me in case of only positive and negative forces as it is an interpolation of the center of application of pressure of a larger contact surface. We experienced it it looks fine.

If you accept this (coarse) physical meaning, you can need to accept the proposed solution in #11 as well, because it behaves exactly the same. Try it. Having two force locations doesn't make sense to me.

For the case of positive and negative forces, it would make sense for me, to move the attac point more to the positive force, independent of which vector is longer.

Based on which weighting this should be done?

So maybe we should just consider positive values for the position and normal averaging?

If there is a negative force applied (only), why shouldn't it contribute to the contact position?
You both try to cover some special cases, but there is no generic rule to apply.

The proposed solution computes the contact location based on a weighted averaging of the individual contact points. Using positive-only weights (that sum up to 1) is mandatory for this because otherwise we wouldn't compute a convex sum of the extremals.

[guihomework]

I will play the bag files we have this afternoon, and see how the solution of the PR shows up. But I think it is artificial, and my point of view should be weighted less than the real users.

Let me suggest a last idea, because you don't like 2 average points of "contact.
negative values "only" is fine to compute the average center.

if positive and negative, the contribution of positive and of negative could be computed separately, and the one that is the highest in absolute decides for the average point. If one is more pulling than pushing, the "average" center of the pulling is the one selected, and if one is more pushing, the average center of the pushing is considered.
I think this should result in a more intuitive center, and also no issues with the denominator (no substraction of positive negative weights)