When enabling two entityTypes, is one surface expected to provide 2 hit test results?

[thetuvix]

When a developer enables both the "plane" and "mesh" entityType, what should they expect when hit-testing against a relatively flat surface?

• Would they get two XRHitTestResult instances for each surface, one for the surface's plane geometry and one for the same surface's mesh geometry?
  • If so, is it random whether the site sees the plane or the mesh first in the list, based on whether the mesh geometry happens to dip above or below the idealized plane along that particular ray?
  • Note that there is no output entityType on an XRHitTestResult, and so an app can only pick the first result's position and normal. If there is a material difference in normal stability for "plane" vs. "mesh" hit test results, the placed object may judder unexpectedly as the user scans the ray across areas where the mesh dips above and below the plane.
• Would they get one XRHitTestResult instance for each surface, with the UA hiding the mesh collision at points where the same surface has a higher-quality "plane" normal to offer?
  • In this approach, the site's request for "mesh" entities is primary an opt-in to hits against additional curved surfaces that don't have planes.
  • If UAs go this way, is there a difference between requesting ["mesh"] entities vs. requesting ["mesh", "plane"] entities? Does a single entityType member or a planesOnly bool (as discussed in Can "point" just be subsumed into "mesh"? #66) make more sense then?

If different UAs diverge in the path they choose here, it will likely cause non-conformance across devices for sites that request ["mesh", "plane"] hits. We should be more prescriptive in the spec on how many hits at most are expected per real-world surface.

[bialpio]

Ideally, we should probably eventually expose the entity type on the hit test result (and the entity itself, see #23) and let the application decide what they want to do. Currently, I'd expect the page to get two instances for each surface (assuming that the native hit test returns both the result for plane and for mesh). The order is not random - the result that is closer to the ray origin along the ray should be returned first (so the judder will only potentially happen due to the hills above the plane if the app uses the first result).

I'd say that the app should only issue a hit test with ["mesh", "plane"] if it doesn't care about the potential judder. If judder is a concern, I think the acceptable approach would be to create 2 hit test sources, one using planes and the other using meshes & behave as if the entityType is present on the results - what do you think?

[thetuvix]

Agreed that the judder is not actually random - however, it's precisely those hills of mesh tipping above the plane that I was worried about.

If we did either expose the entity type on the result or encourage apps to make two sources, how do you see apps avoiding this problem? Would they establish a quantitative heuristic, preferring plane hits over mesh hits, so long as the plane hit is not more than some number of cm behind the mesh hit? (which could signify that it's a different surface)

On HoloLens, the native Scene Understanding SDK that will power WebXR hit tests has SceneMesh and SceneQuad entities parented to a root SceneObject entity that represents the physical surface. A UA there could easily exclude a hit against a SceneMesh if the ray is also intersecting a SceneQuad for the same SceneObject.

I want to ensure that the hit-test module spec allows for UAs to explicitly provide just one hit per surface when two entity types are selected, providing additional value to applications beyond what they could accomplish themselves with two separate hit test sources.

[bialpio]

If we did either expose the entity type on the result or encourage apps to make two sources, how do you see apps avoiding this problem?

I was thinking about the apps relying on the "plane" results first, and if they are unavailable, the app could then fall back to leverage "mesh" results, but that might not be something that the app always wants - the heuristic you are describing would probably be a better approach in the general case.

I want to ensure that the hit-test module spec allows for UAs to explicitly provide just one hit per surface when two entity types are selected, providing additional value to applications beyond what they could accomplish themselves with two separate hit test sources.

That makes sense. The issue will be there for platforms that support both planes and meshes and where the same physical object might have both plane and mesh representation internally. Adding the second approach that you described to the specification would make the behavior well-defined, but then, if/when we decide to expose the entity being hit, we will no longer be able to change this behavior in a non-breaking way - we should make sure that this is something we're fine with.

Alternative approach would be to add something akin to the SceneObject to XRHitTestResult - this additional property could then be used purely to check whether different hit test results come from the same physical object.

[mounirlamouri]

@thetuvix what would be the ideal behaviour in your opinion? How does Hololens generate the planes? Are they generated from the meshes or are they both generated individually? Is there an always better answer to give when both meshes and planes are requested by the hit test?

[blairmacintyre]

I think we should look over at #66 ... I don't think we should be exposing anything with detailed semantics like planes and points, so I don't think this issue needs to be resolved at all.