Not loading tiles intersecting the camera view frustum + tile-loading performance improvements

[jo-chemla]

Describe the bug

We've been experimenting for a few weeks with a 3DTilesRendererJS-based r3f + theatrejs app, and are in the process of trying to optimize the tiles loading and performance. The whole 3d-scan (multiple meshes for the whole San Pietro basilica in Vaticano, around 15B triangles total) mesh has been simplified to 400M triangles with usually a few (4-16) 16k textures. RealityCapture is used to output an OGC 3d-tiles 1.0 tileset with b3dm tiles, then we use 3D-tiles-tools to upgrade it to version 1.1 - and are considering optimizing via batching gltf-transform or gltf-pipeline for draco geometry compression and ktx2 texture compression.

After noticing some performance drop, we began investigating improving performance. There have been cases where we did merge multiple (eg 8, with texturing more controlled) tilesets into one, with no hierarchy at the root level - just references to each individual tileset - which impacted pretty heavily performance vs loading a single tileset.

In order to have the most vanilla measure of performance, we started from a blank r3f app with a 3DTilesRendererJS component - no more theatrejs - and only a single tileset loaded at once. When searching for related mentions of performance optimization you implemented throughout the history of this repo, I stumbled around a few very interesting cases like #632 #480 #490 #678 and I think one which is most impacting to us is #376.

Plus there are a few closed PRs/issues which suggest the below (old or more recent) properties on the renderer or cache can be tweaked to have better performance:

• TilesRenderer: errorTarget * errorThreshold
• PriorityQueue: maxJobs
• LRUCache: maxSize, minSize, unloadPercent, maxBytesSize and minBytesSize

There are cases where the user moves the camera, and the renderer seems it want to handle queued requests, without reordering them, and tiles in the background gets processed before some closer to the camera - which might be tweakable by one of the above parameters?

At some point, I noticed something that do looks like a bug, only noticeable on low-resolution tiles being loaded. It seems that when tiles are completely within viewport frustum, tiles are loaded - and eventually refined - and the opposite when completely outside frustum. However, when tiles bbox is at the intersection of the frustum, it seems that they are not loaded or discarded - although they should be loaded. Or is it connected to this load-sibling question

To Reproduce

Minimal example on the giza plateau:
Steps to reproduce the behavior:

1. Go to https://3d.iconem.com/partners/interactifs/tilesrenderer/ (skip password which tries to load favicon)
2. Click on navigate around the kufu pyramid
3. See some low-level tiles not loading although they should - issue is not really visible when higher resolution tiles are loaded
4. PS we do have other tilesets that cannot be open where this issue is more visible, where tiles do not always load.

Live example

• https://3d.iconem.com/partners/interactifs/tilesrenderer/

Expected behavior

All tiles that do intersect with the frustum should be loaded and not discarded. Or is this because the RealityCapture tileset has wrong bbox defined per tile at low resolution?

Screenshots

Platform:

• Device: Desktop
• OS: Windows
• Browser: Firefox
• Library version: 0.3.37
• Three.js version: [r???]

[gkjohnson]

Hi Jo - I think there are too many separate topics here for one issue which makes it difficult to have a focused conversation. Can you split this up into at least a couple issues focused on performance vs the possible missing tile bug you're seeing? And / or respond in some of the dedicated issues that are covering topics you're interested in?

Also for any repro cases please provide a live, editable example showing the issue using something like JS Fiddle and vanilla js with only the required packages, three.js and 3d tiles renderer. Otherwise it's hard to glean anything from a bundled demo I can't poke around or modify.

[gkjohnson]

Closing for now until there's an editable example I can take a look at to understand what's going on