3D graphics and glTF format support

[josuah]

In the same way wepb and webm were selected as standard format, by GOOG, GLTF was announced as file format of choice for 3D mesh graphics.

Some users started discussing about AR and Unity integration. Frame is not having any GPU to do rendering in real-time nor having enough throughput for streaming a 3D rendering, and the IMU is on the device itself. This limits how much AR can be attempted on that device.

However, the vector graphics library can be leveraged to render wireframes or simple low-poly flat-shading based models.
Trigonometry would turn coordinates of triangles into faces, compute normals, compute z-index and intersection, select a color according to the face orientation (flat shading without texture), and cut the outlines of each shape to draw as a geometric shape.

This is a lot of math work, but no pixel or image manipulation: only trigonometry on coordinates, with the last step: filling the triangles, done by the FPGA with the vector library.

A wireframe-based model can be a lot less work to render and have more chance to be rendered by software in real-time to use the IMU to realign the model frequently.

Whether Frame can keep-up the pace remains to be proven!

https://discord.com/channels/963222352534048818/1240218103603003442
https://discord.com/channels/963222352534048818/984966420603482182/1240681494335459329

[josuah]

https://discord.com/channels/963222352534048818/1242455966713253979/1244287731383926925

[siliconwitch]

This would be really cool! Hopefully once the vector engine is in, someone can figure out how to extend it to support this