Draft for a course model extractor

[Tmcg2]

Should be able to extract arbitrary vtx lists with a little
adjustment

[Tmcg2]

This PR is very work-in-progress, leaving it up as a draft for commentary purposes

[MegaMech]

Looks good to me!

[Tmcg2]

Updated to also fetch face data from the base ROM. So now the course.obj file has both vertices and faces in it.

It should be possible to get texture coordinates in there too but that'll take some extra processing. Need to convert the given integers into floats in the 0.0 to 1.0 range. Somehow.

Vertex normals need extra processing too since the flags are bit stuffed into them. Although I'm not entirely sure we need to export them since they are calculatable after the fact (at least that's what one source says)

[iProgramMC]

Need to convert the given integers into floats in the 0.0 to 1.0 range. Somehow

The N64's UV coordinate format uses 16-bit fixed point from -1024 to +1023. You need 11 bits for that, so 5 are left over, thus simply dividing each texture coordinate by 32, and then by the relevant texture size minus one (width for U, height for V) should do the trick. (Minus 1 because a U coordinate of 1024 already shows the next pixel)

Converting them back without introducing UV artifacting is a little tricky unless you do not care about generating vertex data that is accurate to the source material. My SM64 level converter tool takes the middle point of the UV triangle, and then moves all the points such that fmod(centerPointCoord, 1) == newCenterPointCoord.

[Tmcg2]

That works for regular Vtx structs, my confusion has to do with converting the S/T values of the mk64_Vtx(_n) struct.
For example:
Mario Raceway's course vertices list starts with:

mk64_Vtx d_course_mario_raceway_vertex[] = {
 {{ 2583,      31,    -586}, {  8985,  1023}, {0xfc, 0xfc, 0xfc, 0x00}},
...

These non-standard vertices eventually get straight copied (ignoring how flags is handled) into regular Vtx structs in func_802A86A8 found in memory.c.
So obviously I try the expected method. Take 8985, divide by 32 (to convert from S10.5 to float), then further divide by 32 (the texture's width) and you get 8.7744...
So clearly I'm missing something.
Its unclear from my reading of the N64 documentation what values the hardware itself is expecting in S/T. Clearly it can handle things outside of the range an OBJ file desires. Or maybe there's something extra going on here, maybe there's some extra math that's required that I can't figure out.

[Tmcg2]

I have found that leaving the texture coordinates as they are after the expected divide by 32, divide by texture width/height, even if that leaves them outside of the 0 to 1 range, is fine. I don't know if Blender is doing something in the background or if the OBJ file spec is lying.

[Tmcg2]

The texture list stuff for each course is going to be a pain in the ass. I think I know what we need to do to get that stuff properly linked though. I'll work on that sometime soon(tm).

[iProgramMC]

I don't know if Blender is doing something in the background or if the OBJ file spec is lying.

No. Actually it's a perfectly fine texture coordinate. In order for the texture to repeat several times you must obviously go beyond the [0, 1) bounds.

[Tmcg2]

I'm done messing with this for the moment. Its in a mostly good state, although some courses have some very noticeable flaws when the OBJ file is imported into Blender. The short version is that several courses make use of the texture clamping feature which Blender does not support.

This is most noticeable for Moo Moo Farm. where the starting/finish line texture is clamped to create both the race course and the grassy out-of-bounds area. A similar issue is observed in Rainbow Road and Mario Raceway road (albeit less dramatically) and the ocean in Koopa Troopa Beach (not 100% what's going on there really).

In some cases a tunnel (like the pipe in Mario Raceway) has its entrance/exit covered with a texture which obviously doesn't happen in practice. I'm not sure what's going on there.

OBJ files don't support vertex colors and Blender doesn't support texture mapping and vertex coloring simultaneously so some courses aren't fully detailed. Things like the Mario/Luigi hats (in their respective courses) and the rainbow ring in Rainbow Road are left uncolored since they're supposed to handled by vertex colors.

Finally, any textures that are meant to be mirrored aren't designated as such (I don't think OBJ files can handle it) so some courses like Royal Raceway are extremely close to be correct, but the road texture will be slightly off due to it repeating/wrapping/tiling instead of mirroring.

In the future exporting the courses in either COLLADA or glTF2 would likely be best, but those formats are far more complicated than OBJ (simplicity is its only asset). glTF2 appears to support repeating/mirroring/clamping in the U and V directions independently, which is nice since the N64 has that capability too. I have had trouble finding good examples of how to use those formats however and I really don't feel like dragging this PR out any longer.

[MegaMech]

Is this intentional? I'm fine with it. But it's part of a utils.c just wanna check this won't mess anything up.

[Tmcg2]

Yeah, it seems like the basename definition here is conflicting with something in C++'s STL (I think?). It goes unused in all our tools so commenting it out shouldn't hurt anything.

[MegaMech]

Please add how to use the tool to your PR desc.

Or does it just work when you build the repo?

[Tmcg2]

Usage:
./tools/vtx_extractor /path/to/baserom.us.z64
How to build:
make -C tools -f vtx_extract.mk

[MegaMech]

If this doesn't build when you run make can we instead integrate this into the normal tools makefile.

Ie. I think the program should be built by default. But only ran if the user desires to do so.

Honestly, it's probably fine if it runs automatically. I'll leave that decision up to you.