WIP: Metal backend

[SamoZ256]

This PR implements a Metal backend for macOS. It should bring better performance compared to Vulkan, as well as fix some issues that wouldn't be possible with Vulkan due to MoltenVK limitations. I haven't tested many games, but those that I tried worked pretty well.

TODO:

• sRGB
• fix low resolution
• proper present
• fix black textures in sm3dw
• buffer copies in a vertex shader
• pipeline cache with binary archives
• geometry shaders
• GamePad screen in a separate window
• fix texture readback
• output shader
• ImGui
• Intel Macs support
• proper surface copy
• extend the shader hash
• triangle fans
• 1D shadow sampler emulation
• instancing with mesh shaders
• occlusion queries
• async shader compilation
• load pipeline binaries at startup
• async pipeline compilation
• accurate occlusion queries
• fix CI
• synchronize resource access manually
• binary archives

Screenshots:

New Super Mario Bros U

Captain Toad Treasure Tracker

Super Mario 3D World

Mario Kart 8

The Legend of Zelda: Breath of the Wild

Super Mario Maker

[deReeperJosh]

I have had some issues testing with the Metal backend when launching into Lego Dimensions - the first time I launch the game with Vulkan it progresses fine (albeit with half the screen cut off), but then when I close and try to launch with the metal backend, it crashes and hangs halfway through the first menu/cutscene screens (as well as seemingly duplicating the menus to infinity in the top left of the screens). Then when I close and re-open and switch back to the Vulkan backend, it crashes/hangs at a similar stage during the first menu/cutscenes. Clearing shader caches and re-launching with Vulkan works sporadically, but most of the time I have to completely remove the Cemu folder in Library/Application Support in order to be able to launch with Vulkan again and not crash/hang. The Metal backend seems to always crash, no matter whether shaders are cleared or Library/Application Support/Cemu is deleted. Here are some videos of what I mean:

first launch with vulkan

then launch with metal

launch again with vulkan

Would love to help debug this as I normally do my dev work on Mac, and it would be excellent if I could test out my PR (#1371) rebased on top of this branch, rather than needing to do my testing on my average Windows laptop

Edit: After enabling metal logging, I got a lot of "not implemented", and "Screenshots are not yet supported on Metal" so I guess I will need to stick with windows for the interim

[SamoZ256]

I have had some issues testing with the Metal backend when launching into Lego Dimensions - the first time I launch the game with Vulkan it progresses fine (albeit with half the screen cut off), but then when I close and try to launch with the metal backend, it crashes and hangs halfway through the first menu/cutscene screens (as well as seemingly duplicating the menus to infinity in the top left of the screens). Then when I close and re-open and switch back to the Vulkan backend, it crashes/hangs at a similar stage during the first menu/cutscenes. Clearing shader caches and re-launching with Vulkan works sporadically, but most of the time I have to completely remove the Cemu folder in Library/Application Support in order to be able to launch with Vulkan again and not crash/hang. The Metal backend seems to always crash, no matter whether shaders are cleared or Library/Application Support/Cemu is deleted. Here are some videos of what I mean:

first launch with vulkan

then launch with metal

launch again with vulkan

Would love to help debug this as I normally do my dev work on Mac, and it would be excellent if I could test out my PR (#1371) rebased on top of this branch, rather than needing to do my testing on my average Windows laptop

Edit: After enabling metal logging, I got a lot of "not implemented", and "Screenshots are not yet supported on Metal" so I guess I will need to stick with windows for the interim

Thanks for testing! The bug with duplicated frame happened on Minecraft for Wii U some time ago, but I think it got fixed. As for the Vulkan issue, other people have reported as well that Vulkan doesn't behave properly on the metal branch. I will have to investigate that. Do you possibly have a log for the Vulkan crash?

[deReeperJosh]

Thanks for testing! The bug with duplicated frame happened on Minecraft for Wii U some time ago, but I think it got fixed. As for the Vulkan issue, other people have reported as well that Vulkan doesn't behave properly on the metal branch. I will have to investigate that. Do you possibly have a log for the Vulkan crash?

I have a log but would it be more helpful if I enabled some more logging settings/run on the debug build?

metallog.txt
vulkanlog.txt

[SamoZ256]

Thanks for testing! The bug with duplicated frame happened on Minecraft for Wii U some time ago, but I think it got fixed. As for the Vulkan issue, other people have reported as well that Vulkan doesn't behave properly on the metal branch. I will have to investigate that. Do you possibly have a log for the Vulkan crash?

I have a log but would it be more helpful if I enabled some more logging settings/run on the debug build?

metallog.txt vulkanlog.txt

Thanks for the logs. Debug build would probably give more info, but I see that the Metal log is full of screenshot request messages, so I am going to focus on that.

[deReeperJosh]

Built again with the debug version - the duplicated frame issue has gone away so must have not had some of your latest commits in, I am still finding the game hangs around the same place on Metal, even with the screenshot implementation, log attached:
log.txt

[indianajson]

First of all, awesome work on this. We definitely needed a Metal implementation.

Seems the crash I mentioned earlier must have been an issue with my device, as it is now running the Metal engine smoothly. Awesome work!

[SamoZ256]

First of all, awesome work on this. We definitely needed a Metal implementation.

Seems the crash I mentioned earlier must have been an issue with my device, as it is now running the Metal engine smoothly. Awesome work!

Thanks! And exactly what game was crashing?

[SamoZ256]

Built again with the debug version - the duplicated frame issue has gone away so must have not had some of your latest commits in, I am still finding the game hangs around the same place on Metal, even with the screenshot implementation, log attached:

log.txt

Alright, good to hear that. I have no idea what could be causing the game to freeze though.

[indianajson]

Thanks! And exactly what game was crashing?

Paper Mario Color Splash. When I first launched the game using Metal FPS was 1-2 then when I got past the start screen it hard crashed (macOS froze), but after the restart I tried it again with Vulkan and it was fine so I decided to risk it and suddenly Metal was working. Still not sure if it was my device or Cemu.

[PotionBelt]

Amazing work on this project! I'm on an M1 MacBook Pro and gave your Metal backend a go with Wind Waker HD.

In general, things are much snappier than Vulkan with no graphical weirdness I can see, though the areas with many particles e.g. Dragonroost Island are still very choppy / low framerate (fluctuates between 8-15fps on my system). Other places with the same issue include the purple sparkles on the title screen and bomb explosion particles, which also tank the framerate.

Is there a fix for this in the pipeline? Would be happy to keep testing.

[SamoZ256]

Amazing work on this project! I'm on an M1 MacBook Pro and gave your Metal backend a go with Wind Waker HD.

In general, things are much snappier than Vulkan with no graphical weirdness I can see, though the areas with many particles e.g. Dragonroost Island are still very choppy / low framerate (fluctuates between 8-15fps on my system). Other places with the same issue include the purple sparkles on the title screen and bomb explosion particles, which also tank the framerate.

Is there a fix for this in the pipeline? Would be happy to keep testing.

Thank you for your feedback!

I have heard about the frame drop in Wind Waker HD. I don’t know if there is much I can do about it, but if you would make a frame capture (guide: https://github.com/SamoZ256/cemu-frame-capture), I could take a look at it and see if I could make a hack specifically for this game to improve it.

Also, is the frame rate better than Vulkan in the areas with many particles?

[PotionBelt]

Amazing work on this project! I'm on an M1 MacBook Pro and gave your Metal backend a go with Wind Waker HD.
In general, things are much snappier than Vulkan with no graphical weirdness I can see, though the areas with many particles e.g. Dragonroost Island are still very choppy / low framerate (fluctuates between 8-15fps on my system). Other places with the same issue include the purple sparkles on the title screen and bomb explosion particles, which also tank the framerate.
Is there a fix for this in the pipeline? Would be happy to keep testing.

Thank you for your feedback!

I have heard about the frame drop in Wind Waker HD. I don’t know if there is much I can do about it, but if you would make a frame capture (guide: https://github.com/SamoZ256/cemu-frame-capture), I could take a look at it and see if I could make a hack specifically for this game to improve it.

Also, is the frame rate better than Vulkan in the areas with many particles?

Any workaround would be amazing! Have Discord DM'd you the frame captures.

The frame rate is perhaps very slightly better in the particle-heavy areas on Dragonroost Island. With Vulkan that area is more like 8-13fps.

[darkuser89]

log.txt

thanks for the hard work. Unfortunately, Mario Kart doesn't start the second time

[SamoZ256]

log.txt

thanks for the hard work. Unfortunately, Mario Kart doesn't start the second time

Hi! This could probably (temporarily) be solved by clearing the shader cache. Also, make sure you are on the latest testing release.