Incorrect function pointers when using VULKAN_LOADER with vkGetInstanceProcAddr

[MaikKlein]

I am the maintainer for a Vulkan library for Rust and recently I got an interesting bug report.

The user is creating a portability library and they try to use the LunarG Vulkan samples as a test case. Behind the scenes the portability library is calling my library.

We are setting the loader in the cmake, like this set (VULKAN_LOADER "/home/maik/src/portability/target/debug/libportability.so").

The bug is that the vk* functions are just calling themselves until they overflow the stack.

Behind the scenes in my Vulkan library I use vkGetInstanceProcAddr and vkGetDeviceProcAddr at run time. vkGetInstanceProcAddr is the only symbol that I load from the libvulkan.so.1 directly.

I printed the function pointers to the console like this

StaticFn
get_instance_proc_addr => 0x7f38e8be1c00


EntryFn
create_instance => 0x7f38eaa77990
enumerate_instance_extension_properties => 0x7f38eaa7b340
enumerate_instance_layer_properties => 0x7f38eaa789f0

port: enumerate_instance_extension_properties = 0x7f38eaa7b340 // <== wrong

You can see here that 0x7f38eaa7b340 appears two times. My Vulkan library got the function pointer of the portability lib, where it should have gotten the function pointer of the actual Vulkan function. And I got this by calling vkGetInstanceProcAddr with "vkEnumerateInstanceExtensionProperties".

Could this be related to how the LunarG samples uses the VULKAN_LOADER?

When I load all the symbols from the libvulkan.so.1 directly, everything seems to work out.

Any idea why vkGetInstanceProcAddr would give me the wrong function pointer?

[MarkY-LunarG]

I'm not clear on something, you point out that 0x7f38eaa7b340 appears twice, but both are for enumerate_instance_extension_properties. What is supposed to be the difference between these two? Is one supposed to be the call from you, and the second one supposed to be the portability library calling itself again?

EntryFn: enumerate_instance_extension_properties => 0x7f38eaa7b340
port: enumerate_instance_extension_properties = 0x7f38eaa7b340 // <== wrong

Just for clarity, your call stack is this, correct?

Rust App-->Rust Vulkan Library-->gfx-rs/portability-->Vulkan Loader-->Vulkan ICD

And all of this (except the ICD of course) are statically linked?
And, the issue is occurring somewhere after the "Rust Vulkan Library" but before the "Vulkan ICD", right?

If that's the case, then it seems like it must be in the portability library that the GetInstanceProcAddr is returning the wrong value.

Do you happen to have an actual call-stack?

As far as setting the VULKAN_LOADER in CMake, I'm not sure if that works. I've never done it. Perhaps Lenny can give you more feedback on that.

[kvark]

Hi @MarkY-LunarG and thanks for getting back quickly!
@MaikKlein is out of time zone now, so I'll try to answer a bit on their behalf:

you point out that 0x7f38eaa7b340 appears twice, but both are for enumerate_instance_extension_properties. What is supposed to be the difference between these two?

The first one is taken from "libvulkan.so.1" dynamic library by

calling vkGetInstanceProcAddr with "vkEnumerateInstanceExtensionProperties"

The second one is the function from the portability layer ("libportability.so") that is pointed to by VULKAN_LOADER.

Just for clarity, your call stack is this, correct?

Rust App-->Rust Vulkan Library-->gfx-rs/portability-->Vulkan Loader-->Vulkan ICD

And all of this (except the ICD of course) are statically linked?

We are talking about running Vulkan samples here, so the call stack is (with marked linking methods):

"Vulkan API Sample" -dynamic> "gfx-rs/portability" -static> "gfx-rs" -static> "Rust Vulkan library (ash)" -dynamic> "Vulkan Loader/ICD"

And, the issue is occurring somewhere after the "Rust Vulkan Library" but before the "Vulkan ICD", right?

I don't know. Maybe it's Vulkan ICD that gives ash pointers to the portability library instead of the actual driver?

Do you happen to have an actual call-stack?

My call stack can be seen at ash-rs/ash#47 (comment)