Thunks: Add new pointer annotations to assist data layout analysis

[neobrain]

Overview

This PR adds a number of annotations related to pointer struct members and pointer function parameters. This allows the data layout analysis framework added in #3156 to operate on a wider range of thunked APIs.

The initial version of the data layout analysis framework is limited to "simple" structures. "Simple" means that pointer members are only supported if they pointee-type was inferred to have compatible data layout itself. This excludes a couple of important use cases:

• Structs containing pointers to "opaque" types that have no visible definition (where the referenced data is only ever created/read/written on the host side)
• Structs containing types that have consistent data layout across architectures, but the data layout analysis module doesn't support processing them yet (e.g. union types)
• Functions where only one or two parameters are pointers to non-compatible data (i.e. automation is desired for the other parameters but not supported yet)
• Self-referencing structs such as linked-list

To handle these use cases, two new types of annotations are added (in contrast to fex_gen_config, which describes only functions):

• Parameter annotations: fex_gen_param describes specific parameters within a function
• Type annotations: fex_gen_type describes types, such that the described behavior will be applied to all functions/structs they are used in

Implementation

opaque_type/assume_compatible_data_layout annotations

These instruct data analysis to assume the underlying data is compatible across all architectures. assume_compatible_data_layout can be used for both pointers and non-pointers, whereas opaque_type ensures the type is only ever used as a pointer.

Importantly, this means assume_compatible_data_layout can handle types embedded as non-pointers in structs. This should only be used over opaque_type if the layout has actually been manually inspected though, of course.

opaque_type is purely a type annotation (since this property shouldn't depend on context), whereas assume_compatible_data_layout may be used as either a type annotation or a function parameter annotation.

Typical use cases are "handle types", union types, and linked list types.

ptr_passthrough annotation

This function parameter annotation is used to force passing individual function parameters without modification to a custom host implementation (fexgen::custom_host_impl) even if the parameter type is known to differ between architectures. A new guest_layout wrapper template is added to make this behavior explicit: The argument for the host function then changes in type from T* to guest_layout<T*>.

For example:

 int fexfn_impl_wl_proxy_add_listener
        wl_proxy *proxy,
        guest_layout<void (**)()> array_of_callbacks,
        guest_layout<void*> data) { ... }

For now, guest_layout is a loaded foot gun: It provides no extra guarantees or layout conversion to deal with data layout incompatibilities. A later PR will remedy this. Even now the wrapper adds value though, because we've been using the foot gun all along but we didn't make it explicit.

Typical use cases are pointers-to-pointers, function-pointer-parameters. More generally, it allows for complex manual repacking of struct arguments.

Embed annotations into the signature for Vulkan-esque function pointers

Some APIs (prominently vkGetDeviceProcAddr or glXGetProcAddr) return host-function pointers to the guest. The trampolines we insert guest-side to handle this used to be unique per-signature, but additionally they now need to be unique per set-of-annotations.

Since this adds up to a lot of information to carry around at compile time, this turns template<typename Result, typename... Args> void ForIndirectCall(void*); into this rather heavy code:

template<typename Result, typename... Args>
struct GuestWrapperForHostFunction {
  template<ParameterAnnotations... Annotations>
  void Call(void* argsv);
};

Impact

Since strict mode (see #3156) is still only enabled for unit tests, impact on real libraries is limited. Functionally there should be no difference, but the generated code will look slightly different for functions using ptr_passthrough. Similarly, since Vulkan-esque function pointers now carry around annotation at compile-time, the callback list is slightly different.

C++20 is required for the callback change due to the use of non-type template parameters. This version bump could be delayed but other features in my PR queue will also require C++20. If C++17 support is desirable, I propose temporarily requiring C++20 and restoring C++17 support once this PR series is complete.

Future

The next PR after this will add the new annotations to all of our existing thunk libraries. We'll then be able to turn on strict mode for data layout analysis unconditionally.

[neobrain]

Revisit this. Adding a fixed prefix hasn't caused any problems so far, but to be on the safe side it may indeed be needed to add the full function name.

[neobrain]

Actually this is safe, since the type name for a function pointer type will never conflict with a function name.

[Sonicadvance1]

C++20 is required for the callback change due to the use of non-type template parameters. This version bump could be delayed but other features in my PR queue will also require C++20. If C++17 support is desirable, I propose temporarily requiring C++20 and restoring C++17 support once this PR series is complete.

We already compile both FEXCore and FEX's frontend as C++20, no reason for thunks to require anything older.

[Sonicadvance1]

Seems reasonable to me with a first pass.