Prep work for running test suite on macos-latest

[jnikula]

GitHub allows running workflows on macos in addition to ubuntu. I'd potentially like to start running tests on it, but there are some hiccups. I'm not quite there yet with fixing everything in setting up the workflow, but here are two small fixes for starters.

EDIT: This is expanding quite a bit...

EDIT: This + the actual workflow is #249. I'm keeping these separate as this is fairly straightforward prep work, and the workflow doesn't pass.

EDIT: And back to basics. This is just the good stuff now.

[BrunoMSantos]

Good goal and it's a tentative approve from me, but see my other comments 1st.

[jnikula]

Switched to using os.path.realpath(srcdir) to handle the symlinks, which is a more general approach I think. Also improved the commit messages around get_message() use slightly.

Testing the macos workflows in a private repo, I'm getting pretty close to passing the tests, but not quite.

[jnikula]

With the xcode (or whatever the "stock" toolchain is called) version of libclang, I'm hitting e.g.:

E         + WARNING: using-alias.cpp:2: alias declarations are a C++11 extension
E         + WARNING: using-alias.cpp:5: alias declarations are a C++11 extension
E         + WARNING: using-alias.cpp:9: alias declarations are a C++11 extension
E         + WARNING: using-alias.cpp:12: variadic templates are a C++11 extension
E         + WARNING: using-alias.cpp:13: alias declarations are a C++11 extension
E         + WARNING: using-alias.cpp:16: variadic templates are a C++11 extension
E         + WARNING: using-alias.cpp:17: alias declarations are a C++11 extension

and

E         + ERROR: struct-class.cpp:26: unknown type name 'constexpr'
E         + /Users/runner/work/github-actions-playground/github-actions-playground/test/cpp/struct-class.cpp:23: WARNING: Error in declarator or parameters-and-qualifiers
E         + If pointer to member declarator:
E         +   Invalid C++ declaration: Expected identifier in nested name. [error at 48]
E         +     public static constexpr  static_constexpr_member
E         +     ------------------------------------------------^
E         + If declarator-id:
E         +   Invalid C++ declaration: Expected identifier in nested name. [error at 48]
E         +     public static constexpr  static_constexpr_member
E         +     ------------------------------------------------^
E         +

and

E         + ERROR: class.cpp:28: unknown type name 'constexpr'
E         + ERROR: class.cpp:34: expected ';' at end of declaration list
E         + ERROR: class.cpp:37: unknown type name 'constexpr'
E         + WARNING: class.cpp:70: deleted function definitions are a C++11 extension
E         + WARNING: class.cpp:73: 'override' keyword is a C++11 extension
E         + WARNING: class.cpp:91: defaulted function definitions are a C++11 extension

and

E         + ERROR: namespace.cpp:26: unknown type name 'constexpr'
E         + WARNING: namespace.cpp:52: scoped enumerations are a C++11 extension

and

E         + ERROR: class.cpp:28: unknown type name 'constexpr'
E         + ERROR: class.cpp:34: expected ';' at end of declaration list
E         + ERROR: class.cpp:37: unknown type name 'constexpr'
E         + WARNING: class.cpp:70: deleted function definitions are a C++11 extension
E         + WARNING: class.cpp:73: 'override' keyword is a C++11 extension
E         + WARNING: class.cpp:91: defaulted function definitions are a C++11 extension
E         + /Users/runner/work/github-actions-playground/github-actions-playground/test/cpp/class.cpp:25: WARNING: Error in declarator or parameters-and-qualifiers
E         + If pointer to member declarator:
E         +   Invalid C++ declaration: Expected identifier in nested name. [error at 48]
E         +     public static constexpr  static_constexpr_member
E         +     ------------------------------------------------^
E         + If declarator-id:
E         +   Invalid C++ declaration: Expected identifier in nested name. [error at 48]
E         +     public static constexpr  static_constexpr_member
E         +     ------------------------------------------------^

Additionally, there are some failures to find C++ headers. #252 improved the situation in that the paths aren't wrong, but still issues.

[jnikula]

With the homebrew version of libclang, I think all of the issues in both C and C++ boil down to inability to find correct C headers. Looks like the C++ headers are found just fine, but the C++ headers including C headers also hit issues. For example, test_parser[cpp/template]:

  + CRITICAL: wchar.h:89: 'stdarg.h' file not found

but template.cpp only includes <vector> which is found just fine.

Again, I think the situation is slightly better with #252 but I don't know what gives.

[jnikula]

Anyway, the pull request at hand is a prerequisite no matter what. #253 has also been very helpful.

[BrunoMSantos]

Good to merge I think!

[jnikula]

Good to merge I think!

Thanks, merged!

Can you see the detailed github action results, with logs etc, for my pull requests?

[BrunoMSantos]

Can you see the detailed github action results, with logs etc, for my pull requests?

I've had a quick look yesterday, but I need to take a bit longer on that one. Haven't seen your latest updates at all though.

[jnikula]

I've had a quick look yesterday, but I need to take a bit longer on that one. Haven't seen your latest updates at all though.

Okay, with the prep work merged, I can just send a workflow PR. Mostly I've been playing in a private repo.

[jnikula]

@BrunoMSantos Increasingly the wrong place to discuss, but anyway: Looks like we need to specify the C++ standard to use for a lot of tests. The default is different on macos. I just added -std=c++17 to the .yaml files of a bunch of tests. Is that okay, or should we add a default in testenv.py that could be overridden in .yaml?

Still having include issues, and it's driving me nuts. Xcode libclang can't find C++ headers, but homebrew version can. Xcode libclang can find C headers, but homebrew version can't. If one or the other worked, I'd just call it a day.

[BrunoMSantos]

I only have #249 to go by. Care to make a draft PR with the latest version?

Looks like we need to specify the C++ standard to use for a lot of tests. The default is different on macos. I just added -std=c++17 to the .yaml files of a bunch of tests. Is that okay, or should we add a default in testenv.py that could be overridden in .yaml?

Other than that, assuming a default may be wise (perhaps the difference is in the clang version each system uses by default, in which case it doesn't hurt to be explicit). Adding it in every test case is not so good I think as the test developer may not immediately know he needs to.

Still having include issues, and it's driving me nuts. Xcode libclang can't find C++ headers, but homebrew version can. Xcode libclang can find C headers, but homebrew version can't. If one or the other worked, I'd just call it a day.

Alas I know nothing of macos, but that's weird indeed. How are you testing this? The CI itself or do you have a system on your end? I.e. can I see the output?

[jnikula]

I only have #249 to go by. Care to make a draft PR with the latest version?

Sure. #254