Added Apple Silicon (M1 & M2) and fixed macOS builds #237
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Currently getting this failure in the build: |
|
Hi @Ishaanahuja7, it looks like a similar PR was already submitted prior to this one. That PR doesn't include the |
|
Hi @space-r7, I have made a pull request to merge darwin_sigar.c changes here: rapid7/sigar#2 Looks like that repo is pretty unmaintained so until they merge that change, we can merge and keep that change here. Once rapid7/sigar maintainers merge my pull request and update libsigar to 1.6.4, we can update the deps folder with that and remove the fixed darwin_sigar.c |
This was referenced Nov 5, 2022
|
@Ishaanahuja7 rapid7 is the maintainers for rapid7/sigar#2, I have updated the target branch for that PR to |
|
@space-r7 @jmartin-r7 rapid7/sigar#2 has landed with new release tag 1.6.5. I have updated the deps dir with the new tar.gz file and fixed Makefile.libsigar to work with the new version. All tests seem to pass and mettle can now be built on Apple Silicon. Please merge this PR soon! |
|
@timwr @space-r7 @jmartin-r7 Can anybody merge this PR? |
|
@Ishaanahuja7 This PR still needs to be reviewed to make sure that it doesn't break any existing functionality in Mettle. Additionally the team may need to source a Apple Silicon machine to test these changes on. The team is currently focused on final changes before we release 6.3 of Metasploit Framework. Once that has been released, we will have more time to circle back around and review this PR. |
|
@gwillcox-r7 Sorry for the ping, didn't know there were other requirements before this could be merged. Sounds good! |
|
@Ishaanahuja7 All good, all part of learning and contributing 👍 |
|
Getting the same error as above during build on M1. On Monterey with Xcode 13.3. According to this issue, this may have been introduced in version 13.: |
|
This error was in Sigar and was fixed in this PR rapid7/sigar#2 with the release of 1.6.5 @busterb confirmed that building and running Sigar was successful on macOS 10.13.6 and macOS 13.1.0 here rapid7/sigar#2 (comment) I'm able to build mettle on M1 with Xcode 14.2 on macOS 13.2 with these changes so unsure why this error is coming up again. Here is my system information and build log: |
|
Sorry for the delay here. I specifically can't build this on Monterey on the M1, still getting the sigar error. Building on an intel macbook running Monterey was successful. Building on an M1 macbook running Ventura was successful: M1 Monterey BuildM1 Ventura Build |
|
Turns out that nfsstats was changed in macOS 10.14.6. Fixed it in a new PR rapid7/sigar#4 Can build Sigar and Mettle on Monterey on M1 now. I don't have an intel Mac but still should work on it as well. I'll update the code to reflect the newer Sigar version when that PR gets merged. |
|
rapid7/sigar#4 is merged with new tag 1.6.6 Updated the code to reflect that. Sigar and Mettle now builds on macOS Monterey as well |
|
This is going to take a few more days to get our automation in place for the new artifact to build, it looks like we will need a newer version of Xcode in our pipeline for this artifact as we build on the oldest compatible toolchain available to support legacy devices as much as possible. Looking at this reference we will need to update to Xcode 12 or later in our automation. |
|
Any progress or updates? Is it possible for you to share the automation code, I can give it a shot on my end as well. |
|
I tested both the x86_64 and aarch64 payloads on Friday, and they worked / passed tests. I still need to test the mic and screenshot functionality. Here's output for x86_64 arch: x86_64 apple darwin |
|
Work is in progress on the automation now, the primary issue is that this update requires building on a new toolchain for macOS. Currently the build is run in macOS 10.11 with a full Xcode 8.2 install to allow for iOS builds. To build Work is ongoing to determine if a newer OS/toolchain is best or if we need to expand on the source. |
|
Looks like this change also has issues executing the x86_64 compile on M1 systems: I am still working out what the build pattern needs to be for this, I may implement a temporary build process to compile M1 binaries on an M1 system for the short term, but this will take a few more days to test out. |
|
The x86_64 cross compile issue on M1 was tracked down to #241. These PRs can land together once we have a working pattern for the build env we would like to utilize going forward. |
|
According to this, macOS 10.15.4 with Xcode 12.2 is ideal as a bare minimum build environment for building both x86_64 and arm64. |
|
@msjenkins-r7 test this please. This comment is to trigger testing of build automation support for the added target. With any luck this should be the final puzzle piece to land this PR. |
|
@jmartin-r7 Mettle Compilation/Packaging test PASSED. Any way to check what the automation code is or logs for this test? When I click details, it sends me back to the pull request page |
|
The test results and artifacts are being validated now, unfortunately access to results currently requires internal team permissions at this time. We are looking into ways we might enhance this process to have github actions expose the process more. That work will be outside the scope of landing this PR. The current build process utilizes macOS VMs created from https://github.com/rapid7/metasploit-vagrant-builders/ building all original artifacts on v1.0.3 which is macOS 10.11 with Xcode 8.2 installed and building the new |
|
The new builds worked for me. Still some work for the Metasploit side. Thank you @Ishaanahuja7 and @jmartin-r7! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
1). Used config.update.sh to update config.guess and config.sub for fixing json-c and mettle build on Apple Silicon
2). Updated rapid7/sigar with rapid7/sigar#2, rapid7/sigar#4, and rapid7/sigar#5 to update it to 1.6.7 to get it to build on Apple Silicon
3). Replaced almost all deprecated macOS APIs