script failing on appcenter (cannot find nuget)

[roubachof]

Hi Jonathan !
I tried to use boots to be able to compile C# 9 on appcenter and used it like this:

# App Center custom build scripts: https://aka.ms/docs/build/custom/scripts

export DOTNET_SKIP_FIRST_TIME_EXPERIENCE=true

dotnet tool install --global boots

# Workaround instead of restarting shell
# see: https://github.com/dotnet/cli/issues/9114#issuecomment-494226139
export PATH="$PATH:~/.dotnet/tools"
export DOTNET_ROOT="$(dirname "$(readlink "$(command -v dotnet)")")"

# https://www.mono-project.com/download/stable/#download-mac
boots https://download.mono-project.com/archive/6.12.0/macos-10-universal/MonoFramework-MDK-6.12.0.145.macos10.xamarin.universal.pkg

# https://github.com/xamarin/xamarin-android
boots https://aka.ms/xamarin-android-commercial-d16-8-macos

The post clone step is doing fine and retrieve successfully the 2 sdks.
BUT, when it hits the nuget restore step, it fails with the following logs:

##[section]Starting: Restore Nuget
==============================================================================
Task         : Command Line
Description  : Run a command line with arguments
Version      : 1.1.3
Author       : Microsoft Corporation
Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=613735)
==============================================================================
[command]/bin/bash -c /usr/bin/find . -name '*.sln' -type f -print0 | /usr/bin/xargs -0 grep -l 'Operation.Stealth.Android.csproj' | /usr/bin/xargs -I '{}' nuget restore '{}' -DisableParallelProcessing
(node:3225) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3225) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3225) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3225) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3225) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3225) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3225) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3225) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3225) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3225) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3225) Warning: Use Cipheriv for counter mode of aes-256-ctr
xargs: nuget: No such file or directory
##[error]/bin/bash failed with return code: 127
##[error]/bin/bash failed with error: /bin/bash failed with return code: 127

Maybe I need to install a nuget command line with boots ?
The boots step is breaking the usage of nuget ?

I must be missing something.

[jonathanpeppers]

The nuget command is part of Mono, you can see where it is by running:

% which nuget
/Library/Frameworks/Mono.framework/Versions/Current/Commands/nuget

nuget works because /Library/Frameworks/Mono.framework/Versions/Current/Commands is added to $PATH delimited by :.

% echo $PATH
...:/Library/Frameworks/Mono.framework/Versions/Current/Commands:...

NOTE: I put the ... just because the output was long

Your script installs a newer Mono, so I'd suggest printing out which nuget and echo $PATH to figure out what is happening here.

[roubachof]

drum rolls
here are the results:

/Library/Frameworks/Mono.framework/Versions/Current/Commands/nuget

/Library/Frameworks/Mono.framework/Versions/4/bin:/usr/local/opt/pipx_bin:/Users/runner/.cargo/bin:/usr/local/lib/ruby/gems/2.7.0/bin:/usr/local/opt/[email protected]/bin:/usr/local/opt/curl/bin:/usr/local/bin:/usr/local/sbin:/Users/runner/bin:/Users/runner/.yarn/bin:/Users/runner/Library/Android/sdk/tools:/Users/runner/Library/Android/sdk/platform-tools:/Users/runner/Library/Android/sdk/ndk-bundle:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/bin:/bin:/usr/sbin:/sbin:/Users/runner/.dotnet/tools:/Users/runner/.ghcup/bin:/Users/runner/hostedtoolcache/stack/2.7.1/x64:~/.dotnet/tools

So it seems fine :_|

[jonathanpeppers]

In the same place you printed this, if you just run nuget by itself, does it print out the help text?

[roubachof]

gosh nuget is running just fine captain !

NuGet Version: 5.9.0.7134
usage: NuGet <command> [args] [options] 
Type 'NuGet help <command>' for help on a specific command.

Available commands:

 add                                  Adds the given package to a hierarchical source. http sources are not supported. For more info, goto https://docs.nuget.org/consume/command-line-reference#add-command.

 client-certs                         Provides the ability to manage list of client certificates located in NuGet.config files

 config                               Gets or sets NuGet config values.

 delete                               Deletes a package from the server.

 help (?)                             Displays general help information and help information about other commands.

 init                                 Adds all the packages from the <srcPackageSourcePath> to the hierarchical <destPackageSourcePath>. http feeds are not supported. For more info, goto https://docs.nuget.org/consume/command-line-reference#init-command.

 install                              Installs a package using the specified sources. If no sources are specified, all sources defined in the NuGet configuration file are used. If the configuration file specifies no sources, uses the default NuGet feed.

 list                                 Displays a list of packages from a given source. If no sources are specified, all sources defined in %AppData%\NuGet\NuGet.config are used. If NuGet.config specifies no sources, uses the default NuGet feed.

 locals                               Clears or lists local NuGet resources such as http requests cache, temp cache or machine-wide global packages folder.

 pack                                 Creates a NuGet package based on the specified nuspec or project file.

 push                                 Pushes a package to the server and publishes it.
                   NuGet's default configuration is obtained by loading %AppData%\NuGet\NuGet.config, then loading any nuget.config or .nuget\nuget.config starting from root of drive and ending in current directory.

 restore                              Restores NuGet packages.

 search                               Searches a given source using the query string provided. If no sources are specified, all sources defined in %AppData%\NuGet\NuGet.config are used.

 setApiKey                            Saves an API key for a given server URL. When no URL is provided API key is saved for the NuGet gallery.

 sign                                 Signs a NuGet package with the specified certificate.

 sources                              Provides the ability to manage list of sources located in NuGet.config files.

 spec                                 Generates a nuspec for a new package. If this command is run in the same folder as a project file (.csproj, .vbproj, .fsproj), it will create a tokenized nuspec file.

 trusted-signers                      Provides the ability to manage the list of trusted signers.

 update                               Update packages to latest available versions. This command also updates NuGet.exe itself.

 verify                               Verifies a signed NuGet package.

[jonathanpeppers]

Maybe try not updating Mono at all, just update Xamarin.iOS, Xamarin.Android, .etc?

Updating Mono would only really affect the MSBuild version you're building with, so probably wouldn't change the resulting app.

[roubachof]

So I commented out getting latest mono preview and got latest xamarin android.

# https://www.mono-project.com/download/stable/#download-mac
# boots https://download.mono-project.com/archive/6.12.0/macos-10-universal/MonoFramework-MDK-6.12.0.145.macos10.xamarin.universal.pkg

# https://github.com/xamarin/xamarin-android
boots https://aka.ms/xamarin-android-commercial-d16-9-macos

nuget is now working fine, but as expected, it fails to compile c# 9 (since only mono preview supports c# 9).

[roubachof]

I also tried latest mono and latest xamarin android, back to the nuget issue...

# https://www.mono-project.com/download/stable/#download-mac
boots https://download.mono-project.com/archive/6.12.0/macos-10-universal/MonoFramework-MDK-6.12.0.145.macos10.xamarin.universal.pkg

# https://github.com/xamarin/xamarin-android
boots https://aka.ms/xamarin-android-commercial-d16-9-macos

Result:

[command]/bin/bash -c /usr/bin/find . -name '*.sln' -type f -print0 | /usr/bin/xargs -0 grep -l 'Operation.Stealth.Android.csproj' | /usr/bin/xargs -I '{}' nuget restore '{}' -DisableParallelProcessing
xargs: nuget: No such file or directory
##[error]/bin/bash failed with return code: 127
##[error]/bin/bash failed with error: /bin/bash failed with return code: 127

[jonathanpeppers]

If you select this, is Mono new enough for C# 9?

[roubachof]

nope, latest stable mono doesn't support C# 9...

[jonathanpeppers]

This is a hack, but maybe your script could make a symbolic link in the current directory.

Untested:

ln -s /Library/Frameworks/Mono.framework/Versions/Current/Commands/nuget nuget

I'm not sure which one goes first off the top of my head, the source or the destination.

[roubachof]

I reached the support of appcenter, we'll see what they can do.

[roubachof]

So finally the answer was to add the script for retrieving mono preview at the Pre-build step and not at the Post-clone step.
You should maybe update the blog post: https://devblogs.microsoft.com/xamarin/boots-xamarin-ci/

[mwerickson]

I have boots installing --preview mono in the pre-build script and I'm now I'm seeing the NETSDK1005 errors.

[beeradmoore]

I tried the ln -s /Library/Frameworks/Mono.framework/Versions/Current/Commands/nuget nuget thing that @jonathanpeppers suggested. I think its not needed (but also doesn't work)

My appcenter-post-clone.sh looks like this.

#!/usr/bin/env bash

which nuget
nuget help
echo $PATH

export DOTNET_SKIP_FIRST_TIME_EXPERIENCE=true

dotnet tool install --global boots

# Workaround instead of restarting shell
# see: https://github.com/dotnet/cli/issues/9114#issuecomment-494226139
export PATH="$PATH:~/.dotnet/tools"
export DOTNET_ROOT="$(dirname "$(readlink "$(command -v dotnet)")")"

boots https://download.mono-project.com/archive/6.12.0/macos-10-universal/MonoFramework-MDK-6.12.0.140.macos10.xamarin.universal.pkg
boots https://aka.ms/xamarin-android-commercial-d16-10-macos

ln -s /Library/Frameworks/Mono.framework/Versions/Current/Commands/nuget nuget

which nuget
nuget help
./nuget help
echo $PATH

The juicy outputs of all of this is

#which nuget
/Library/Frameworks/Mono.framework/Versions/Current/Commands/nuget

#nuget help
NuGet Version: 5.6.0.6489

#echo $PATH
/Library/Frameworks/Mono.framework/Versions/4/bin:/usr/local/opt/pipx_bin:/Users/runner/.cargo/bin:/usr/local/lib/ruby/gems/2.7.0/bin:/usr/local/opt/[email protected]/bin:/usr/local/opt/curl/bin:/usr/local/bin:/usr/local/sbin:/Users/runner/bin:/Users/runner/.yarn/bin:/Users/runner/Library/Android/sdk/tools:/Users/runner/Library/Android/sdk/platform-tools:/Users/runner/Library/Android/sdk/ndk-bundle:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/bin:/bin:/usr/sbin:/sbin:/Users/runner/.dotnet/tools:/Users/runner/.ghcup/bin:/Users/runner/hostedtoolcache/stack/2.7.3/x64

# install everything

#which nuget
/Library/Frameworks/Mono.framework/Versions/Current/Commands/nuget

#nuget help
NuGet Version: 5.9.0.7134

#./nuget help
NuGet Version: 5.9.0.7134

#echo $PATH
/Library/Frameworks/Mono.framework/Versions/4/bin:/usr/local/opt/pipx_bin:/Users/runner/.cargo/bin:/usr/local/lib/ruby/gems/2.7.0/bin:/usr/local/opt/[email protected]/bin:/usr/local/opt/curl/bin:/usr/local/bin:/usr/local/sbin:/Users/runner/bin:/Users/runner/.yarn/bin:/Users/runner/Library/Android/sdk/tools:/Users/runner/Library/Android/sdk/platform-tools:/Users/runner/Library/Android/sdk/ndk-bundle:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/bin:/bin:/usr/sbin:/sbin:/Users/runner/.dotnet/tools:/Users/runner/.ghcup/bin:/Users/runner/hostedtoolcache/stack/2.7.3/x64:~/.dotnet/tools

This still resulted in nuget not found when it tries to do a restore like so

[command]/bin/bash -c /usr/bin/find . -name '*.sln' -type f -print0 | /usr/bin/xargs -0 grep -l 'MyProject.Droid.csproj' | /usr/bin/xargs -I '{}' nuget restore '{}' -DisableParallelProcessing
xargs: nuget: No such file or directory

As far as I can tell the linked file wouldn't work because it executes nuget restore not ./nuget restore (not sure if executing directly from /bin/bash -c changes that though). The weird thing is that the binary at /Library/Frameworks/Mono.framework/Versions/Current/Commands/nuget did get changed, the environment path only changed with :~/.dotnet/tools on the end, AND we can still execute nuget just fine in this script. Does that mean when this script ends end the next starts there is something funky going on with environment variables or something? What would cause nuget to not be found in the find + xargs line?

Moving the build file from appcenter-post-clode.sh to appcenter-pre-build.sh did allow it to work, however I don't think it actually worked. From the build log:

Microsoft (R) Build Engine version 16.6.0 for Mono
Copyright (C) Microsoft Corporation. All rights reserved.
...
_ResolveXamarinAndroidTools:
   Found Xamarin.Android 11.1.0.26

This should be Mono 16.12 and Xamarin.Android 11.3.0.1. So it appears neither solution works for AppCenter build? Unsure if its an AppCenter problem or a boots problem, I think I'm leaning towards AppCenter though.

For me, this is an issue where Android 11 builds can be built from AppCenter but can't be deployed to a device. So back to their repo to report the problem.

[softlion]

After lots of tests and tricks, i get the exact same results.

But my motivation is different: there is a bug in the latest mono distributions: they include a nuget app which is unable to restore a UWP project. And the restore happens even if the UWP project is not used in the current solution configuration.

So i would like to go back to 6.12.0.145. But in the post clone step (as appcenter executes nuget restore between post clone and pre build). But if i do that, i get the "nuget not found" error above too and nothing works (reinstalling nuget, updating it, replacing by symbolic links, ...).

[roubachof]

@mwerickson @softlion try to add:

echo "Update nuget (fix NETSDK1005)"
sudo nuget update -self

at the end of your appcenter-post-clone.sh

[softlion]

Already tried. @roubachof

[CallumHoughton]

Also having the same Xamarin Forms and App Center issues as @softlion, and the post-clone script also doesn't work for me with the same xargs: nuget: No such file or directory error after installing a earlier version of mono via the script.

some people reporting removing the UWP project allows the App Center builds to work (as a temporary fix).

the mono update bug is referenced in mono/issues/21180

[softlion]

@CallumHoughton I removed the uwp apps from the solution and it works great.
I removed all pre build script, and used only a post clone script with "sed" entries to dynamically remove uwp libs and app fromt the solution.
So C#9 is definitely supported by appcenter now.

[CallumHoughton]

@softlion No problem and that's a really good idea! Would you mind sharing your post clone script please?

[softlion]

SLN_PATH="$APPCENTER_SOURCE_DIRECTORY/xxxxx.sln"
#sed on mac requires a backup extension (here:'')
sed -i '' '6,15d' $SLN_PATH
sed -i '' '/ABCDEFAB-1234-1234-1234-99885E1AC6B5/d' $SLN_PATH
sed -i '' '/ABCDEFAB-1234-1234-1234-99885E1AC6B5/d' $SLN_PATH
sed -i '' '/ABCDEFAB-1234-1234-1234-99885E1AC6B5/d' $SLN_PATH
sed -i '' '/ABCDEFAB-1234-1234-1234-99885E1AC6B5/d' $SLN_PATH

[Haapsaari-Juha]

@softlion and @CallumHoughton My problem also that I have XF 5 app and UWP project in solution. Build fails now with the mono/iOS/Android combos available in AppCenter. My solution that I can use also in future (scripts can be always in iOS and Android folders) was to use following bash script as appcenter-post-clone.sh (using dotnet sln to remove UWP project):
#!/usr/bin/env bash
SLN_PATH="$APPCENTER_SOURCE_DIRECTORY/filename.sln"
UWP_PATH="$APPCENTER_SOURCE_DIRECTORY/path to UWP project file/UWP project filename.csproj"
dotnet sln $SLN_PATH remove $UWP_PATH

[mgierlasinski]

@Haapsaari-Juha Thanks for sharing, that also worked for me 🥳 That was odd, I'm building Android and iOS csproj and not sln, but still having UWP as part of sln was failing restore.