Skip to content

Commit

Permalink
Build MediaInfo native libraries in GitHub CI
Browse files Browse the repository at this point in the history
This produces highly-optimized MediaInfo library binaries for:

  * Windows x64 (x86-64/AMD64/EM64T/Intel 64)
    (statically linked, so no particular C++ runtime required)
  * macOS x86_64 + arm64 (AArch64) (in one "universal binary" file)
    (all .NET-supported macOS versions)
  * glibc-based Linux x86-64 (Red Hat Enterprise Linux 7 and newer)
  * glibc-based Linux AArch64 (Red Hat Enterprise Linux 7 and newer)
  * musl-based Linux x86-64 (all .NET-supported Alpine versions)
  * musl-based Linux AArch64 (all .NET-supported Alpine versions)

which are the same architectures that AVDump3 currently supports plus
other OS versions/distributions for the same architectures that are
supported by .NET 6 even if not yet supported by AVDump3NativeLib.

Due to the large number of binaries that will need to be included in
the AVDump3 distribution package, they are optimized primarily for size,
potentially at the expense of speed. In my experience, the bulk of time
in AVDump is spent on hashing rather than on MediaInfo, so this should
not be a problem. Furthermore, many of the optimizations improve both
size and speed.

MediaInfo's functionality is heavily culled to include only things that
AVDump3 is actually going to use. These are not general-purpose builds.

I have tried Autotools and CMake for non-Windows and settled on CMake
because it builds faster in every case. (Purely as a guess, perhaps
this is due to Autotools using the notoriously slow Libtool.)
Notably, the slowest build jobs, namely, Linux on emulated ARM,
take ~27min with CMake vs ~35min with Autotools.

The amount of customization required to get optimal output is somewhat
smaller for Autotools, but it is somewhat more arcane, although CMake's
macOS-specific tweaks are arcane as well. CMake has some nicely-named
(if verbose) variables providing native support for -Os and LTO,
but then it lacks configure's --enable-minimal option and uses
suboptimal compiler and linker flags on macOS that cannot be tweaked
without some undocumented variable hacking.

Some of the CMake hacks may no longer be required in the future
if/when I submit the relevant fixes to MediaInfo and if/when CMake
implements nicer overrides for the relevant compiler flags.

Potentially, CMake could also be used on Windows. I have tried
and gotten it far enough to run a build, but it built zlib as a
dynamic library and then tried to reference a static version of it,
failing the build. For what it is worth, I suspect the Visual Studio
project files are better maintained than the CMake files and may
contain more appropriate configuration for Windows. On the other hand,
it would be nice to use a common build process on all platforms.
Furthermore, with CMake it should be possible to use Clang as the
compiler on Windows, which may or may not produce smaller code.

CMake (unlike Autotools) could also be configured to link standard
libraries statically on musl-based Linux, producing binaries that
could in theory run on all Linux distributions, negating the need
for separate glibc-and musl-based binaries. However, when I tried
this and supplied the static-musl MediaInfo.so to AVDump3 on a
glibc-based system, .NET crashed. I was unable to understand why.

For reference, the equivalent Autotools code (without comments) is:

    for project in ZenLib MediaInfoLib; do
      (cd $project/Project/GNU/Library; ./autogen.sh)
      sed -i~ s/-O2/-Os/g $project/Project/GNU/Library/configure
      # or -Oz on macOS
    done
    mv MediaInfoLib/Project/GNU/Library/AddThisToRoot_DLL_compile.sh SO_Compile.sh

    # on Linux:
    . ./setup_devtools.sh
    export AR=gcc-ar NM=gcc-nm RANLIB=gcc-ranlib

    # on macOS:
    export MACOSX_DEPLOYMENT_TARGET='${{ matrix.macosx_version_min }}'
    export Common_Options='--enable-arch-${{ matrix.arch }}'

    export CFLAGS='-flto -fvisibility=hidden'
    export CXXFLAGS='-flto -fvisibility=hidden -fvisibility-inlines-hidden -fno-rtti'
    export LDFLAGS="..."
    export MediaInfoLib_Options='--enable-minimal'

    . ./SO_Compile.sh ${Common_Options}
    # fetch the binary from MediaInfoLib/Project/GNU/Library/.libs/

Possible further optimizations in any case:

MEDIAINFO_ADVANCED_NO could also be defined, but I *think* that would
render mil.Get("Audio_Codec_List") in MediaInfoLibProvider nonfunctional.
But for what it is worth, I think this is the only thing it would affect.

MediaInfo_Option includes a lot of code to handle all the various possible
options that is never used by AVDump3 because it only ever supplies two
specific options. I can dream that it could be somehow removed.

For MSVC (Windows), <ExceptionHandling>false</ExceptionHandling>
could be added to reduce the final size by another 200 KB.
However, if I am reading [MSDN] correctly, this would cause MediaInfo's
defensive catch(...) clauses to catch & mask things like invalid memory
access or division by zero, which should normally crash the application.

[MSDN]: https://docs.microsoft.com/en-us/cpp/build/reference/eh-exception-handling-model
  • Loading branch information
astiob committed Feb 7, 2022
1 parent 6214bc3 commit 676e50e
Showing 1 changed file with 411 additions and 0 deletions.
Loading

0 comments on commit 676e50e

Please sign in to comment.