Compression

This page describes how libdragon achieves data compression in general for different type of assets (executables, graphics, audio). It gives some general guidance and explains the rationale of the various defaults.

Overview of compression in libdragon

Libdragon ships with three different data compression algorithms, generically identified by level numbers from 1 to 3 (with "level 0" sometimes being used to refer to "no compression"). Increasing the level means slower compression during builds, slower decompressions at runtime, but more compression ratio.

The actual algorithm behind each level is considered an implementation detail; as new algorithms are tested on N64, libdragon can change the algorithms at any point, even in the stable branch (that is, a change of compression algorithm is not considered a breaking change). It is thus important that compression of data happens during build, and compressed data is not committed to the game repository: such pre-compressed, committed data might in fact turn out to be unusable with future libdragon versions.

• Level 1: this is fastest compression level. The ratio is not great (normally worse than gzip, just to give a ballpark) but it is so fast at decompressing that it achieves an important property: data compressed at level 1 is faster at loading and decompressing, than loading uncompressed data. This surprising result happens because the time saved by loading data fewer bytes from ROM is enough to counterbalance the time spent in decompressing the data. This property has been proven true with so many different kind of data, that can be considered true for all data. Currently, the algorithm used is LZ4 by Yann Collet.
• Level 2: this is a good intermediate compression level. The ratio is quite good (better than gzip -9 in most cases), but it is still decently fast at decompressing. Currently, the algorithm uses is LHA-LH5 in the stable branch, and Aplib in the preview branch.
• Level 3: this is an ultra high compression level. It is tuned for absolutely maximum compression at the expense of decompression time. The compression ratio is incredibly high, and it approaches best-in-class on PC too (like xz or 7zip), thanks also to the fact that N64 files are generally quite small by today's standard. Decompression time is quite high though: turning it on for all assets will definitely, noticeably increase loading times for your game, and possibly making them too long. It is mostly meant to be used for rarely loaded assets (eg: error screens) or in situations where loading times are not important. It is currently available only in the preview branch, and it uses the Shrinkler algorithm

This is a table that recaps some datapoints (measured against a fixed corpus of game assets) and give some advice.

Level	Ratio	Ratio vs gzip	Load+Decomp speed	Notes
1	20%	+2%	100 KiB/s	Faster than uncompressed, used by default on everything
2	30%	+10%	200 KiB/s	Good for assets loaded during level changes; might not be fast enough for data streamed during game
3	50%	+40%	30 KiB/s	Loading is very slow; good for assets loaded very rarely like error screen, or for situations where loading time is not a huge issue

Compressing and loading images (sprites, textures) and fonts

Normally, images are processed via the (mksprite)[https://github.com/DragonMinded/libdragon/wiki/Mksprite) tool and converted into the .sprite format. Fonts instead are processed via the mkfont tool and converted into the .font64 format.

Both of these formats accept the command line option -c or --compress which allows to specify the compression level to be used for these files. By default, level 1 is used. To change the compression level, you can add the option to your Makefile like this:

filesystem/%.sprite: assets/%.png
	@mkdir -p $(dir $@)
	@echo "    [SPRITE] $@"
	@$(N64_MKSPRITE) --compress 2 -o "$(dir $@)" "$<"

Note that libdragon does not support any form of lossy image compression at the moment (es: jpeg).

Compressing game code

Note

Compression of game code is only supported in the preview branch at the moment.

Libdragon bootcode (IPL3) supports decompression of game code at boot time. Libdragon makefiles using n64.mk are already setup for that: by default, it compresses the game code with level 1 compression, which makes the ROM actually boots faster than uncompressed.

To change the compression level of game code, simply adds this variable to your Makefile:

# Set game code compression to level 2
N64_ROM_ELFCOMPRESS = 2

DSOs (overlays, that is dynamic libraries of code that is loaded at runtime) are also compressed by default with level 1. To change that, a separate Makefile variable is used:

# Set DSOs compression to level 2
DSO_COMPRESS_LEVEL = 2

This is an example testing the various compression levels on the "Brew Volley" game example in libdragon:

Level	Size	Ratio	Compression time
0	412 KiB	100.0%	0.00 s
1	245 KiB	59.5%	0.39 s
2	177 KiB	43.0%	2.93 s
3	155 KiB	37.6%	6.09 s

Compression times were measured on an Apple MacBook Pro 2021 with M1 Pro. Notice that compression times do impact the development cycle when activated in the Makefile, as the time is paid every time a new ROM is made. It is advised to keep compression at level 1 for the standard development cycle.

For comparison, gzip --best produces a 188 KiB file, and xz --best produces a 150 KiB file.