feat: Verified mods auto-downloading

[Alystrasz]

This pull request introduces automatic mods fetching.

NorthstarLauncher associated pull request: R2Northstar/NorthstarLauncher#362

---

This allows for the client to download and install mods that are required by a server (client+server-side mods, like new game mods or maps for instance). List of verified mods is local (compiled in the dll), but it can be fetched from master server in the future, to ease the addition of new verified mods.

With this feature, if users want to join a server requiring a gamemode mod or a map mod, their game can automatically download it for them, instead of forcing them to go through manual mod installation process (which can be hard for some people). While downloading a mod, progress download is displayed and live-updated on a dialog pop-up.

Verified mod entries

A verified mod entry looks like this:

{
    "Fifty.mp_frostbite": {
        "DependencyPrefix": "Fifty-Frostbite",
        "Versions": [
            { "Version": "0.0.1", "Checksum": "99fc4f0b3363e44985f20525a2bf079..." },
            { "Version": "0.0.2", "Checksum": "40780308e0e4a70f7f35f50f2ca188a..." }
        ]
    }
}

Versions are checked individually to ensure no malicious code has been added during mod updates.

Please note that it is up to the community to decide what's needed for a mod to be "verified", and how such mod is "verified".

Mod installation process

1. Check if mod is verified
   a. Check if it is listed
   b. Check if version is listed
2. Download mod archive from Thunderstore
   a. Fetch zip from API
   b. Compare checksum with local checksum
3. Unzip mod archive content directly into game folder
4. Reload mods list
5. Remove downloaded archive

Please note that downloaded archive is checksumed, to verify it is the very archive that has been verified.

Changes

Northstar.Client

• OnServerSelected method becomes async, to allow waiting for mod verification;
• _OnServerSelected method now displays some dialogs:
  • One error message when one attemps to join a server requiring a mod which cannot be auto-downloaded;
  • One loading message, displaying while a verified mod is being downloaded;
  • One error message when mod downloading/extraction failed;
• If a needed mod is already present but disabled, this activates it.

Launcher

• Includes a new library, libzip
• Includes the already-included but not linked openssl lib;
• All verified-mods-related code is stored in verifiedmods.cpp
• Squirrel-exposed methods:
  • bool NSIsModVerified(string modName, string modVersion): tells if a mod is verified or not;
  • void NSDownloadMod(string modname, string modVersion): starts downloading a mod; returns nothing because actual mod downloading is done in a thread, as not to block UI;
  • bool NSIsModBeingDownloaded(string modName, string modVersion): tells if a mod is being downloaded; this is used by UI to check if it should display loading dialog to the user;
  • array<float> NSGetCurrentDownloadProgress(): returns progression in percentage of current download/extraction;
  • string NSGetModExtractionResult(): returns result of mod download/extraction;

Testing

I tested this PR with two mods:

• Juicy.Zircon_Spitfire, a HUGE (1.7GB) skin mod (set as client-required for the sake of testing this);
• Fifty.mp_frostbite, a map mod.

Maps mods are a good example of mods that would benefit from auto-downloading, because they are required on both client and server sides.

Media

If mod is not verified

If something goes wrong during mod download/extraction

If mod is verified

TODOs

• Display an error dialog if something went wrong during download
• Localization

[ASpoonPlaysGames]

Imo, the percentage downloaded for the currently downloading mod should be shown since mods like map mods will end up being a significantly sized download.

Also, it might be better to get all of the auto-downloadable mods before downloading each of them individually, so that we could show the number of mods downloaded/downloading

[pg9182]

I can implement the master server side of this whenever you need it.

[GeckoEidechse]

I can implement the master server side of this whenever you need it.

Master server would just pull from some repo, right? So that we have a way of organising approving verified mods via PRs

[Alystrasz]

Imo, the percentage downloaded for the currently downloading mod should be shown since mods like map mods will end up being a significantly sized download.

Do you have an idea of the size of a complete map? Maybe an example of a "big" mod from Thunderstore?

I can implement the master server side of this whenever you need it.

Appreciate the help! I will definitely ping you when this is merged.

Master server would just pull from some repo, right?

That's what was agreed upon, yep. @GeckoEidechse could you create it? 😄

[ASpoonPlaysGames]

Do you have an idea of the size of a complete map? Maybe an example of a "big" mod from Thunderstore?

Likely a few hundred megabytes when compressed tbh, given they would include textures as well as the actual bsps/stbsps

An example of a "big mod" would be Juicy's Zircon Spitfire (about a 200MB download?)

[Alystrasz]

Imo, the percentage downloaded for the currently downloading mod should be shown since mods like map mods will end up being a significantly sized download.

This has been added in 2b7075a.

[BobTheBob9]

Imo, the percentage downloaded for the currently downloading mod should be shown since mods like map mods will end up being a significantly sized download.

This has been added in 2b7075a.

could actual size of download and amount downloaded also be shown in megabytes potentially? likely in the content of the message rather than header

[Alystrasz]

could actual size of download and amount downloaded also be shown in megabytes potentially? likely in the content of the message rather than header

Sure! I added this in c39221 (launcher) and 250a18c (mods).

Since zip decompression takes much more time than download (1min VS 6s), I'll have to display zip extraction progression as well

[Alystrasz]

Ok so I would like to get some reviews on this already-existing code basis, even if it needs to be reworked after whole mod system rework (see issue R2Northstar/NorthstarLauncher#396).

---

How to test?

1. Use the Northstar.Client mod from this PR (feat: Verified mods auto-downloading #542);
2. Build launcher files from NorthstarLauncher PR (feat: Verified mods auto-downloading NorthstarLauncher#362) and move them to your game folder;
3. Launch game (if you're missing DLLs, you can download them here);
4. Try to join "Alystrasz's realm" server, you should see mods being downloaded before you join the server.

[uniboi]

Just some naming things, the rest looks fine

[ASpoonPlaysGames]

@Alystrasz I know that you are working on the Launcher side of this (in a separate branch though right?)

Should this PR be considered obsolete?

[Alystrasz]

@Alystrasz I know that you are working on the Launcher side of this (in a separate branch though right?)

Should this PR be considered obsolete?

This one is obsolete yep, I think I'll create a new one, just like the launcher side

[GeckoEidechse]

With #761 merged I think we can close this PR if I'm not mistaken ^^