Merge branch 'develop-sync' into newer-firmware

obsidiansystems · Mar 14, 2023 · ddcd405 · ddcd405
2 parents 5421c4f + be85fb5
commit ddcd405
Show file tree

Hide file tree

Showing 8 changed files with 164 additions and 20 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1,5 +1,6 @@
 target
 app.json
+Cargo.nix
 
 result
 result-*
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -1,12 +1,28 @@
 # Developing an Alamgu ledger App
 
-## Building the app from source
+## Dependencies and development environments
 
-This application has been packaged up with [Nix](https://nixos.org/).
+### Background info
 
-### Nix/Linux
+An Alamgu ledger app is mostly written in Rust, and efforts have been taken to ensure it is developed in a fairly standard way (for embedded Rust programs).
 
-Using Nix, from the root level of this repo, run:
+These dependencies are needed:
+
+ - Rust toolchain with unstable features (including `rustc` and LLD the LLVM Linker).
+ - C toolchain targeting freestanding ARM (including `newlib` as a rump libc)
+ - ["linker wrapper" script from our fork of the SDK.](https://github.com/alamgu/ledger-nanos-sdk/blob/memory-fixes/scripts/link_wrap.sh)
+
+Additionally for testing these are needed:
+
+ - Node
+ - Yarn
+ - [Speculos] the official Ledger emulator
+
+[Speculos]: https://github.com/ledgerHQ/speculos
+
+### Getting a development environment with Nix
+
+The easiest way to get all these dependencies is with Nix:
 
 ```bash
 nix-shell -A $DEVICE.rustShell
@@ -18,16 +34,25 @@ where `DEVICE` is one of
  - `nanox` for Nano X
  - `nanosplus` for Nano S+
 
-Note as described in the main [read-me](./README.md),
-it is currently not possible to side-load apps on the on Nano X, so one can only test in the emulator.
+### Getting a development environment without Nix
 
-The [cargo-ledger](https://github.com/LedgerHQ/cargo-ledger.git) builds, outputs a `hex` file and a manifest file for `ledgerctl`, and loads it on a device in a single `cargo-ledger ledger -l nanos` command in the rust-app folder within app directory.
+Exact instructions are not provided.
+See the [`./docker`](./docker) subdirectory or [`GitHub Actions`](.github/workflows/rust.yml) secondary CI.
+
+[main read-me]: ./README.md
+
+## Updating the lock file (`Cargo.lock`)
 
-You do not need to install cargo-ledger outside of the nix-shell.
+Nix needs additional information not provided in the Cargo lock file for the most robust form of supply-chain integrity for git dependencies.
+This information is contained in `crate-hashes.json` at the root of this repo.
 
-Before installing, please ensure that your device is plugged, unlocked, and on the device home screen.
+After modifying `Cargo.lock`, please run
+```
+./update-crate-hashes.sh
+```
+in order to regenerate this file and keep it up to date.
 
-## Running tests
+## Running automated tests with Speculos
 
 Using Nix, from the root level of this repo, run:
 
@@ -36,3 +61,20 @@ nix-shell -A $DEVICE.rustShell
 cd rust-app/
 cargo test --target=$DEVICE.json
 ```
+
+## Deploying development builds to real hardware
+
+The easiest thing to do is just run a Nix build as described in the [main read-me].
+Nix will always track the latest changes, freshly rebuilding components as needed.
+
+That said, it is also possible to use Cargo build.
+This useful for the quickest "debug loop".
+
+The [cargo-ledger](https://github.com/LedgerHQ/cargo-ledger.git) builds, outputs a `hex` file and a manifest file for `ledgerctl`, and loads it on a device in a single `cargo-ledger ledger -l nanos` command in the rust-app folder within app directory.
+
+(You do not need to install `cargo-ledger` if you are using the nix-provided development shell, as it provides it.)
+
+Before installing with either method, please ensure that your device is plugged, unlocked, and on the device home screen.
+
+Note as described in the [main read-me](./README.md),
+it is currently not possible to side-load apps on the on Nano X, so one can only test in the emulator.
diff --git a/README.md b/README.md
@@ -34,6 +34,40 @@ There is a separate tarball for each device.
 
 #### Build one yourself, with Nix
 
+##### Set up build caches (optional)
+
+In addition to this app itself, the entire toolchain is packaged from source with Nix.
+That means that with usuing a pre-populated source of build artifacts, the first build will take a **very long time** as everything specific to Alamgu is built from source.
+(Other packages could also be built from source, but Nix by default ships configured to use the official `cache.nixos.org` build artifacts cache.)
+
+If you are comfortable trusting Obsidian Systems's build farm, you can use our public open source cache for this purpose:
+
+  - Store URL: `s3://obsidian-open-source`
+  - Public key (for build artifact signatures): `obsidian-open-source:KP1UbL7OIibSjFo9/2tiHCYLm/gJMfy8Tim7+7P4o0I=`
+
+To do this:
+
+1. First you want to include these two in your `/etc/nix/nix.conf` settings file.
+   After doing so, it should have two lines like this:
+   ```
+   substituters = https://cache.nixos.org/ s3://obsidian-open-source
+   trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= obsidian-open-source:KP1UbL7OIibSjFo9/2tiHCYLm/gJMfy8Tim7+7P4o0I=
+   ```
+   (The new values are each appended at the end of a space-separated list.)
+
+2. After updating that file, you probably need to restart your Nix daemon:
+
+   - On macOS:
+     - `sudo launchctl stop org.nixos.nix-daemon`
+     - `sudo launchctl start org.nixos.nix-daemon`
+   - On Linux:
+     - `sudo systemctl stop nix-daemon`
+     - `sudo systemctl start nix-daemon`
+
+(On NixOS these tasks are done differently, consult the NixOS documentation for how to update your system configuration which includes these settings and will restart the daemon.)
+
+##### Building
+
 There is a separate tarball for each device.
 To build one, run:
 ```bash
@@ -102,7 +136,7 @@ For example, it might be `~/Downloads/release.tar.gz` if you downloaded a pre-bu
 
 #### Without Nix
 
-Without using Nix, the `ledgerctl` can be used directly to install the app with the following commands.
+Without using Nix, the [`ledgerctl`](https://github.com/LedgerHQ/ledgerctl) can be used directly to install the app with the following commands.
 For more information on how to install and use that tool see the [instructions from LedgerHQ](https://github.com/LedgerHQ/ledgerctl).
 
 ```bash
@@ -113,7 +147,7 @@ ledgerctl install -f app.json
 
 ## Using the app with generic CLI tool
 
-The bundled `generic-cli` tool can be used to obtaining the public key and do signing.
+The bundled [`generic-cli`](https://github.com/alamgu/alamgu-generic-cli) tool can be used to obtaining the public key and do signing.
 
 To use this tool using Nix, from the root level of this repo, run this command to enter a shell with all the tools you'll need:
 ```bash

diff --git a/crate-hashes.json b/crate-hashes.json
@@ -0,0 +1,11 @@
+{
+  "enum-init 0.1.0 (git+https://github.com/alamgu/enum-init#5706da92215f1a5816d704229f8924d95d954a77)": "0vh10l6glg4fry3z8am9skdlg2xcs0maqz5f24awwd9vl4ac6fv6",
+  "include_gif 0.1.0 (git+https://github.com/LedgerHQ/sdk_include_gif#699d28c6157518c4493899e2eeaa8af08346e5e7)": "185gra3i4s8cbmdhrxk2fyww0l6zlmyf2l04zbkvl9n9b6hd04pl",
+  "ledger-crypto-helpers 0.2.0 (git+https://github.com/alamgu/ledger-crypto-helpers?branch=dn-sdk-bump-fixes#ba04aa35372c2ebd62a98aeea42b8a8264775289)": "0gc0k1cwwi37vssrx0a3qgqwqm5v98nr58i03iydak5ypd2hp2dp",
+  "ledger-log 0.2.0 (git+https://github.com/alamgu/ledger-log#a417ebe6a426bbc6f92e92272c944b89765c384d)": "1qg502avfqiy30k17xi8gwwnawx5p53v0rcd3c566s4s5ga746jb",
+  "ledger-parser-combinators 0.1.0 (git+https://github.com/alamgu/ledger-parser-combinators#42e75ab7b8cd900889c553dab7908acd6cd8b9c9)": "0qvyf2xswi0k5fhbm1wln76j4ymd6wp407ywwy66gdp8sc9kybsr",
+  "ledger-prompts-ui 0.1.0 (git+https://github.com/alamgu/ledger-prompts-ui#87c29519a43b5144058c97eb298a5297b342e0be)": "0vvj15z52gn2gc4dlf1afqri4y95w592yhv7iimjhyv3hda863vv",
+  "nanos_sdk 0.2.0 (git+https://github.com/alamgu/ledger-nanos-sdk.git?branch=relocating-loader-w-fixes#27c4df41a062cfc5382ef733a7f26aff10d6c6f7)": "0zg82w7xj66caxw6zf1qslz5znfhpjl16ajjc1dn7f5m9h4lfg7c",
+  "nanos_ui 0.2.0 (git+https://github.com/obsidiansystems/ledger-nanos-ui?branch=dn-make-from-pub#7d68e90f7a35736a44b8ab9cb620ed4d6cbc4495)": "0qfp96v7ay9a9wy4slcw9fd7qcdml36vs7cycmyskg42mxm0yh6r",
+  "testmacro 0.1.0 (git+https://github.com/yhql/testmacro#29a16cb8718d61b67c6c391e60c8347c064e921e)": "0agl46ilx2psfvwvg8iwfmq203xapw435h03qskxqf5dnwhy1kir"
+}
diff --git a/default.nix b/default.nix
@@ -11,6 +11,7 @@ rec {
   app-nix = alamgu.crate2nix-tools.generatedCargoNix {
     name = "${appName}-nix";
     src = builtins.filterSource (p: _: p != toString "./rust-app/target") ./rust-app;
+    additionalCrateHashes = builtins.fromJSON (builtins.readFile ./crate-hashes.json);
   };
 
   makeApp = { rootFeatures ? [ "default" ], release ? true, device }:
@@ -71,6 +72,7 @@ rec {
       alamgu.cargo-ledger
       alamgu.ledgerRustPlatform.rust.cargo
     ];
+    strictDeps = true;
   } (alamgu.cargoLedgerPreHook + ''
 
     cp ${./rust-app/Cargo.toml} ./Cargo.toml
@@ -114,6 +116,7 @@ rec {
     nativeBuildInputs = [
       pkgs.wget alamgu.speculos.speculos testScript
     ];
+    strictDeps = true;
   } ''
     mkdir $out
     (
@@ -140,6 +143,7 @@ rec {
   makeStackCheck = { rootCrate, device, memLimit, variant ? "" }:
   pkgs.runCommandNoCC "stack-check-${device}${variant}" {
     nativeBuildInputs = [ alamgu.stack-sizes ];
+    strictDeps = true;
   } ''
     stack-sizes --mem-limit=${toString memLimit} ${rootCrate}/bin/${appName} ${rootCrate}/bin/*.o | tee $out
   '';
@@ -218,6 +222,21 @@ rec {
   nanosplus = appForDevice "nanosplus";
   nanox = appForDevice "nanox";
 
+  cargoFmtCheck = pkgs.stdenv.mkDerivation {
+    pname = "cargo-fmt-${appName}";
+    inherit (nanos.rootCrate) version src;
+    nativeBuildInputs = [
+      pkgs.alamguRustPackages.cargo
+      pkgs.alamguRustPackages.rustfmt
+    ];
+    buildPhase = ''
+      cargo fmt --all --check
+    '';
+    installPhase = ''
+      touch "$out"
+    '';
+  };
+
   inherit (pkgs.nodePackages) node2nix;
 
 }
diff --git a/dep/alamgu/github.json b/dep/alamgu/github.json
@@ -3,6 +3,6 @@
   "repo": "alamgu",
   "branch": "newer-firmware",
   "private": false,
-  "rev": "a243173385abc9d584918e594b567aadbd786f29",
-  "sha256": "1xgn1x64y1c3n9f90cpqla7k1ks5db2hf91mdqm0k8b0q1scdq3n"
+  "rev": "bc765884916fd120ec6b5759d885ff5bf0e5aa78",
+  "sha256": "1hsb1bdl2gksqwzm2q8qyv3rq5kh8x7hv44vyvn9kd7zsik0gdld"
 }
diff --git a/rust-app/src/test_parsers.rs b/rust-app/src/test_parsers.rs
@@ -1,12 +1,12 @@
 use crate::utils::*;
 use arrayvec::ArrayVec;
 use core::fmt::Write;
-use ledger_prompts_ui::ScrollerError;
 use ledger_parser_combinators::core_parsers::*;
 use ledger_parser_combinators::endianness::*;
 use ledger_parser_combinators::interp_parser::{
     DefaultInterp, InterpParser, MoveAction, SubInterp,
 };
+use ledger_prompts_ui::ScrollerError;
 
 // Try out all possible param types
 pub type TestParsersSchema = ((BytesParams, U16Params), (U64Params, DArrayParams));
@@ -39,7 +39,14 @@ const fn bytes_params_parser() -> BytesParamsT {
         mkmvfn(
             |(v1, v2): (Option<u8>, Option<[u8; 32]>), destination: &mut Option<()>| {
                 *destination = Some(());
-                scroller_paginated("Got Bytes", |w| Ok(write!(w, "v1: {}, v2: {:02x?}", v1.as_ref().ok_or(ScrollerError)?, v2.as_ref().ok_or(ScrollerError)?)?))
+                scroller_paginated("Got Bytes", |w| {
+                    Ok(write!(
+                        w,
+                        "v1: {}, v2: {:02x?}",
+                        v1.as_ref().ok_or(ScrollerError)?,
+                        v2.as_ref().ok_or(ScrollerError)?
+                    )?)
+                })
             },
         ),
     )
@@ -52,7 +59,14 @@ const fn u16_params_parser() -> U16ParamsT {
         mkmvfn(
             |(v1, v2): (Option<u16>, Option<u16>), destination: &mut Option<()>| {
                 *destination = Some(());
-                scroller("Got U16", |w| Ok(write!(w, "v1: {}, v2: {}", v1.as_ref().ok_or(ScrollerError)?, v2.as_ref().ok_or(ScrollerError)?)?))
+                scroller("Got U16", |w| {
+                    Ok(write!(
+                        w,
+                        "v1: {}, v2: {}",
+                        v1.as_ref().ok_or(ScrollerError)?,
+                        v2.as_ref().ok_or(ScrollerError)?
+                    )?)
+                })
             },
         ),
     )
@@ -65,7 +79,14 @@ const fn u32_params_parser() -> U32ParamsT {
         mkmvfn(
             |(v1, v2): (Option<u32>, Option<u32>), destination: &mut Option<()>| {
                 *destination = Some(());
-                scroller("Got U32", |w| Ok(write!(w, "v1: {}, v2: {}", v1.as_ref().ok_or(ScrollerError)?, v2.as_ref().ok_or(ScrollerError)?)?))
+                scroller("Got U32", |w| {
+                    Ok(write!(
+                        w,
+                        "v1: {}, v2: {}",
+                        v1.as_ref().ok_or(ScrollerError)?,
+                        v2.as_ref().ok_or(ScrollerError)?
+                    )?)
+                })
             },
         ),
     )
@@ -78,7 +99,14 @@ const fn u64_params_parser() -> U64ParamsT {
         mkmvfn(
             |(v1, v2): (Option<u64>, Option<u64>), destination: &mut Option<()>| {
                 *destination = Some(());
-                scroller_paginated("Got U64", |w| Ok(write!(w, "v1: {}, v2: {}", v1.as_ref().ok_or(ScrollerError)?, v2.as_ref().ok_or(ScrollerError)?)?))
+                scroller_paginated("Got U64", |w| {
+                    Ok(write!(
+                        w,
+                        "v1: {}, v2: {}",
+                        v1.as_ref().ok_or(ScrollerError)?,
+                        v2.as_ref().ok_or(ScrollerError)?
+                    )?)
+                })
             },
         ),
     )
@@ -92,7 +120,9 @@ const fn darray_params_parser() -> DArrayParamsT {
             |(v1, _v2): (Option<ArrayVec<u8, 24>>, Option<ArrayVec<(), 4>>),
              destination: &mut Option<()>| {
                 *destination = Some(());
-                scroller("Got Darray", |w| Ok(write!(w, "v1: {:02x?}", v1.as_ref().ok_or(ScrollerError)?)?))
+                scroller("Got Darray", |w| {
+                    Ok(write!(w, "v1: {:02x?}", v1.as_ref().ok_or(ScrollerError)?)?)
+                })
             },
         ),
     )

diff --git a/update-crate-hashes.sh b/update-crate-hashes.sh
@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+
+cd -- "$(dirname -- "${BASH_SOURCE[0]}")"
+
+crate2nix="$(nix-build --no-out-link -A alamgu.crate2nix)/bin/crate2nix"
+nix-shell -A nanosplus.rustShell --command "$crate2nix generate -f rust-app/Cargo.toml"
+rm Cargo.nix