Preparing 0.2.1 release

- Update documentation - Update translations - Minor optimizations - Speed up CI/CD local builds
rimerosolutions · Jul 8, 2022 · ea2acc0 · ea2acc0
1 parent 46996aa
commit ea2acc0
Show file tree

Hide file tree

Showing 17 changed files with 73 additions and 124 deletions.
diff --git a/README.org b/README.org
@@ -7,6 +7,7 @@
 =Entrusted= is a document sanitizer tool that converts "/potentially suspicious files/" into /safe PDFs/:
 - This is achieved by transforming documents inside a "lightweight sandbox" ([[https://www.ibm.com/cloud/learn/containerization][containerization]]): =containerization= itself does not provide absolute security guarantees...
 - Supported file formats include PDF, common image formats, Office documents ([[https://www.office.com/][Microsoft Office]] or [[https://www.libreoffice.org/discover/what-is-opendocument/][OpenDocument]]). For Office documents, supported is limited to text documents, spreadsheets and presentations.
+- The progam was originally a [[https://www.rust-lang.org/][Rust]] port of [[https://dangerzone.rocks/][Dangerzone]], for experimentation purposes
 
 *NOTE*: Unless you use the live CD, you MUST install either [[https://www.docker.com/products/docker-desktop/][Docker]] or [[https://podman.io/getting-started/][Podman]] prior running the program.
 
@@ -18,8 +19,7 @@ Below are some screenshots of the graphical Desktop client running on Mac OS.
 
 ** Features
 
-This application is a [[https://www.rust-lang.org/][Rust]] port of [[https://dangerzone.rocks/][Dangerzone]], originally for experimentation purposes (/not an exact port/).
-- Similarly to the original /Dangerzone/ tool, documents or images are converted inside a "sandbox"
+- Documents or images are converted inside a "sandbox"
 - Documents can be converted in batch (sequential conversions for now)
 - There is support for password-protected documents (known Office document formats and PDF files)
 - An optional Web interface is available and can act as an "online service" (towards multi-tenancy)
@@ -28,15 +28,15 @@ This application is a [[https://www.rust-lang.org/][Rust]] port of [[https://dan
   - Pre-installed [[https://xebia.com/blog/podman-the-free-container-engine-alternative-to-docker/][container solution]], with no required user configuration ([[./entrusted_webserver][entrusted_webserver]])
   - Extra isolation layer with a virtual machine
 - The application supports multiple languages (English and French for now)
-- On Mac OS, [[https://github.com/lima-vm/lima][Lima]] is supported as an alternative to [[https://www.docker.com/products/docker-desktop/][Docker]]
+- On Mac OS, [[https://github.com/lima-vm/lima][Lima]] is supported as an alternative to [[https://www.docker.com/products/docker-desktop/][Docker]] (default /Lima/ instance)
 
 * What is available?
 
 ** User interfaces
 
 There are 3 user interfaces ([[./entrusted_client][Desktop and CLI]], [[./entrusted_webserver][Web]])
 - Most users will likely want the Desktop version
-- If you prefer the Web interface, please download the live CD ISO for an "out of the box non-involved user experience" (/unless you'd rather setup the Web server yourself/):
+- If you prefer the Web interface, please download the live CD for an out-of-the-box user experience:
   - You can run the live CD with tools such as [[https://www.qemu.org/][QEMU]], [[https://www.virtualbox.org/wiki/Downloads][VirtualBox]], [[https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v][Hyper-V]], [[https://www.vmware.com/nl/products/workstation-player.html][VmWare]], [[https://www.parallels.com/][Parallels]], etc.
   - The Web UI is available at =http://localhost:13000=
   - Replace =localhost= by the IP address of your virtual machine in the above-mentioned link
@@ -56,7 +56,7 @@ For downloads, please visit the [[https://github.com/rimerosolutions/entrusted/r
 | =MS Windows=     | =entrusted-windows-amd64-<version>.exe= | Installer with the Desktop and CLI clients |
 |                  | =entrusted-windows-amd64-<version>.zip= | Archive with all the programs              |
 |------------------+-----------------------------------------+--------------------------------------------|
-| =Live CD=        | =entrusted-livecd-amd64-<version>.iso=  | Web interface deployed in VM               |
+| =Live CD=        | =entrusted-livecd-amd64-<version>.iso=  | Web interface Live CD                      |
 |------------------+-----------------------------------------+--------------------------------------------|
 
 ** Key software components
@@ -71,7 +71,7 @@ Most users will interact with the =entrusted_client=, via its Desktop user inter
 | [[./entrusted_container][entrusted_container]] | Document processing "sandbox"               |
 | [[./entrusted_client][entrusted_client]]    | Command-line and graphical user interfaces  |
 | [[./entrusted_webserver][entrusted_webserver]] | Online service feature with a Web interface |
-| [[./entrusted_webclient][entrusted_webclient]] | Command-line client for the =webserver=     |
+| [[./entrusted_webclient][entrusted_webclient]] | Command-line client for the =web server=    |
 | [[./entrusted_l10n][entrusted_l10n]]      | Shared library for translation features     |
 |---------------------+---------------------------------------------|
 
@@ -88,9 +88,9 @@ Below is a high-level architecture overview, the solution relies on a container
   - Mostly Linux has been tested
   - BSD users can probably use the "Online service" from the Live CD ISO (via VirtualBox, etc.)
 - *Container Runtime*: [[https://podman.io/][Podman]], [[https://www.docker.com/][Docker]] or [[https://github.com/lima-vm/lima][Lima]]
-  - On Linux, please install Podman
-  - On Windows, Docker is recommended
+  - On Linux, please install Podman or Docker
   - On Mac OS, please use either Docker or Lima
+  - On Windows, Docker is recommended
 
 Conversions will fail if the container solution is not running or available in your [[https://www.java.com/en/download/help/path.html]["PATH settings"]]:
   - The tool doesn't attempt to modify software on your machine (install or change other programs)

diff --git a/RELEASING.org b/RELEASING.org
@@ -2,20 +2,21 @@
 
 General steps:
 
-- Review and update the README page for impacted components as needed
-- Bump the version of the application  
+- Review and update the README page for impacted components (/if needed/)
+- Bump the version of the application at several places for now... 
   - Bump the =version= field in all the projects (=Cargo.toml= files)
     - The container image tag is inferred from the release version to standardize as much as possible
     - Please try to follow closely [[https://semver.org/][semantic versioning]]
   - Bump the =version= field in the /Dockerfile/ for the label =org.opencontainers.image.version=
+  - Bump the version in the HTML title for =entrusted_webserver/web-assets/index.html=
 - On a recent Ubuntu Linux installation, build all the release artifacts =ci_cd/build_all.sh=
-  - This generates all the release artifacts under a new folder called =packages=
+  - This generates release artifacts under a new folder called =packages=
   - You can then test the resulting archives and executables under Linux, Mac OS and Windows
   - You can test the live CD ISO image with a tool such as [[https://www.qemu.org/][QEMU]] or the software of your choice
 - Merge the current branch into the =main= branch, and push the changes
 - Tag the main branch with the new =release version= (i.e. =1.1.1=) and push the changes
 - Push the container image to [[https://hub.docker.com/r/uycyjnzgntrn/entrusted_container][Docker Hub]] per [[./entrusted_container/README.org][publishing instructions]]
-- Create a new release from the GitHub Web interface:
+- Create a new GitHub release from its Web interface:
   - Add the release notes
   - Upload all release artifacts (=exe=, =dmg=, =zip=, =tar= and =iso= files)
   - Publish the release
diff --git a/TODO.org b/TODO.org
@@ -3,30 +3,30 @@
 * Next steps
 
 - [-] Code maintenance
+  - [X] Eliminate shell commands inside the container image
   - [ ] Add automated tests (unit & integration)
   - [ ] Code cleanup and refactoring
   - [ ] More idiomatic Rust code
   - [ ] Simplify and clarify the code
-  - [X] Eliminate shell commands inside the container image
 - [ ] Online Web Service
   - [ ] Serve Web UI translations at once, instead of a second request on page load
   - [ ] Use the =Content-Language= header in response (translations)
   - [ ] Revisit notifications SSE + HTTP2, for the "Online Service"
 - [ ] Investigate if specific support is required for Tails Linux
 - [-] CI/CD
-  - [ ] Transition CI/CD scripts to Github actions or TravisCI or something else?
   - [X] Adopt Git flow and delete all the previous spiral branches
+  - [ ] Transition CI/CD scripts to Github actions or TravisCI or something else?
   - [ ] Create few specific packages (DEB, RPM)
 - [ ] Hardening
   - [ ] Review container execution options (podman v.s podman-remote, security options, etc.)
   - [ ] Revisit Flatpak support requirements and =flatpak-spawn=
   - [ ] Review some of the code and read few security-related articles
 - [-] Improved conversions
-  - [ ] Parallel conversions
-  - [ ] Minimal resources management to avoid exhausting computer CPU and memory
-  - [ ] Cancellable conversions for those that haven't started yet
   - [X] Custom output location for individual documents in the conversion queue
   - [X] Support for password-protected documents
+  - [ ] Cancellable conversions for those that haven't started yet
+  - [ ] Parallel conversions
+  - [ ] Minimal resources management to avoid exhausting computer CPU and memory
 - [ ] Performance
   - [ ] Mime-type detection
     - [ ] Avoid reading entire files for mime type detection, just what is required...

diff --git a/ci_cd/linux/build.sh b/ci_cd/linux/build.sh
@@ -29,34 +29,18 @@ if [ $retVal -ne 0 ]; then
   exit 1
 fi
 
-echo "Building entrusted_client (entrusted-cli)"
+echo "Building other Linux binaries"
 cd ${PROJECTDIR}
-podman run --rm --volume "${PWD}":/root/src --workdir /root/src docker.io/joseluisq/rust-linux-darwin-builder:1.60.0 sh -c "RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-musl --manifest-path /root/src/entrusted_client/Cargo.toml --bin entrusted-cli"
-retVal=$?
-if [ $retVal -ne 0 ]; then
-	echo "Failure"
-  exit 1
-fi
-cp ${PROJECTDIR}/entrusted_client/target/x86_64-unknown-linux-musl/release/entrusted-cli ${ARTIFACTSDIR}
+podman run --rm --volume "${PWD}":/root/src --workdir /root/src docker.io/joseluisq/rust-linux-darwin-builder:1.60.0 sh -c "RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-musl --manifest-path /root/src/entrusted_client/Cargo.toml --bin entrusted-cli && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-musl --manifest-path /root/src/entrusted_webserver/Cargo.toml && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-musl --manifest-path /root/src/entrusted_webclient/Cargo.toml"
 
-echo "Building entrusted_webserver"
-cd ${PROJECTDIR}
-podman run --rm --volume "${PWD}":/root/src --workdir /root/src docker.io/joseluisq/rust-linux-darwin-builder:1.60.0 sh -c "RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-musl --manifest-path /root/src/entrusted_webserver/Cargo.toml"
 retVal=$?
 if [ $retVal -ne 0 ]; then
 	echo "Failure"
   exit 1
 fi
-cp ${PROJECTDIR}/entrusted_webserver/target/x86_64-unknown-linux-musl/release/entrusted-webserver ${ARTIFACTSDIR}
 
-echo "Building entrusted_webclient"
-cd ${PROJECTDIR}
-podman run --rm --volume "${PWD}":/root/src --workdir /root/src docker.io/joseluisq/rust-linux-darwin-builder:1.60.0 sh -c "RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-musl --manifest-path /root/src/entrusted_webclient/Cargo.toml"
-retVal=$?
-if [ $retVal -ne 0 ]; then
-	echo "Failure"
-  exit 1
-fi
+cp ${PROJECTDIR}/entrusted_client/target/x86_64-unknown-linux-musl/release/entrusted-cli ${ARTIFACTSDIR}
+cp ${PROJECTDIR}/entrusted_webserver/target/x86_64-unknown-linux-musl/release/entrusted-webserver ${ARTIFACTSDIR}
 cp ${PROJECTDIR}/entrusted_webclient/target/x86_64-unknown-linux-musl/release/entrusted-webclient ${ARTIFACTSDIR}
 
 cp ${SCRIPTDIR}/release_README.txt ${ARTIFACTSDIR}/README.txt

diff --git a/ci_cd/macos/build.sh b/ci_cd/macos/build.sh
@@ -16,44 +16,21 @@ rm -rf ${PROJECTDIR}/entrusted_webserver/target
 
 cd ${PROJECTDIR}
 
-echo "Building entrusted_client"
+echo "Building all Mac OS binaries"
 podman run --rm \
     --volume "${PROJECTDIR}":/root/src \
     --workdir /root/src \
     docker.io/joseluisq/rust-linux-darwin-builder:1.60.0 \
-    sh -c "export CC=/usr/local/osxcross/target/bin/o64-clang; export CXX=/usr/local/osxcross/target/bin/o64-clang++; cd /root/src/entrusted_client && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-apple-darwin"
+    sh -c "export CC=/usr/local/osxcross/target/bin/o64-clang; export CXX=/usr/local/osxcross/target/bin/o64-clang++; cd /root/src/entrusted_client && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-apple-darwin; cd /root/src/entrusted_webclient && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-apple-darwin; cd /root/src/entrusted_webserver && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-apple-darwin"
 retVal=$?
 if [ $retVal -ne 0 ]; then
 	echo "Failure"
   exit 1
 fi
+
 cp ${PROJECTDIR}/entrusted_client/target/x86_64-apple-darwin/release/entrusted-cli ${ARTIFACTSDIR}
 cp ${PROJECTDIR}/entrusted_client/target/x86_64-apple-darwin/release/entrusted-gui ${ARTIFACTSDIR}
-
-echo "Building entrusted_webclient"
-podman run --rm \
-    --volume "${PROJECTDIR}":/root/src \
-    --workdir /root/src \
-    docker.io/joseluisq/rust-linux-darwin-builder:1.60.0 \
-    sh -c "export CC=/usr/local/osxcross/target/bin/o64-clang; export CXX=/usr/local/osxcross/target/bin/o64-clang++; cd /root/src/entrusted_webclient && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-apple-darwin"
-retVal=$?
-if [ $retVal -ne 0 ]; then
-	echo "Failure"
-  exit 1
-fi
 cp ${PROJECTDIR}/entrusted_webclient/target/x86_64-apple-darwin/release/entrusted-webclient ${ARTIFACTSDIR}
-
-echo "Building entrusted_webserver"
-podman run --rm \
-    --volume "${PROJECTDIR}":/root/src \
-    --workdir /root/src \
-    docker.io/joseluisq/rust-linux-darwin-builder:1.60.0 \
-    sh -c "export CC=/usr/local/osxcross/target/bin/o64-clang; export CXX=/usr/local/osxcross/target/bin/o64-clang++; cd /root/src/entrusted_webserver && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-apple-darwin"
-retVal=$?
-if [ $retVal -ne 0 ]; then
-	echo "Failure"
-  exit 1
-fi
 cp ${PROJECTDIR}/entrusted_webserver/target/x86_64-apple-darwin/release/entrusted-webserver ${ARTIFACTSDIR}
 
 

diff --git a/ci_cd/windows/build.sh b/ci_cd/windows/build.sh
@@ -18,33 +18,17 @@ rm -rf ${PROJECTDIR}/entrusted_webserver/target
 
 cd ${PROJECTDIR}
 
-echo "Building entrusted_client"
-podman run --rm --privileged -v "${PROJECTDIR}":/src docker.io/uycyjnzgntrn/rust-windows:1.60.0 sh -c "cd /src/entrusted_client && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-pc-windows-gnu"
+echo "Building all Windows binaries"
+podman run --rm --privileged -v "${PROJECTDIR}":/src docker.io/uycyjnzgntrn/rust-windows:1.60.0 sh -c "cd /src/entrusted_client && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-pc-windows-gnu && cd /src/entrusted_webserver && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-pc-windows-gnu && cd /src/entrusted_webclient && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-pc-windows-gnu"
 retVal=$?
 if [ $retVal -ne 0 ]; then
 	echo "Failure"
   exit 1
 fi
+
 cp ${PROJECTDIR}/entrusted_client/target/x86_64-pc-windows-gnu/release/entrusted-cli.exe ${ARTIFACTSDIR}/
 cp ${PROJECTDIR}/entrusted_client/target/x86_64-pc-windows-gnu/release/entrusted-gui.exe ${ARTIFACTSDIR}/
-
-
-echo "Building entrusted_webserver"
-podman run --rm  --privileged -v "${PROJECTDIR}":/src docker.io/uycyjnzgntrn/rust-windows:1.60.0 sh -c "cd /src/entrusted_webserver && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-pc-windows-gnu"
-retVal=$?
-if [ $retVal -ne 0 ]; then
-	echo "Failure"
-  exit 1
-fi
 cp ${PROJECTDIR}/entrusted_webserver/target/x86_64-pc-windows-gnu/release/entrusted-webserver.exe ${ARTIFACTSDIR}/
-
-echo "Building entrusted_webclient"
-podman run --rm --privileged -v "${PROJECTDIR}":/src docker.io/uycyjnzgntrn/rust-windows:1.60.0 sh -c "cd /src/entrusted_webclient && RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-pc-windows-gnu"
-retVal=$?
-if [ $retVal -ne 0 ]; then
-	echo "Failure"
-  exit 1
-fi
 cp ${PROJECTDIR}/entrusted_webclient/target/x86_64-pc-windows-gnu/release/entrusted-webclient.exe ${ARTIFACTSDIR}/
 
 echo "Generate windows installer"

diff --git a/entrusted_client/src/common.rs b/entrusted_client/src/common.rs
@@ -5,6 +5,7 @@ use serde::{Deserialize, Serialize};
 
 pub const CONTAINER_IMAGE_EXE: &str = "/usr/local/bin/entrusted-container";
 pub const ENV_VAR_ENTRUSTED_DOC_PASSWD: &str = "ENTRUSTED_DOC_PASSWD";
+pub const LOG_FORMAT_JSON: &str = "json";
 
 #[macro_export]
 macro_rules! incl_gettext_files {
@@ -97,9 +98,10 @@ pub fn container_runtime_path<'a>() -> Option<ContainerProgram<'a>> {
             ContainerProgramStub::Podman(cmd, sub_cmd_args, cmd_args, tmp_dir_opt) |
             ContainerProgramStub::Lima(cmd, sub_cmd_args, cmd_args, tmp_dir_opt) => {
                 if let Some(path_container_exe) = executable_find(cmd) {
-                    let suggested_tmp_dir = match tmp_dir_opt {
-                        None => None,
-                        Some(tmp_dir) => Some(PathBuf::from(tmp_dir))
+                    let suggested_tmp_dir = if let Some(tmp_dir) = tmp_dir_opt {
+                        Some(PathBuf::from(tmp_dir))
+                    } else {
+                        None
                     };
                     return Some(ContainerProgram::new(path_container_exe, sub_cmd_args.clone(), cmd_args.clone(), suggested_tmp_dir));
                 }
@@ -115,7 +117,7 @@ pub fn default_output_path(input: PathBuf, file_suffix: String) -> Result<PathBu
     let output_filename_opt = input.parent().map(|i| i.to_path_buf());
 
     if let (Some(input_name), Some(mut output_filename)) = (input_name_opt, output_filename_opt) {
-        let filename = format!("{}-{}.pdf", input_name.to_owned(), file_suffix);
+        let filename = format!("{}-{}.pdf", &input_name, &file_suffix);
         output_filename.push(filename);
         Ok(output_filename)
     } else {

diff --git a/entrusted_client/src/container.rs b/entrusted_client/src/container.rs
@@ -79,7 +79,7 @@ fn exec_crt_command (container_program: common::ContainerProgram, args: Vec<&str
             // mitigate apparent 139 exit codes that don't happen with Debian bullseye
             // Sadly it's a much bigger image than with Alpine so we decide to mitigate the issue
             // Apparently a vsyscall=emulate argument needs to be added to /proc/cmdline depending on the kernel version
-            // Essentially not mitigating the issue would be a problem in a non-controlled environment (i.e. Web Server running outside the Live CD ISO)
+            // Essentially not mitigating the issue would be a problem in a non-controlled environment (i.e. Live CD ISO)
             if let Some(exit_code) = exit_status.code() {
                 if exit_code == 139 {
                     return Ok(());
@@ -138,9 +138,9 @@ pub fn convert(input_path: PathBuf, output_path: PathBuf, convert_options: commo
     }
 
     let printer: Box<dyn LogPrinter> = if convert_options.log_format == "plain".to_string() {
-        Box::new(PlainLogPrinter {})
+        Box::new(PlainLogPrinter)
     } else {
-        Box::new(JsonLogPrinter {})
+        Box::new(JsonLogPrinter)
     };
 
     tx.send(printer.print(1, format!("{} {}.", trans.gettext("Converting"), input_path.display())))?;
@@ -305,8 +305,8 @@ pub fn convert(input_path: PathBuf, output_path: PathBuf, convert_options: commo
             err_msg.push_str(&trans.gettext("Please install Docker."));
         } else if cfg!(any(target_os="macos")) {
             err_msg.push_str(&trans.gettext("Please install Docker or Lima."));
-        } else {
-            err_msg.push_str(&trans.gettext("Please install Podman."));
+        } else { // Linux and others
+            err_msg.push_str(&trans.gettext("Please install Docker or Podman."));
         }
     }