Add optee-utee-build crate

[ivila]

Reason

There are some reasons that makes us struggle with current build process:

1) ta_static.rs

This file was copy again and again, and it is meaningless for the TA, and TA need to write some configurations consts with specific names to combine with it, developers keep repeating themselves and make some new comers confusing, especially the EXT_PROP_VALUE_1 and EXT_PROP_VALUE_2.

2) the linking process

There are bunch of codes copy again and again, some of the new comers confuse about the codes in the build.rs, why they are here, what did they do, what is the ta.lds, etc.

3) the upgrade process

Sometimes we need to update the codes, for example:

1. update build and link script as you do: examples: polish linking script
2. update #[no_mangle] to #[unsafe(no_mangle)] as rust 2024 required
3. add some extra properties

Every time we do so, we need to update every crates in our repo, and must provide a detail description letting developers know which line to change, however developers expect they should just upgrade the version rather than modify the ta_static.rs, build.rs, main.rs again and again.

Proposal

We should add a optee-utee-build crate, this crate can:

1. provide a TAConfig struct and generate user_ta_header.rs automatically: just like prost-build for prost, developers could just include the generated file in their src/main.rs, save them from ta.static.rs and configurations consts with specific names.
2. handle linking automatically: generate ta.lds and link to it automatically, with other linking process.
3. provide a easy way for upgrade: every time we need to change something about the building process, we just upgrade this crate.

By using this crate, the build.rs in hello-world-rs change to:

// new build.rs
use proto;
use optee_utee_build::{TAConfig, RustEdition, Error};

fn main() -> Result<(), Error> {
    let config = TAConfig::new_standard("0.1", "This is a hello world example.", "Hello World TA");
    optee_utee_build::build(RustEdition::Before2024, proto::UUID, config)

}

And for reference, current codes:

incubator-teaclave-trustzone-sdk/examples/hello_world-rs/ta/build.rs

Lines 18 to 103 in 8bb315f

	use proto;
	use std::env;
	use std::fs::File;
	use std::io::{BufRead, BufReader, Write};
	use std::path::{Path, PathBuf};
	use uuid::Uuid;
	fn main() -> std::io::Result<()> {
	let out = &PathBuf::from(env::var_os("OUT_DIR").unwrap());
	let mut buffer = File::create(out.join("user_ta_header.rs"))?;
	buffer.write_all(include_bytes!("ta_static.rs"))?;
	let tee_uuid = Uuid::parse_str(proto::UUID).unwrap();
	let (time_low, time_mid, time_hi_and_version, clock_seq_and_node) = tee_uuid.as_fields();
	write!(buffer, "\n")?;
	write!(
	buffer,
	"const TA_UUID: optee_utee_sys::TEE_UUID = optee_utee_sys::TEE_UUID {{
	timeLow: {:#x},
	timeMid: {:#x},
	timeHiAndVersion: {:#x},
	clockSeqAndNode: {:#x?},
	}};",
	time_low, time_mid, time_hi_and_version, clock_seq_and_node
	)?;
	let mut aarch64_flag = true;
	match env::var("TARGET_TA") {
	Ok(ref v) if v == "arm-unknown-linux-gnueabihf" || v == "arm-unknown-optee" => {
	println!("cargo:rustc-link-arg=--no-warn-mismatch");
	aarch64_flag = false;
	},
	_ => {}
	};
	let optee_os_dir = env::var("TA_DEV_KIT_DIR").unwrap();
	let search_path = Path::new(&optee_os_dir).join("lib");
	let optee_os_path = &PathBuf::from(optee_os_dir.clone());
	let mut ta_lds = File::create(out.join("ta.lds"))?;
	let f = File::open(optee_os_path.join("src/ta.ld.S"))?;
	let f = BufReader::new(f);
	for line in f.lines() {
	let l = line?;
	if aarch64_flag {
	if l.starts_with('#') ||
	l == "OUTPUT_FORMAT(\"elf32-littlearm\")" ||
	l == "OUTPUT_ARCH(arm)" {
	continue;
	}
	} else {
	if l.starts_with('#') ||
	l == "OUTPUT_FORMAT(\"elf64-littleaarch64\")" ||
	l == "OUTPUT_ARCH(aarch64)" {
	continue;
	}
	}
	if l == "\t. = ALIGN(4096);" {
	write!(ta_lds, "\t. = ALIGN(65536);\n")?;
	} else {
	write!(ta_lds, "{}\n", l)?;
	}
	}
	println!("cargo:rustc-link-search={}", out.display());
	println!("cargo:rerun-if-changed=ta.lds");
	println!("cargo:rustc-link-search={}", search_path.display());
	println!("cargo:rustc-link-lib=static=utee");
	println!("cargo:rustc-link-lib=static=utils");
	println!("cargo:rustc-link-arg=-Tta.lds");
	println!("cargo:rustc-link-arg=-e__ta_entry");
	println!("cargo:rustc-link-arg=-pie");
	println!("cargo:rustc-link-arg=-Os");
	println!("cargo:rustc-link-arg=--sort-section=alignment");
	let mut dyn_list = File::create(out.join("dyn_list"))?;
	write!(dyn_list, "{{ __elf_phdr_info; trace_ext_prefix; trace_level; ta_head; }};\n")?;
	println!("cargo:rustc-link-arg=--dynamic-list=dyn_list");
	Ok(())
	}

And people can remove the configuration consts:

/* TA configurations was removed, and should set by TAConfig in build.rs
// TA configurations
const TA_FLAGS: u32 = 0;
const TA_DATA_SIZE: u32 = 32 * 1024;
const TA_STACK_SIZE: u32 = 2 * 1024;
const TA_VERSION: &[u8] = b"0.1\0";
const TA_DESCRIPTION: &[u8] = b"This is a hello world example.\0";
const EXT_PROP_VALUE_1: &[u8] = b"Hello World TA\0";
const EXT_PROP_VALUE_2: u32 = 0x0010;
const TRACE_LEVEL: i32 = 4;
const TRACE_EXT_PREFIX: &[u8] = b"TA\0";
const TA_FRAMEWORK_STACK_SIZE: u32 = 2048;
*/

include!(concat!(env!("OUT_DIR"), "/user_ta_header.rs"));  // this keeps

Demo

Please check the optee-utee-build branch in my fork. You can also view the changes by this link
All the pipeline passed, check this
The core changes are:

1. add optee-utee-crate

1. Add TAConfig struct, people use this to set the configuration of TA.

#[derive(Debug, Clone)]
pub struct TAConfig {
    pub ta_flags: u32,
    pub ta_data_size: u32,
    pub ta_stack_size: u32,
    pub ta_version: String,
    pub ta_description: String,
    pub trace_level: i32,
    pub trace_ext_prefix: String,
    pub ta_framework_stack_size: u32,
    pub ext_properties: Vec<Property>,
}

2. Add Config struct, developers can use it to set the configuration of building process.

pub struct Config {
    out_dir: Option<PathBuf>,
    edition: RustEdition,
    header_file_name: Option<String>,
    ta_config: TAConfig,
}

3. Add RustEdition enum, just like RustEdition in bindgen, we need this for code generation.

2. use optee-utee-crate to build the hello world example

1. remove ta_static.rs
2. add optee-utee-build as build-dependencies and removed uuid from build-dependencies
3. in build.rs, use optee-utee-build instead of custom scripts.
4. in Makefile, use gcc as linker to fix the problem when building on ARM host (just like this issue, I didn't realize in the examples we use ld.bfd as linker😂In my team we always use gcc as default linker).

3. fix pipeline

change pipeline of OPTEE-repo-build-and-run-examples-32bit-TAs and OPTEE-repo-build-and-run-examples-32bit-TAs, set the version of manifest to 4.4.0, as they update the qemu_v8.yml 3 days ago, makes the pipeline failed.
I have tried to fix the pipeline when using latest qemu_v8.yml, but seems it need to:

1. add tomli dependency for python3: this can be done by python3 -m pip install tomli
2. upgrade libc from 2.64.6 to >=2.66.0: I think we need to upgrade the docker image, this cannot be done by myself, so I downgrade the version of qemu_v8.yml instead.
   

[DemesneGH]

I agree with this proposal, please feel free to open the PR. Since it affects all examples, just modifying one example is okay for the first review stage (as what you do in your repo), thanks!

[mssun]

Thanks for bring this up. This is really a good idea!

[b49020]

Thanks @ivila for proposing this, it's really nice improvement. It was one of the things I had in my ToDo list after working on no-std but never got to it. Please feel free to propose a PR.

[ivila]

PR created.