Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simplify build and install for CCF #468

Merged
merged 3 commits into from
Feb 15, 2024
Merged

Simplify build and install for CCF #468

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c9cf579
Simplify build and install for CCF
cmickeyb Feb 14, 2024
6b1d6cd
Copy the member keys from the xfer directory
cmickeyb Feb 14, 2024
27ea18b
Small fixes to address PR feedback
cmickeyb Feb 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docker/tools/build_ccf.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,8 @@ source /project/pdo/tools/environment.sh
export PDO_HOSTNAME=
export PDO_LEDGER_URL=

# -----------------------------------------------------------------
# set up the basic structure in ${PDO_INSTALL_ROOT}
# -----------------------------------------------------------------
make -C ${PDO_SOURCE_ROOT}/ledgers/ccf environment
make -C ${PDO_SOURCE_ROOT}/ledgers/ccf install
9 changes: 8 additions & 1 deletion docker/tools/environment.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ else
export PDO_DEFAULT_SIGCURVE=SECP256K1
fi

export CCF_BASE=/opt/ccf_virtual
export XFER_DIR=${XFER_DIR:-/project/pdo/xfer}

# if the container is running HW mode, then we will grab the
Expand All @@ -62,3 +61,11 @@ export PDO_ENCLAVE_CODE_SIGN_PEM=${PDO_SGX_KEY_ROOT}/enclave_code_sign.pem
# they are not used at build or run time
export PDO_SPID="$(cat ${PDO_SGX_KEY_ROOT}/sgx_spid.txt)"
export PDO_SPID_API_KEY="$(cat ${PDO_SGX_KEY_ROOT}/sgx_spid_api_key.txt)"

# set up the ccf directories, ccf_base is where the ccf
# core is installed, ccf_pdo_dir is where the pdo tp
# components will be installed, and ccf_ledger_dir is
# where the ccf python virtual environment will be built
export CCF_BASE=/opt/ccf_virtual
export CCF_PDO_DIR=${PDO_INSTALL_ROOT}
export CCF_LEDGER_DIR=${PDO_HOME}/ccf
7 changes: 4 additions & 3 deletions docker/tools/run_ccf_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,10 @@ mkdir -p ${PDO_LEDGER_KEY_ROOT}
# -----------------------------------------------------------------
yell start the ccf service
# -----------------------------------------------------------------
. ${PDO_HOME}/ccf/bin/activate
yell ${PDO_HOME}/ccf/bin/start_ccf_network.sh -i ${PDO_LEDGER_ADDRESS}
try ${PDO_HOME}/ccf/bin/start_ccf_network.sh -i ${PDO_LEDGER_ADDRESS}
yell ${CCF_LEDGER_DIR}/bin/start_ccf_network.sh -i ${PDO_LEDGER_ADDRESS} \
--pdo-dir ${CCF_PDO_DIR} --ledger-dir ${CCF_LEDGER_DIR}
try ${CCF_LEDGER_DIR}/bin/start_ccf_network.sh -i ${PDO_LEDGER_ADDRESS} \
--pdo-dir ${CCF_PDO_DIR} --ledger-dir ${CCF_LEDGER_DIR}

# -----------------------------------------------------------------
yell copy the ledger keys
Expand Down
7 changes: 7 additions & 0 deletions docker/tools/run_services_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,13 @@ yell check for registration
# -----------------------------------------------------------------
# this probably requires additional CCF keys, need to test this
if [ "$SGX_MODE" == "HW" ]; then
if [ ! -f ${XFER}/ccf/keys/memberccf_privk.pem ] ; then
die unable to locate CCF policies keys
fi

try cp ${XFER_DIR}/ccf/keys/memberccf_cert.pem ${PDO_LEDGER_KEY_ROOT}/
try cp ${XFER_DIR}/ccf/keys/memberccf_privk.pem ${PDO_LEDGER_KEY_ROOT}/
Comment on lines +63 to +64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, these should make PDO_LEDGER_KEY_ROOT the authoritative place where to look for the ledger keys -- to be double checked with HW mode tests.

Copy link
Contributor Author

@cmickeyb cmickeyb Feb 15, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

umm... it already was (and it is pretty well documented):

PDO_LEDGER_KEY_ROOT is the root directory where the system keys are stored for ledger integration; files in this directory are not automatically generated. When ccf is used as ledger, the ccf network cert {networkcert.pem} must be placed under this folder. These keys get generated during ccf deployment.

and:

xfer -- this directory is used to pass configuration information and keys between the container and the host; for example, to push a previously built configuration into the container, put the files in the appropriate subdirectory in xfer.


try make -C ${PDO_SOURCE_ROOT}/build register
fi

Expand Down
6 changes: 4 additions & 2 deletions docker/tools/start_ccf.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,9 +79,11 @@ mkdir -p ${PDO_LEDGER_KEY_ROOT}
# -----------------------------------------------------------------
say start the ccf network
# -----------------------------------------------------------------
. ${PDO_HOME}/ccf/bin/activate
if [ ${F_NETWORK_MODE} == "start" ] ; then
try ${PDO_HOME}/ccf/bin/start_ccf_network.sh -i ${PDO_LEDGER_ADDRESS}
yell ${CCF_LEDGER_DIR}/bin/start_ccf_network.sh -i ${PDO_LEDGER_ADDRESS} \
--pdo-dir ${CCF_PDO_DIR} --ledger-dir ${CCF_LEDGER_DIR}
try ${CCF_LEDGER_DIR}/bin/start_ccf_network.sh -i ${PDO_LEDGER_ADDRESS} \
--pdo-dir ${CCF_PDO_DIR} --ledger-dir ${CCF_LEDGER_DIR}
elif [ ${F_NETWORK_MODE} == "join" ] ; then
die "joining a network is not yet supported"
else
Expand Down
7 changes: 7 additions & 0 deletions docker/tools/start_services.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,13 @@ try cp ${XFER_DIR}/ccf/keys/networkcert.pem ${PDO_LEDGER_KEY_ROOT}/
yell register the enclave if necessary
# -----------------------------------------------------------------
if [ "${F_REGISTER,,}" == 'yes' ]; then
if [ ! -f ${XFER}/ccf/keys/memberccf_privk.pem ] ; then
die unable to locate CCF policies keys
fi

try cp ${XFER_DIR}/ccf/keys/memberccf_cert.pem ${PDO_LEDGER_KEY_ROOT}/
try cp ${XFER_DIR}/ccf/keys/memberccf_privk.pem ${PDO_LEDGER_KEY_ROOT}/
Comment on lines +122 to +123
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(same as above)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above.


try make -C ${PDO_SOURCE_ROOT}/build register
fi

Expand Down
127 changes: 77 additions & 50 deletions ledgers/ccf/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,72 +28,99 @@ ifndef PDO_SOURCE_ROOT
$(error Incomplete configuration, PDO_SOURCE_ROOT is not defined)
endif

CCF_COMPILE_TARGET ?= virtual
CCF_VERSION ?= 4.0.1
CCF_LEDGER_DIR ?= $(PDO_INSTALL_ROOT)/opt/pdo/ccf

NINJA ?= ninja $(NINJA_OPTIONS)

SCRIPTDIR ?= $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
CCFDSTDIR ?= $(PDO_INSTALL_ROOT)/opt/pdo/ccf
SRCDIR ?= $(PDO_SOURCE_ROOT)
WORKSPACEDIR := $(CCFDSTDIR)/workspace
BINDIR := $(CCFDSTDIR)/bin/
BLDDIR := $(SCRIPTDIR)/build

PYTHON_DIR=$(CCFDSTDIR)/lib/python3.8

COMPILE_TARGET=virtual

all: environment install

build: build-pdo-tp
# -----------------------------------------------------------------
# There are two environments that need to be created: the PDO
# environment and the ledger environment. This is necessitated by
# the incompatibilities between the client requirements (which requires
# ccf 1.0.19 which is the last python package with the ccf client modules)
# and the ledger requirements (which installs the python modules from
# our current verson of CCF). The client environment may be useful on
# any pdo installation (e.g. ledger ping test makes sense on any client)
# while the ledger environment is only useful where ccf nodes are run
# -----------------------------------------------------------------
environment: pdo-environment ledger-environment

pdo-environment : $(PDO_INSTALL_ROOT)

ledger-environment : $(CCF_LEDGER_DIR)/lib/python3.8
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth declaring the python version as a variable to prepare for the maybe-case that ccf starts supporting also ubuntu >20.04? In any case, maybe having it close to CCF_VERSION and commenting that this comes from the current intrinsic 20.04-implying-python-3.8 invariant?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

possibly. although in this case the python version is completely determined by ccf.


# This directory indicates whether the pdo python virtual
# environment has been created
$(PDO_INSTALL_ROOT) :
make -C $(PDO_SOURCE_ROOT)/build environment
make -C $(PDO_SOURCE_ROOT)/bin install
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

installing all of PDO environment inside the CCF container seems to be an overkill. Doesn't hurt, I guess all that we want is ccf 1.0.19, do we need anything else ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think you have a misunderstanding of how your code worked in the past. you were already installing everything in python_requirements.txt which is all this does (except cleaner).

AND...

I absolutely think we should be installing pdo client packages (and all of these scripts should be installed in every pdo client). there is a separate issue to track that discussion (#469 )


# This directory indicates whether the ccf ledger python
# virtual environment has been created
$(CCF_LEDGER_DIR)/lib/python3.8 :
mkdir -p $(CCF_LEDGER_DIR)
mkdir -p $(CCF_LEDGER_DIR)/workspace
virtualenv -p python3.8 --no-download $(PDO_HOME)/ccf
$(CCF_LEDGER_DIR)/bin/pip install --upgrade pip
$(CCF_LEDGER_DIR)/bin/pip install --upgrade -r $(CCF_BASE)/bin/requirements.txt
$(CCF_LEDGER_DIR)/bin/pip install ccf==$(CCF_VERSION)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious, the pre-reqs installed in the prior step do not depend on the ccf version?

BTW: this target in principle is also cacheable and takes even with docker-cached packages one minute but will be re-executed after a make clean as it's after the COPY in the docker file. Maybe this 1 minute is not worth optimizing given that make clean test takes 10 mins, but just a thought in case we want to improve that time eventually ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well... (and i think this is in one of the comments) for the MOMENT we require ccf 1.0.19 for the PDO installation and for the ccf pdo scripts. HOWEVER, the actual ccf nodes require a different version. i tried to get the version out of the CCF_BASE (something like: $(<"${CCF_BASE}/share/version")) but i couldn't get make to execute it in a reasonable way. feel free to suggest an alternative that would compute it. AND... the CCF_BASE requirements already have the correct list for that version

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one more note continuing the python version discussion... all of the ccf scripts have the python version hardcoded in them. there is no place i could find where the python dependency is expressed in a way i can automate configuration. so 1) i don't think hard coding it is such a problem right now and 2) i don't think there is an easy way to derive it from the ccf version


# -----------------------------------------------------------------
# build it
# -----------------------------------------------------------------
build : build-pdo-tp

build-pdo-tp : $(BLDDIR)
cd $(BLDDIR) && cmake .. -GNinja \
-DCCF_DIR=$(CCF_BASE) \
-DCOMPILE_TARGET=$(COMPILE_TARGET) \
-DCMAKE_INSTALL_PREFIX=$(CCFDSTDIR)
cd $(BLDDIR) && $(NINJA)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't that somewhat cleaner with newer cmake as 'cmake --build $(BLDDIR)? (I guess we don't have a test target for an additionalcmake --test $(BLDDIR)`?)

Copy link
Contributor Author

@cmickeyb cmickeyb Feb 15, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. i'm getting there. there are several other modern cmake improvements that can be made. in this case i also don't know the best way to build with ninja in cmake. all things queued up for later investigation.


clean : clean-build clean-install
$(BLDDIR) :
cmake -S . -B $(BLDDIR) -GNinja \
-DCCF_DIR=$(CCF_BASE) \
-DCOMPILE_TARGET=$(CCF_COMPILE_TARGET) \
-DCMAKE_INSTALL_PREFIX=$(CCF_LEDGER_DIR)

# -----------------------------------------------------------------
# clean up
# -----------------------------------------------------------------
clean : clean-build clean-ledger

clean-build:
rm -rf $(BLDDIR)

clean-install :
rm -rf $(CCFDSTDIR)
clean-ledger :
rm -rf $(CCF_LEDGER_DIR)

environment : $(CCFDSTDIR) $(PYTHON_DIR)
# -----------------------------------------------------------------
# install the pdo tp library and scripts in the appropriate
# directories; the library and bash scripts are only necessary
# for running the ledger and go in the ledger directory; the
# python scripts may be useful on any client so they are installed
# in the pdo install root directory where the rest of the pdo
# scripts are installed. future work to move the python scripts
# to an installable wheel file
# -----------------------------------------------------------------

$(BLDDIR) :
@echo CREATE BUILD DIRECTORY $(BLDDIR)
mkdir -p $(BLDDIR)

# build out the entire pdo directory tree since
# we may use it for placement of the ledger keys
$(CCFDSTDIR) :
@echo CREATE INSTALLATION DIRECTORY $(CCFDSTDIR)
@make -C $(PDO_SOURCE_ROOT)/build $(PDO_INSTALL_ROOT)
@mkdir -p $(CCFDSTDIR)
@mkdir -p $(WORKSPACEDIR)

$(PYTHON_DIR) :
echo ${PYTHON_DIR}
virtualenv -p python3.8 --no-download $(CCFDSTDIR)
. $(abspath $(CCFDSTDIR)/bin/activate) && pip install --upgrade pip
. $(abspath $(CCFDSTDIR)/bin/activate) && pip install --upgrade setuptools
. $(abspath $(CCFDSTDIR)/bin/activate) && pip install --upgrade -r $(CCF_BASE)/bin/requirements.txt
. $(abspath $(CCFDSTDIR)/bin/activate) && pip install --upgrade ccf==1.0.19 toml

install : install-pdo-tp

PDO_BASH_SCRIPTS=start_ccf_network.sh stop_cchost.sh
PDO_PYTHON_SCRIPTS=configure_ccf_network.py generate_ledger_authority.py fetch_ledger_authority.py register_enclave_attestation_verification_policy.py
PDO_BASH_SCRIPTS = $(wildcard scripts/*.sh)
PDO_PYTHON_SCRIPTS = $(wildcard scripts/*.py)

install : install-pdo-tp install-pdo-scripts

install-pdo-tp : build-pdo-tp
@ cd $(BLDDIR) && $(NINJA) install
@ cp $(addprefix scripts/,$(PDO_BASH_SCRIPTS)) $(BINDIR)
@ cp $(addprefix scripts/,$(PDO_PYTHON_SCRIPTS)) $(BINDIR)
@ make -C ${PDO_SOURCE_ROOT}/bin install

.PHONY : all build build-pdo-tp
.PHONY : clean clean-build clean-install
.PHONY : environment
.PHONY : install install-pdo-tp
cd $(BLDDIR) && $(NINJA) install
cp $(PDO_BASH_SCRIPTS) $(CCF_LEDGER_DIR)/bin

install-pdo-scripts :
cp $(PDO_PYTHON_SCRIPTS) $(PDO_INSTALL_ROOT)/bin

# -----------------------------------------------------------------
.PHONY : all
.PHONY : environment pdo-environment ledger-environment
.PHONY : build build-pdo-tp
.PHONY : clean clean-build clean-ledger
.PHONY : install install-pdo-tp install-pdo-scripts
Loading
Loading