Skip to content

Latest commit

 

History

History
237 lines (180 loc) · 10.7 KB

testing.md

File metadata and controls

237 lines (180 loc) · 10.7 KB
Raw

Testing Demikernel

This document contains instructions on how to test Demikernel.

The instructions listed in this file assume that you have at your disposal at least a pair of machines with Demikernel's development environment set up. For more information on how to set up Demikernel's development environment, check out instructions in the README.md file.

Table of Contents

What Are Unit Tests

Unit tests exercise internal components of Demikernel, individually.

Demikernel has several unit tests, and all of them spread across the source tree in demikernel/src/. To check their implementation, search for mod tests and cfg(test) in the code base.

How to Run Unit Tests

Unit tests may be run in multiple ways:

Regardless the method you choose to follow:

  • You should have at your disposal a pair of machines with Demikernel's development environment set up. For more information on how to set up Demikernel's development environment, check out instructions in the README.md file.
  • Catnip and Catpowder LibOSes require you to have superuser privileges on the testing machines.

Running Unit Tests with Demikernel's CI Tool (i.e., demikernel-ci)

In order to use this method, you should first do the following setup once:

  • Setup RSA-based authentication for both server and host machines.
  • Setup SSH aliases for connecting to both server and host machines.
# Set parameters for remote machines.
export SERVER_HOSTNAME=demikernel-vm0
export SERVER_IPV4_ADDR=192.0.2.10
export CLIENT_HOSTNAME=demikernel-vm1
export CLIENT_IPV4_ADDR=192.0.2.11
export DEMIKERNEL_PATH=/path/to/demikernel
export DEMIKERNEL_BRANCH=dev

# Run unit tests with Catnap LibOS.
# If you want to run unit tests on a different LibOS, then set the LIBOS flag accordingly.
export LIBOS=catnap

# Run all system tests for the target LIBOS.
python3 tools/demikernel_ci.py \
    --server $SERVER_HOSTNAME \
    --client $CLIENT_HOSTNAME \
    --repository $DEMIKERNEL_PATH \
    --branch $DEMIKERNEL_BRANCH \
    --libos $LIBOS \
    --test-unit \
    --server-addr $SERVER_IPV4_ADDR \
    --client-addr $CLIENT_IPV4_ADDR

Running Unit Tests from the Shell (i.e., bash)

# Run unit tests with Catnap LibOS.
# If you want to run unit tests on a different LibOS, then set the LIBOS flag accordingly.
export LIBOS=catnap

# Run all unit tests.
make LIBOS=$LIBOS test-unit

What Are System-Level Tests

System-level tests exercise the user-level API that is exposed by Demikernel.

These tests are located in demikernel/examples/ and detailed in the following tables.

The following system-level tests are supported by catnap, catnip, catpowder.

Test Name Description
tcp-push-pop Tests for fixed size, unidirectional data transfers using TCP.
tcp-ping-pong Tests for fixed size, bi-directional, symmetric data transfers using TCP.
udp-push-pop Tests for fixed size, unidirectional data transfers using UDP.
udp-ping-pong Tests for fixed size, bi-directional, symmetric data transfers using UDP.

The following system-level tests are supported by catmem.

Test Name Description
pipe-push-pop Tests for fixed size, unidirectional data transfers.
pipe-ping-pong Tests for fixed size, bi-directional, symmetric data transfers.

How to Run System-Level Tests

System-level tests may be run in multiple ways:

Regardless the method you choose to follow:

  • You should have at your disposal a pair of machines with Demikernel's development environment set up. For more information on how to set up Demikernel's development environment, check out instructions in the README.md file.
  • Catnip and Catpowder LibOSes require you to have superuser privileges on the testing machines.

Running System-Level Tests with Demikernel's CI Tool (i.e., demikernel-ci)

In order to use this method, you should first do the following setup once:

  • Setup RSA-based authentication for both server and host machines.
  • Setup SSH aliases for connecting to both server and host machines.
# Set parameters for remote machines.
export SERVER_HOSTNAME=demikernel-vm0
export SERVER_IPV4_ADDR=192.0.2.10
export CLIENT_HOSTNAME=demikernel-vm1
export CLIENT_IPV4_ADDR=192.0.2.11
export DEMIKERNEL_PATH=/path/to/demikernel
export DEMIKERNEL_BRANCH=dev

# Run system-level tests with Catnap LibOS, unless otherwise stated.
# If you want to run system-level tests on a different LibOS, then set the LIBOS flag accordingly.
export LIBOS=catnap

# Run all system-level tests for the target LIBOS.
python3 tools/demikernel_ci.py \
    --server $SERVER_HOSTNAME \
    --client $CLIENT_HOSTNAME \
    --repository $DEMIKERNEL_PATH \
    --branch $DEMIKERNEL_BRANCH \
    --libos $LIBOS \
    --test-system all \
    --server-addr $SERVER_IPV4_ADDR \
    --client-addr $CLIENT_IPV4_ADDR

Running System-Level Tests with Demikernel's Build System (i.e., make)

# Set parameters for Demikernel's TCP/UDP stack.
export SERVER_IPV4_ADDR=192.0.2.10:56789
export CLIENT_IPV4_ADDR=192.0.2.11:56789

# Run system-level tests with Catnap LibOS, unless otherwise stated.
# If you want to run system-level tests on a different LibOS, then set the LIBOS flag accordingly.
export LIBOS=catnap

# Run tcp-push-pop.
make test-system-rust TEST=tcp-push-pop ARGS='--server $SERVER_IPV4_ADDR' # Run this on server host.
make test-system-rust TEST=tcp-push-pop ARGS='--client $SERVER_IPV4_ADDR' # Run this on client host.

# Run tcp-ping-pong.
make test-system-rust TEST=tcp-ping-pong ARGS='--server $SERVER_IPV4_ADDR' # Run this on server host.
make test-system-rust TEST=tcp-ping-pong ARGS='--client $SERVER_IPV4_ADDR' # Run this on client host.

# Run udp-push-pop.
make test-system-rust TEST=udp-push-pop ARGS='--server $SERVER_IPV4_ADDR $CLIENT_IPV4_ADDR' # Run this on server host.
make test-system-rust TEST=udp-push-pop ARGS='--client $CLIENT_IPV4_ADDR $SERVER_IPV4_ADDR' # Run this on client host.

# Run udp-ping-pong.
make test-system-rust TEST=udp-ping-pong ARGS='--server $SERVER_IPV4_ADDR $CLIENT_IPV4_ADDR' # Run this on server host.
make test-system-rust TEST=udp-ping-pong ARGS='--client $CLIENT_IPV4_ADDR $SERVER_IPV4_ADDR' # Run this on client host.

# Run pipe-push-pop.
# - Note 1: Both commands should be executed on the same host.
# - Note 2: The 'demikernel-pipe-name' parameter is used to name the underlying pipe.
# - Note 3: You may change the pipe name to whatever you want but both ends should agree on the same name.
make test-system-rust LIBOS=catmem TEST=pipe-push-pop ARGS='--server demikernel-pipe-name'
make test-system-rust LIBOS=catmem TEST=pipe-push-pop ARGS='--client demikernel-pipe-name'

# Run pipe-ping-pong.
# - Note 1: Both commands should be executed on the same host.
# - Note 2: The 'demikernel-pipe-name' parameter is used to name the underlying pipe.
# - Note 3: You may change the pipe name to whatever you want but both ends should agree on the same name.
make test-system-rust LIBOS=catmem TEST=pipe-ping-pong ARGS='--server demikernel-pipe-name'
make test-system-rust LIBOS=catmem TEST=pipe-ping-pong ARGS='--client demikernel-pipe-name'

Running System-Level Tests from the Shell (i.e., bash)

# Set location for Demikernel's config file.
export CONFIG_PATH=/path/to/config.yaml

# Set parameters for Demikernel's TCP/UDP stack.
export MSS=1500
export MTU=1500
export SERVER_IPV4_ADDR=192.0.2.10:56789
export CLIENT_IPV4_ADDR=192.0.2.11:56789

# Run system-level tests with Catnap LibOS, unless otherwise stated.
# If you want to system-level tests on a different LibOS, then set the LIBOS flag accordingly.
export LIBOS=catnap

# Run tcp-push-pop.
bin/examples/rust/tcp-push-pop.elf --server $SERVER_IPV4_ADDR # Run this on server host.
bin/examples/rust/tcp-push-pop.elf --client $SERVER_IPV4_ADDR # Run this on client host.

# Run tcp-ping-pong.
bin/examples/rust/tcp-ping-pong.elf --server $SERVER_IPV4_ADDR # Run this on server host.
bin/examples/rust/tcp-ping-pong.elf --client $SERVER_IPV4_ADDR # Run this on client host.

# Run udp-push-pop.
bin/examples/rust/udp-push-pop.elf --server $SERVER_IPV4_ADDR $CLIENT_IPV4_ADDR # Run this on server host.
bin/examples/rust/udp-push-pop.elf --client $CLIENT_IPV4_ADDR $SERVER_IPV4_ADDR # Run this on client host.

# Run udp-ping-pong.
bin/examples/rust/udp-ping-pong.elf --server $SERVER_IPV4_ADDR $CLIENT_IPV4_ADDR # Run this on server host.
bin/examples/rust/udp-ping-pong.elf --client $CLIENT_IPV4_ADDR $SERVER_IPV4_ADDR # Run this on client host.

# Run pipe-push-pop.
# - Note 1: Both commands should be executed on the same host.
# - Note 2: The 'demikernel-pipe-name' parameter is used to name the underlying pipe.
# - Note 3: You may change the pipe name to whatever you want but both ends should agree on the same name.
LIBOS=catmem bin/examples/rust/pipe-push-pop.elf --server demikernel-pipe-name
LIBOS=catmem bin/examples/rust/pipe-push-pop.elf --client demikernel-pipe-name

# Run pipe-ping-pong.
# - Note 1: Both commands should be executed on the same host.
# - Note 2: The 'demikernel-pipe-name' parameter is used to name the underlying pipe.
# - Note 3: You may change the pipe name to whatever you want but both ends should agree on the same name.
LIBOS=catmem bin/examples/rust/pipe-ping-pong.elf --server demikernel-pipe-name
LIBOS=catmem bin/examples/rust/pipe-ping-pong.elf --client demikernel-pipe-name