Clyde Package Manager will allow you to build C and C++ packages for RTEMS and make development easier with a few helpful tools like documentation templates.
This package is in an early alpha stage. It is not ready to be adopted outside of Vecna, due to some hardcoded dependencies on a gerrit server.
Future versions will support multiple 'backends', to download packages, including arbitrary git urls, github username/package-name specifications and more.
Clyde is a purpose built package manager and build system for C and C++. It has built in support for features that are helfpul in developing firmware using the RTEMS Realtime Operating System.
This project is actively being used at Vecna. However, it is custom tailored to our use cases and may not be useful as is. Future releases may improve the internal stucture to make Clyde more generic, supporting different tool chains, RTOSs, etc.
Clyde is currently broken up into the clyde and clyde2 commands. As we migrate away from the original clyde, more commands will move into clyde2. Eventually clyde will be replaced by clyde2. Packages
Clyde projects are built by a hierarchical collection of one or more clyde packages. Each project consists of a set of C and/or C++ files, header files, and a configuration file. The simplest package will contain a single C file, and a simple configuration file that specifies no dependencies. Clyde takes care of the process of actually compiling projects, including dependencies, and handling most compiler flags. It does this by parsing the configuration file (config.yaml), and generating a build.nina file usable by the ninja build tool which actually is responsible for calling gcc and creating the final executable or library.
Let's take a look at a clyde package.
cd ~/ mkdir clyde-demo && cd clyde-demo clyde2 init application clyde fetch hello # Fetch a hello-world example C++ file:
tree
.
├── build
├── clyde2.log
├── config.yaml
├── include
└── src
└── hello.cpp
- build - An empty directory that will contain build artifacts, such as .o files generated from compiling .c and .cpp files.
- clyde2.log - A log file containing the most detailed output from running clyde. This log system is currently incomplete.
- config.yaml - Main project configuration. This file specifies basic information about a packages, such as name and version, as well as dependencies and more advanced features
- include/clyde-demo - Package include files should live here. Notice that an empty package creates include/package-name.
- src - C and C++ source files for this package.
- src/hello.cpp - A simple C++ file pulled in with clyde fetch hello
Assuming that we have a basic package, we can build it in two steps.
- Use clyde2 to create a build.ninja file from the package. This is done with the clyde2 gen command.
- Use ninja to invoke the actual compilation and linking steps.
For example:
cd ~/clyde-demo clyde2 gen tree ├── build.ninja ├── clyde2.log ├── config.yaml ├── deps ├── include │ └── clyde-demo ├── src │ └── hello.cpp └── versions.txt 4 directories, 5 files ninja [2/2] g++ -MMD -MF prefix/bin/clyde-demo.d -std=c++11 -fdiagnostics-color=always -Iprefix/include -Iinclude build/src/hello.o -o prefix/bin/clyde-demo tree . ├── build │ └── src │ ├── hello.o │ └── hello.o.d ├── build.ninja ├── clyde2.log ├── config.yaml ├── deps ├── include │ └── clyde-demo ├── prefix │ └── bin │ └── clyde-demo ├── src │ └── hello.cpp └── versions.txt 8 directories, 8 files
There area few interesting things to look at:
A build.ninja file was created. You should take a look at it A versions.txt file was created. It lists the exact git tag used to build each package in the dependency tree. For now, it will only list the top level package.
After building, a few more files are created, notably:
build/src/hello.o build/src/hello.o.d prefix/bin/clyde-demo
The two files in build are the object file created from hello.c, and a dependency file calculated by analyzing the the #include directives and include paths in hello.c Right now, this will be almost empty Dependencies
One of the largest features of clyde2 is the package management aspect. Clyde packages can depend on other clyde packages, and clyde will automatically download them and add them to the build.ninja when you re-run gen.
In our extended example, let's add some unit tests by pulling in the CppUTest dependency. Edit config.yaml to look like this
author: Isaac Gutekunst author-email: [email protected] cflags: {gcc: -std=c++11} name: clyde-demo type: application url: http://example.com version: 0.0.0 requires: clyde-demo-lib: version: ">=1.0.0"
A few things to notice: The version of clyde-demo listed is a version range. This represents a version specification to download. However, we could have just as easily used a "==1.0.2", or "*". Clyde understands these version specifications and selects the "best" version by parsing the version strings as a semantic version.
If we re-run clyde2 gen, clyde-demo-lib will be downloaded from git, and placed in the deps folder. The build.ninja file will be updated to include these dependencies.
tree . ├── build │ └── src │ ├── hello.o │ └── hello.o.d ├── build.ninja ├── clyde2.log ├── config.yaml ├── deps │ └── clyde-demo-lib │ ├── config.yaml │ ... Truncated to conserve space ├── include │ └── clyde-demo ├── prefix │ └── bin │ └── clyde-demo ├── src │ └── hello.cpp └── versions.txt
Let's update src/hello.cpp
#include <iostream>
#include <clyde-demo-lib/demo.h>
int main(int argc, char * argv[]) {
print_demo_text("Demo Text");
}
Now, let's build and run our new project:
ninja ./prefix/bin/clyde-demo
Building
Building is performed in two steps
Generating the build.nina using clyde2 gen Actually compiling using ninja
Since the build.ninja describes all dependencies, editing any file, and then running ninja will cause it to properly rebuild the appropriate.
You should re-reun clyde2 gen whenever you update a dependency, or change config.yaml
--fast flag
If you have a lot of dependencies, the time it takes to check the remote server for each version of a package is rather slow. If you pass the --fast flag to clyde2 gen, it will only look at the local git tags. Future versions will have a hybrid approach that will fetch remotes, but only once per invocation of clyde2 gen. Platforms
One unique feature of clyde that other package managers do not have is the notion of platforms and variants. Platforms are places where clyde packages can run, such as on a linux computer, or on a RTEMS embedded system. Currently the main supported platforms are rtems and linux.
You can specify which platform to build for by specifying --platform when invoking clyde2 gen. When you specify --platform rtems, you may get an error Could not find rtems in / . No BSP specified. Specify with --rtems. In this case, you should specify the path to the RTEMS BSP e.g $RTEMS/arm-rtems4.11/stm32f7x/. This is the same as what you would set RTEMS_MAKEFILE_PATH to.
When you set the platform to rtems, clyde will pass RTEMS specific CFLAGS to the build process. If the type of the clyde package is application, clyde will perform a final link, producing an elf file that can be loaded onto a target system. Variants
Clyde has a concept of a variant. A basic package has an implicit variant src. The src variant is enabled by default. This is why the C and C++ files in src are built by default. Additional variants are created by creating new directories in a clyde package that don't conflict with the reserved names (src, include, build, prefix). For example, you can make a test variant, and put unit tests in there. Activating variants
Variants are activated when conditions specified in the config.yam are met. The conditions are specified using a simple language.
All variants must be listed in config.yaml Each variant must have a when clause Each variant can also have a replaces clause Each variant can have a cflags clause Each variant can have a requires clause
When clauses are true when all the predicates are true.
Predicates are statements about the configuration variables variant, platform and compiler.
Some examples might make more sense then a wall of text:
variants:
- test:
when: {variant: test}
requires:
cpputest: {version: "*"}
replaces: src
cflags: -DUNIT_TEST
This example uses all the clauses. It is active when the variant == test. This is true when you run clyde2 with --variant test. If the variaant variable is set to test, the actual variant becomes active. Its effects will then be processed.
In this case, the src variant will be disabled (because of the replaces clause), clyde will pull in the latest version of cpputest (requires clause), and define the UNIT_TEST symbol due to the cflags clause.
These clauses allow you to construct platform specific variants, such as a package that compiles different C++ files depending on whether it is compiled for RTEMS or Linux.
Here's a little example:
variants:
- linux:
when: {platform: linux}
cflags: -DPLATFORM_LINUX
- rtems:
when: {platform: rtems}
cflags: -DPLATFORM_RTEMS
In this case, the rtems or linux variant will be activated depending on the target platform. Note: The platform is specified on the command line with --platform.
Controlling Verbosity
Some clyde2 commands can be suffixed with --verbose to get more detailed output. On a successful build, clyde will print a full dependency graph as JSON. On a failure, --verbose will print a full stack trace, not just a message. Special RTEMS Functionality
As described in variants and platforms, clyde has special functionality for building for the RTEMS target. See those sections for details. Template Support
Clyde supports pulling in project and file templates. For a list of templates, type clyde fetch --list
clyde fetch --list Available templates docs - Sphinx and doxygen generation files gitignore - Gitignore file ycm - YouCompleteMe VIM autocomplete templates empty-test - Empty test skeleton quickstart - Basic clyde package template hello - Basic clyde hello project rtems-init - Basic initialization template for STM32F7 BSP rtems-init-f4 - Basic initialization template for STM32F4 BSP rtems-quickstart - RTEMS Top level project with a all files needed to get going fast
To fetch a specific template, use clyde fetch template-name
clyde fetch rtems-quickstart Loading /home/igutek/.clyde/config Created docs/index.rst Created docs/Makefile Created docs/conf.py Created docs/doxyfile Warning: Refusing to overwrite existing file .gitignore Warning: Refusing to overwrite existing file .ycm_extra_conf.py Created src/rtems_entry_stm32f7.cpp Copied RTEMS initialization for STM32F7. For the F4, fetch rtems-init-f4
This output shows a useful feature: The template system will not overwrite existing files. Template configuration
The clyde python package itself contains a list of supported templates. It lives in clydepm/templates/template_config.yaml Directives
The file is a yaml file with a root entry named templates, containing a list of mappings. templates
Top level list of templates
Each mapping should contain an include, directory, and description. include
Every file in the include will be copied to directory in the package. To rename a file when copying it, wrap the filename in an as statement:
- Makefile.sphinx: {as: Makefile}
This is useful if you have multiple Makefiles that should be named Makefile in the final installation. directory
The target directory for the items in this template. If you want to copy to multiple packages, create multiple packages, each with its own directory, and group those into a meta-package.
description
A short description of the template
message
A message to be printed when the template is instantiated meta-include
If a template has a meta-include directive, it will also instantiate all templates listed under the meta-include statement
Templates themselves will be standard text files using either the Jinja2 or Mustache templating language. As these engines don't support any notion of directory structure, there will be a template configuration file that describes what a template package is.
At the time of this writing, template_config.yaml contained the following configuration.
# All clyde templates are initially configured
# in this file.
templates:
- docs:
description: Sphinx and doxygen generation files
directory: docs
include:
- index.rst
- Makefile.sphinx: {as: Makefile}
- conf.py
- doxyfile
- gitignore:
description: Gitignore file
directory: $PWD
include:
- gitignore: {as: .gitignore}
- ycm:
description: YouCompleteMe VIM autocomplete templates
directory: $PWD
include:
- ycm_extra_conf.py: {as: .ycm_extra_conf.py}
- empty-test:
description: Empty test skeleton
message: Please update your config.yaml to add a test variant
variants:
directory: test
include:
- cpputest-empty.cpp: {as: test.cpp}
- quickstart:
description: Basic clyde package template
meta-include:
- docs
- gitignore
- ycm
- hello
- hello:
description: Basic clyde hello project
message: Installed basic hello project
directory: src
include:
- hello.cpp
- rtems-init:
description: Basic initialization template for STM32F7 BSP
message: Copied RTEMS initialization for STM32F7. For the F4, fetch rtems-init-f4
directory: src
include:
- rtems_entry_stm32f7.cpp
- rtems-init-f4:
description: Basic initialization template for STM32F4 BSP
message: Copied RTEMS initialization for STM32F4. For the F7, fetch rtems-init
directory: src
include:
- rtems_entry_stm32f4.cpp
- rtems-quickstart:
description: RTEMS Top level project with a all files needed to get going fast
meta-include:
- docs
- gitignore
- ycm
- rtems-init
- Supporting variou backends besides gerrit
- Friendly defaults and configuration options
- More generic concept of platforms
- Merge clyde clyde2 and properly deprecate clyde
- Ability to run tests on real hardware (clyde test --variant rtems might do something specil)