Home

23-Jun-2020, KAB: Draft of App Fwk v1 user example introduction

Introduction

This page is intended to provide a step-by-step guide to setting up a work area in which you create software modules that run within Version 1 of the DUNE DAQ software application framework (App Fwk v1). These modules are often called DAQModules since that is the C++ base class that they inherit from. If you are impatient to get started, you can jump to the Step-by-step Instructions now. New users should probably continue reading this introduction, though, to learn a little bit of background information.

This page draws from, and references, several other Wiki pages in the appfwk repository. Links to these pages are listed in the text in the relevant places, and they are a good source of additional information for interested readers. Here are some of these references collected for you in one place on this page:

• Compiling and running the App Fwk
• Creating a new package
• How to write your DAQModule

and last, but certainly not least

• The DUNE DAQ C++ Style Guide

This page assumes a basic understanding of how modules are combined to form processes within App Fwk v1. One reference is the sneak peak talk given by Eric on 15-Jun-2020. As a refresher,

• a DAQProcess contains one or more DAQModules
• when multiple DAQModules are present within a DAQProcess, they communicate with each other via Queues
• there are two classes that wrap Queues to provide the ability to push data onto the queue (DAQSink) or pull data from a queue (DAQSource). An individual DAQModule will only access one side of each Queue. If the module pushes data onto the queue, it will use an instance of the DAQSink class (which wraps the desired Queue), and if the module pops data from the queue, it will use an instance of the DAQSource class.
• the DAQModules that run within a given process (and the Queues between them) are specified in a JSON process configuration file. An example of one such file is given below. The creation of the DAQModules and Queues is handled by the App Fwk.
• at this point in time, we expect that most users will be developing DAQModules and simply using existing Queues from the App Fwk.
• at the moment, there aren't any centrally-provided libraries, tools, or recommendations for inter-process communication. We expect to address this topic soon, but for now, developers can either focus on single-process examples, or use other software for inter-process communication. The instructions on this page focus on single-process examples.
• please remember that there have been a number of compromises or simplifications in the functionality that is provided by App Fwk v1. We have made a good faith attempt to provide a good start on the internal interfaces and layout of DAQProcesses, but things will definitely change over time, and we will be gathering feedback from everyone on ways that things might be improved. (Of course, we already have a fairly good list based on the experience of creating v1.)

Step-by-step Instructions

Here are the suggested steps for setting up a work area, in which you can create your new package:

• Choose a computer/cluster to work on that has access to /cvmfs/dune.opensciencegrid.org/dunedaq/DUNE/products (more details on compiling and running code with the App Fwk are provided on the Compiling and Running page)
  • this CVMFS area is where several necessary dependent software packages are stored (for now)
  • I've used lxplus.cern.ch for my work so far
• Log into that computer/cluster, create a fresh directory to work in, and cd into that directory (I'll call this your WORK_DIR.)
• Run the following three commands (these were copied from the Compiling and Running page)
  • curl -O https://raw.githubusercontent.com/DUNE-DAQ/daq-buildtools/develop/bin/quick-start.sh
  • chmod +x quick-start.sh
  • ./quick-start.sh
• Set up the build environment for this software environment by running the following command:
  • source ./setup_build_environment
• Build the core packages that the quick-start.sh sript installed in your working directory by running this command:
  • ./build_daq_software.sh
  • please note that you'll likely see some messages like nlohmann_json NOT FOUND!. This is not a cause for alarm. The build process automatically downloads and builds the nlohmann::json package for you, after it notices that it wasn't found.
  • there may also be a warning message about an unused variable in ers/src/Issue.cxx. Again, this is nothing to worry about.
• If you would like to try running the example application(s) from the core appfwk repository, you can do that now. There are instructions later on this page for doing that. link

At this point, you could either check out the example DAQModule package, build it, and run the the example application, or you could start creating your own package. These instructions will walk you through doing both, but of course you can skip to the latter by scrolling down half a page.

Here are the steps for adding the App Fwk Example package to your work area, building it, and running the example application... (For information on the DAQModules that are contained in the example application and how they interact, please see this section later on this page.)

• cd into your work area directory (WORK_DIR), if you aren't there already
• run the following command to clone the example package:
  • git clone https://github.com/DUNE-DAQ/listrev.git
• add the following line to the bottom of the top-level CMakeLists.txt file (in WORK_DIR)
  • add_subdirectory(listrev)
• [if not already done in the current shell] set up the build environment for your work area
  • source ./setup_build_environment
• rebuild the software in your work area by running
  • ./build_daq_software.sh
• create a setup_runtime_environment script in your WORK_DIR with the following contents:
  • source ./setup_build_environment
  • source build/appfwk/scripts/setupForRunning.sh
  • export CET_PLUGIN_PATH=`pwd`/build/listrev/src${CET_PLUGIN_PATH:+:$CET_PLUGIN_PATH}
  • export LD_LIBRARY_PATH=`pwd`/build/listrev/src${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}
• set up the runtime environment setup script by running this command:
  • source ./setup_runtime_environment
  • please note that this needs to be done from WORK_DIR (yes, I know that this isn't ideal; I'm just trying to create something that works, for now)
• run the example application using the following command:
  • build/appfwk/apps/daq_application -c QueryResponseCommandFacility -j build/listrev/test/list_reversal_app.json
• once the program is running, and you see the "Enter a command" prompt, you can type in commands like the following:
  • configure
    • This command pretends to set the values of configurable parameters like the number of integers in each randomly generated list
  • start
    • This command is passed to the three DAQModules in a well-specified order (specified in the JSON process configuration file). First, the validator is started, then the reverser, then the generator. Once the generator is started, it begins creating lists of integers and passing them to the other two DAQModules.
    • On the console, you will see ERS LOG messages from each of the DAQModules saying that they started (so you can confirm that the start order is correct), and then you will see ERS DEBUG messages that tell you what each of the DAQModules is doing as they process the lists of integers.
  • stop
    • This command stops the three DAQModules, in the reverse order that they were started (like the start order, the stop order is specified in the JSON process config file).
    • You will need to type this command into the console a little blindly, since the ERS DEBUG messages will be printing to the console as the program runs.
    • After each of the three DAQModules has finished, they print an ERS INFO message to the console with a summary of what they accomplished.
  • unconfigure
    • This command pretends to tear-down whatever configuration was established in the configure step.
  • quit
    • This command exits the program.

Here are the commands to create your own software packageL

• create a new directory underneath WORK_DIR (we'll call this YOUR_PKG_DIR)
• copy the CMakeLists.txt file from the example package into YOUR_PKG_DIR (you can fetch it from here)
  • edit this CMakeLists.txt file to change the name of the package to your package name
  • (we'll do a couple more edits in a bit, but you can save and close this file now
• create a src directory underneath YOUR_PKG_DIR
  • copy one of the DAQModules from the example package into the src directory
  • as an example, let's copy the RandomDataListGenerator DAQModule from the example package. You can fetch the header and source files from here and here.
• edit YOUR_PKG_DIR/CMakeLists.txt to remove the ListReverser and ReversedListValidator add_library and target_link_libraries lines
• add the following line to the bottom of the top-level CMakeLists.txt file (in WORK_DIR)
  • add_subdirectory(YOUR_PKG_DIR)
• cd to WORK_DIR
• [if not already done in the current shell] set up the build environment for your work area
  • source ./setup_build_environment
• rebuild the software in your work area by running
  • ./build_daq_software.sh

Information about the DAQModules in the listrev example

The idea behind the listrev example is to have one DAQModule that generates a list of random integers, another DAQModule that reverses a copy of the list, and a third DAQModule that compares copies of the original and reversed lists to validate that they are equivalent, modulo the reversal.

This is shown in the following diagrams. The first one provides a little description of what each of the DAQModules is doing, and the second one shows the class names of the DAQModules and the configured names of the DAQModules and the Queues, as they are identified in the JSON process configuration file.

Here is a copy of the JSON process configuration file (the official copy of this file in the repo is here). As you can see, the Queues are specified first, then the DAQModules, and the DAQModules include the configuration of which Queues they make use of.

{
  "queues": {
    "primaryDataQueue": {
      "capacity": 10,
      "kind": "FollySPSCQueue"
    },
    "reversedDataQueue": {
      "capacity": 10,
      "kind": "FollySPSCQueue"
    },
    "dataCopyQueue": {
      "capacity": 10,
      "kind": "FollySPSCQueue"
    }
  },
  "modules": {
    "generator": {
      "user_module_type": "RandomDataListGenerator",
      "outputs": [ "primaryDataQueue", "dataCopyQueue" ]
    },
    "reverser": {
      "user_module_type": "ListReverser",
      "input": "primaryDataQueue",
      "output": "reversedDataQueue"
    },
    "validator": {
      "user_module_type": "ReversedListValidator",
      "reversed_data_input": "reversedDataQueue",
      "original_data_input": "dataCopyQueue"
    }
  },
  "commands": {
    "start": [ "validator", "reverser", "generator" ],
    "stop": [ "generator", "reverser", "validator" ]
  }
}

To be added:

• Custom Exceptions/ERS Issues
• Logging and Tracing
  • describe how to view the two types of messages; include references for additional information on ERS and TRACE
  • describe how I used the method entry/exit TRACE messages to understand the program flow
  • describe how I used the do_work TRACE messages to debug my initial mistake in the process configuration file

Running the Fanout example in the appfwk repo

In a fresh shell, here are the steps that you would use to run the Fanout example in the appfwk repo after you have run quick-start.sh and built the software using build_daq_software.sh:

• cd to your WORK_DIR
• run the following commands to set up the build and runtime environments:
  • source ./setup_build_environment
  • source build/appfwk/scripts/setupForRunning.sh
• run the following command to start the example:
  • build/appfwk/apps/daq_application -c QueryResponseCommandFacility -j appfwk/test/producer_consumer_dynamic_test.json
• enter commands like the following:
  • configure
  • start
  • stop
  • 'quit`