Skip to content

Latest commit

 

History

History
193 lines (136 loc) · 8.73 KB

quickstart.md

File metadata and controls

193 lines (136 loc) · 8.73 KB
Raw

Development Getting Started

Introduction

The following describes the steps involved to initialize a CDF development environment from scratch, to build, run and test a project, then finally on how to commit modifications to the source code.

Due to the scripts used as part of both the build and deployment steps, only linux type environments (including macOS) are officially supported.

Configuring the development environment

The following is a one-time setup to configure the CDF development environment.

> git clone https://github.com/aws/aws-connected-device-framework.git
> cd aws-connected-device-framework/source
  • Optionally switch to a specific release tag:
aws-connected-device-framework/source> git switch -c tags/<release>

Build

The CDF monorepo is managed by rush which under the covers is configured to use pnpm as its package manager. The following is a brief introduction of how to use rush:

# If this is your first time at using Rush for this project, remove any node_modules 
# that may have been installed as part of a non-Rush (npm/pnpm) release:
aws-connected-device-framework/source> rm -rf node_modules

# One time setup only, install pinned dependencies based on shrinkwrapped
aws-connected-device-framework/source> rush install

# When running the `clean`, `build`, `lint` or `test` commands you have the option to
# run globally (for all packages), or for a specific package. To run for all packages
# run as follows:
#
#           rush <command>
# 
# To run for a specific package you can either provide a target filter as follows:
#
#           rush <command> -t <package_name>
#
# or alternatively run the following within the package's directory (which is a shortcut
# for `rush <command> -t .`):
#
#           rushx <command>
#

# Taking the above comments into consideration, to build run the following. Note that the first build
# may take time, but subsequent builds will be quicker delta builds:
aws-connected-device-framework/source> rush build

# To lint:
aws-connected-device-framework/source> rush lint

# And to run unit tests:
aws-connected-device-framework/source> rush test

# If you experience issues and need to reset everything you have the following 2 commands available:
#   To remove all build artifacts:
aws-connected-device-framework/source> rush purge        # to purge all node_modules:
aws-connected-device-framework/source> rush clean        # perform a deep clean
aws-connected-device-framework/source> rush install      # refresh dependencies again

Running a module locally

Each module uses dotenv-flow to manage its application configuration. When a module is deployed, the installer module takes care of injecting the configuration into the environment. But when running locally, this configration needs to be provided.

The easiest way to generate this configuration is to use the config-to-env command of the installer module, If you need to manually generate the configuration file, or understand the available configuration, refer to the modules own docs/configuration.md help file.

Once you have the configuration file, you can start a module as follows:

aws-connected-device-framework> cd source/packages/services/<module_name>
aws-connected-device-framework/source/packages/services/<module_name>> export CONFIG_LOCATION=<path-to-env-file>; npm run start

Making changes to an existing module

We adhere to what is known as a GitHub flow as far as our approach to branching is concerned. Basically this boils down to:

  • The main branch always represents a working version of the code, including latest (maybe unofficially released) updates, that may be deployed to a production environment
  • Under no circumstances ever commit directly to main!
  • When starting a new feature or fixing a bug, create a new branch from main. Name the branch feat_*** for new features or fix_*** for hotfixes:
aws-connected-device-framework> git switch -c <new_branch_name>

Switched to a new branch '<new_branch_name>'
  • At suitable points, commit your work by running the following, and following the prompts to describe your commit. Note that you must run rush commit inside the source/ directory whereas you can run the git commands anywhere within the repo.
aws-connected-device-framework> git add -A
aws-connected-device-framework> cd source
aws-connected-device-framework/source> rush commit

  • When you have finished with your implementation, and ensured that all existing unit tests pass as well as creating any new tests, the following steps are required:

    • Merge changes with the main branch:
# pull in main into your branch
aws-connected-device-framework> git merge origin/main

# once any conflicts have been resolved, test
aws-connected-device-framework> cd source
aws-connected-device-framework/source> rush test

# commit changes
aws-connected-device-framework/source> git add -A
aws-connected-device-framework/source> rush commit
    • Generate release notes. the rush change command will analyze all commits on the branch, filter out the projects that changed, then prompt you to enter release notes for the updated project:
# generate release notes
aws-connected-device-framework/source> rush change

# commit release notes
aws-connected-device-framework/source> git add -A
aws-connected-device-framework/source> rush commit
    • Push the branch to the git repo
aws-connected-device-framework> git push
    • Create a pull request

  • Once your pull request has been reviewed, and any issues addressed, merge your implementation back into the main code branch.

Changing package dependencies

If your code changes include changes to the dependencies listed in a package.json file. you need to update the repo-wide "shrinkwrap" file. This file contains the resolved and pinned dependencies for all CDF packages:

rush update
git add source/common/config/rush/pnpm-lock.yaml
git add source/common/config/rush/repo-state.json

Note that these two files cannot be merged. If your branch feat_branch has a merge conflict with main, the steps to resolve are:

  1. Merge main into feat_branch.
  2. Resolve all other merge conflicts, discard all changes in pnpm-lock.yaml and repo-state.json.
  3. Run rush update again.
  4. Commit the resulting pnpm-lock.yaml and repo-state.json to your branch.

Understanding the directory structure

Directory Description
source/cicd/ The CloudFormation template to deploy the cicd pipeline, along with the related CodeBuild scripts
source/common/ All build and package manager related files
source/docs/ CDF core related documentation
source/infrastructure/ The main deployment script for deploying the CDF core modules, along with CloudFormation templates that are not specific to any module
source/packages/integration-tests/ BDD related automated integration tests
source/packages/libraries/ All internal libraries, as well as CDF client libraries
source/packages/services/ Deployable modules, such as the Asset Library

FAQ

  • "Why do I have to use rush commit to commit my work instead of the usual git commit?"

    rush commit is configured to run the commitizen command line utility which forces you to describe commits using a specific format. This is vitally important, as there are steps in our CI/CD pipeline that analyze all git commits since the last release, use these specially formatted messages to intelligently increment the version numbers (e.g. understand if there's a breaking change), and finally auto generates a change log for the release.

  • "What is the need for using (rush) and (pnpm package manager)? What's wrong with just npm?" The aws-connected-device-framework git repo is what is known as a monorepo, a large single repository that contains many different projects.

    The decision to migrate CDF to a monorepo was made to:

    • simplify the development environment by removing the need for an npm private repo (e.g. verdaccio)
    • simplify the dependency management across projects (reduced the development environment footprint from >6GB to 300MB)
    • allow for atomic commits spanning multiple projects, simplifying branching, merging and code reviews

    pnmp in conjunction with rush have features that allow us to efficiently work with monorepos, while still being able to bundle individual modules in the way required by AWS Lambda.