feat: 295 monorepo directory structure design proposal

CONTRIBUTING.md

@@ -73,7 +73,9 @@ For workflow diagrams, see the [diagrams](./docs/workflows/) under the `docs` fo
 #### Code structure
 
 - `actions` - Provides specific logic for `trestle-bot` tasks that are packaged as Actions. See [README.md](./actions/README.md) for more information.
-- `entrypoints` - Provides top level logic for specific user-facing tasks. These tasks are not necessarily related in any way so they are not organized into a hierarchical command structure, but they do inherit logic and flags from a base class.
+- `cli` - Provides top level logic for specific user-facing tasks. These tasks are not necessarily related so they are not organized into a hierarchical command structure, but they do share some common modules.
+- `cli/commands` - Provides top level logic for commands and their associated subcommands. The commands are accessed by the single entrypoint `root.py`.
+- `cli/options` - Provides command line options and arguments that are frequently used within `cli/commands`.
 - `provider.py, github.py, and gitlab.py` - Git provider abstract class and concrete implementations for interacting with the API.
 - `tasks` - Pre-tasks can be configured before the main git logic is run. Any task that does workspace management should go here.
 - `tasks/authored` - The `authored` package contains logic for managing authoring tasks for single instances of a top-level OSCAL model. These encapsulate logic from the `compliance-trestle` library and allows loose coupling between `tasks` and `authored` types.

README.md

@@ -7,33 +7,33 @@
 
 
 
-trestle-bot assists users in leveraging [Compliance-Trestle](https://github.com/oscal-compass/compliance-trestle) in CI/CD workflows for [OSCAL](https://github.com/usnistgov/OSCAL) formatted compliance content management.
+trestle-bot is a CLI tool that assists users in leveraging [Compliance-Trestle](https://github.com/oscal-compass/compliance-trestle) in CI/CD workflows for [OSCAL](https://github.com/usnistgov/OSCAL) formatted compliance content management.
 
 > WARNING: This project is currently under initial development. APIs may be changed incompatibly from one commit to another.
 
 ## Getting Started
 
 ### Available Commands
 
-The `autosync` command will sync trestle-generated Markdown files to OSCAL JSON files in a trestle workspace. All content under the provided markdown directory when the action is run will be transformed. This action supports all top-level models [supported by compliance-trestle for authoring](https://oscal-compass.github.io/compliance-trestle/tutorials/ssp_profile_catalog_authoring/ssp_profile_catalog_authoring/).
+The `autosync` command will sync trestle-generated Markdown files to OSCAL JSON files in a trestle workspace. All content under the provided markdown directory will be transformed when the action is run. This action supports all top-level models [supported by compliance-trestle for authoring](https://oscal-compass.github.io/compliance-trestle/tutorials/ssp_profile_catalog_authoring/ssp_profile_catalog_authoring/).
 
 The `rules-transform` command can be used when managing [OSCAL Component Definitions](https://pages.nist.gov/OSCAL-Reference/models/v1.1.1/component-definition/json-outline/) in a trestle workspace. The action will transform rules defined in the rules YAML view to an OSCAL Component Definition JSON file.
 
-The `create-cd` command can be used to create a new [OSCAL Component Definition](https://pages.nist.gov/OSCAL-Reference/models/v1.1.1/component-definition/json-outline/) in a trestle workspace. The action will create a new Component Definition JSON file and corresponding directories that contain rules YAML files and trestle-generated Markdown files. This action prepares the workspace for use with the `rules-transform` and `autosync` actions.
+The `create compdef` command can be used to create a new [OSCAL Component Definition](https://pages.nist.gov/OSCAL-Reference/models/v1.1.1/component-definition/json-outline/) in a trestle workspace. The action will create a new Component Definition JSON file and corresponding directories that contain rules YAML files and trestle-generated Markdown files. This action prepares the workspace for use with the `rules-transform` and `autosync` actions.
 
-The `sync-upstreams` command can be used to sync and validate upstream OSCAL content stored in a git repository to a local trestle workspace. Which content is synced is determined by the `include_model_names` and `exclude_model_names` inputs.
+The `sync-upstreams` command can be used to sync and validate upstream OSCAL content stored in a git repository to a local trestle workspace. The inputs `include_models` and `exclude_models` determine which content is synced to the trestle workspace.
 
-The `create-ssp` command can be used to create a new [OSCAL System Security Plans](https://pages.nist.gov/OSCAL-Reference/models/v1.1.1/system-security-plan/json-outline/) (SSP) in a trestle workspace. The action will create a new SSP JSON file and corresponding directories that contain trestle-generated Markdown files. This action prepares the workspace for use with the `autosync` action by creating or updating the `ssp-index.json` file. The `ssp-index.json` file is used to track the relationships between the SSP and the other OSCAL content in the workspace for the `autosync` action.
+The `create ssp` command can be used to create a new [OSCAL System Security Plans](https://pages.nist.gov/OSCAL-Reference/models/v1.1.1/system-security-plan/json-outline/) (SSP) in a trestle workspace. The action will create a new SSP JSON file and corresponding directories that contain trestle-generated Markdown files. This action prepares the workspace for use with the `autosync` action by creating or updating the `ssp-index.json` file. The `ssp-index.json` file is used to track the relationships between the SSP and the other OSCAL content in the workspace for the `autosync` action.
 
 Below is a table of the available commands and their current availability as a GitHub Action:
 
-| Command            | Available as a GitHub Action |
-|--------------------|------------------------------|
-| `autosync`         | ✓                     |
-| `rules-transform`  | ✓                     |                   
-| `create-cd`        | ✓                     |
-| `sync-upstreams`   | ✓                     |
-| `create-ssp`       |                              |
+| Command           | Available as a GitHub Action |
+|-------------------|------------------------------|
+| `autosync`        | ✓                     |
+| `rules-transform` | ✓                     |                   
+| `create compdef`  | ✓                     |
+| `sync-upstreams`  | ✓                     |
+| `create ssp`      |                              |
 
 For detailed documentation on how to use each action, see the README.md in each folder under [actions](./actions/).
 
@@ -47,7 +47,7 @@ provider information is supported for GitHub Actions (GitHub) and GitLab CI (Git
 
 ### Run as a Container
 
-> Note: When running the commands in a container, all are prefixed with `trestlebot` (e.g. `trestlebot-autosync`). The default entrypoint for the container is the autosync command.
+> Note: When running the commands in a container, all are prefixed with `trestlebot` (e.g. `trestlebot autosync`). The default entrypoint for the container is the autosync command.
 
 Build and run the container locally:
 
@@ -72,4 +72,4 @@ This project is licensed under the Apache 2.0 License - see the [LICENSE.md](LIC
 
 ## Troubleshooting
 
-See [TROUBLESHOOTING.md](./TROUBLESHOOTING.md) for troubleshooting tips.
+See [TROUBLESHOOTING.md](./TROUBLESHOOTING.md) for troubleshooting tips.

pyproject.toml

@@ -48,6 +48,7 @@ flake8-print = "^5.0.0"
 pre-commit = "^3.4.0"
 mkdocs-material = "^9.5.43"
 markdown-include = "^0.8.1"
+types-pyyaml = "^6.0.12.20240917"
 
 [tool.poetry.group.tests]
 optional = true

tests/trestlebot/cli/test_autosync_cmd.py

@@ -0,0 +1,94 @@
+# SPDX-License-Identifier: Apache-2.0
+# Copyright (c) 2024 Red Hat, Inc.
+
+
+"""Testing module for trestlebot autosync command"""
+import pathlib
+from typing import Tuple
+
+from click.testing import CliRunner
+from git import Repo
+
+from trestlebot.cli.commands.autosync import autosync_cmd
+from trestlebot.cli.config import TrestleBotConfig, write_to_file
+
+
+def test_invalid_oscal_model(tmp_repo: Tuple[str, Repo]) -> None:
+    """Test invalid OSCAl model option."""
+
+    repo_path, _ = tmp_repo
+    runner = CliRunner()
+    result = runner.invoke(
+        autosync_cmd,
+        [
+            "--oscal-model",
+            "invalid",
+            "--repo-path",
+            repo_path,
+            "--markdown-dir",
+            "markdown",
+            "--branch",
+            "main",
+            "--committer-name",
+            "Test User",
+            "--committer-email",
+            "[email protected]",
+        ],
+    )
+    assert "Invalid value for '--oscal-model'" in result.output
+    assert result.exit_code == 2
+
+
+def test_missing_ssp_index_file_option(tmp_repo: Tuple[str, Repo]) -> None:
+    """Test missing ssp_index_file option for autosync ssp."""
+    repo_path, _ = tmp_repo
+    runner = CliRunner()
+    cmd_options = [
+        "--oscal-model",
+        "ssp",
+        "--repo-path",
+        repo_path,
+        "--markdown-dir",
+        "markdown",
+        "--branch",
+        "main",
+        "--committer-name",
+        "Test User",
+        "--committer-email",
+        "[email protected]",
+    ]
+    result = runner.invoke(autosync_cmd, cmd_options)
+    assert result.exit_code == 1
+    assert "Missing option '--ssp-index-file'" in result.output
+
+
+def test_missing_markdown_dir_option(tmp_repo: Tuple[str, Repo]) -> None:
+    # When no markdown_dir setting in trestlebot config file.
+    repo_path, _ = tmp_repo
+    runner = CliRunner()
+    filepath = pathlib.Path(repo_path).joinpath("config.yml")
+    config_obj = TrestleBotConfig(repo_path=repo_path)
+    write_to_file(config_obj, filepath)
+    cmd_options = [
+        "--oscal-model",
+        "compdef",
+        "--repo-path",
+        repo_path,
+        "--branch",
+        "main",
+        "--committer-name",
+        "Test User",
+        "--committer-email",
+        "[email protected]",
+        "--config",
+        str(filepath),
+    ]
+    result = runner.invoke(autosync_cmd, cmd_options)
+    assert result.exit_code == 2
+    assert "Error: Missing option '--markdown-dir'" in result.output
+
+    # With 'markdown_dir' setting in config.yml
+    config_obj = TrestleBotConfig(markdown_dir="markdown")
+    write_to_file(config_obj, filepath)
+    result = runner.invoke(autosync_cmd, cmd_options)
+    assert result.exit_code == 0

tests/trestlebot/cli/test_config.py

@@ -0,0 +1,80 @@
+# SPDX-License-Identifier: Apache-2.0
+# Copyright (c) 2024 Red Hat, Inc.
+
+
+"""Unit tests for CLI config module"""
+import pathlib
+
+import pytest
+import yaml
+
+from trestlebot.cli.config import (
+    TrestleBotConfig,
+    TrestleBotConfigError,
+    UpstreamsConfig,
+    load_from_file,
+    make_config,
+    write_to_file,
+)
+
+
+@pytest.fixture
+def config_obj() -> TrestleBotConfig:
+    return TrestleBotConfig(
+        repo_path="/tmp",
+        markdown_dir="markdown",
+        upstreams=UpstreamsConfig(sources=["repo@main"]),
+    )
+
+
+def test_invalid_config_raises_errors() -> None:
+    """Test create config with invalid directory to raise error."""
+
+    with pytest.raises(TrestleBotConfigError) as ex:
+        _ = make_config(dict(repo_path="0"))
+
+    assert (
+        str(ex.value)
+        == "Invalid config value for repo_path. Path does not point to a directory."
+    )
+
+
+def test_make_config_raises_no_errors(tmp_init_dir: str) -> None:
+    """Test create a valid config object."""
+    values = {
+        "repo_path": tmp_init_dir,
+        "markdown_dir": "markdown",
+        "committer_name": "committer-name",
+        "committer_email": "committer-email",
+        "upstreams": {"sources": ["https://test@main"], "skip_validation": True},
+    }
+    config = make_config(values)
+    assert isinstance(config, TrestleBotConfig)
+    assert config.upstreams is not None
+    assert config.upstreams.sources == ["https://test@main"]
+    assert config.upstreams.skip_validation is True
+    assert config.repo_path == pathlib.Path(tmp_init_dir)
+    assert config.markdown_dir == values["markdown_dir"]
+    assert config.committer_name == values["committer_name"]
+    assert config.committer_email == values["committer_email"]
+
+
+def test_config_write_to_file(config_obj: TrestleBotConfig, tmp_init_dir: str) -> None:
+    """Test config is written to yaml file."""
+    filepath = pathlib.Path(tmp_init_dir).joinpath("config.yml")
+    write_to_file(config_obj, filepath)
+    with open(filepath, "r") as f:
+        yaml_data = yaml.safe_load(f)
+
+    assert yaml_data == config_obj.to_yaml_dict()
+
+
+def test_config_load_from_file(config_obj: TrestleBotConfig, tmp_init_dir: str) -> None:
+    """Test config is read from yaml file into config object."""
+    filepath = pathlib.Path(tmp_init_dir).joinpath("config.yml")
+    with filepath.open("w") as config_file:
+        yaml.dump(config_obj.to_yaml_dict(), config_file)
+
+    config = load_from_file(filepath)
+    assert isinstance(config, TrestleBotConfig)
+    assert config == config_obj