feat: populate cac content product name as component title for CPLYTM-380 and CPLYTM-381 #402

huiwangredhat · 2024-12-25T06:35:22Z

Description

Populate Cac content product name as OSCAL component title

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How has this been tested?

Added the unit test case.
Locally test
I tested it locally. It could create the OSCAL component definition with the title of the Cac content product name. The trestlebot CLI work example as the following:

poetry run trestlebot sync-cac-content --repo-path ~/trestle-repo --dry-run --committer-email [email protected] --committer-name tester --branch main --cac-content-root ~/content --product ocp4 --cac-profile ~/content/controls/nist_ocp4.yml --oscal-profile nist_rev5_800_53

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
Any dependent changes have been merged and published in downstream modules

Signed-off-by: Sophia Wang <[email protected]>

qduanmu · 2024-12-27T06:45:34Z

trestlebot/cli/commands/sync_cac_content.py

+        comp_description=component_description,
+        comp_type=component_definition_type,
+        filter_by_profile=filter_by_profile,
+    )


Is this to create a new component definition directly? I saw an AC item of CPLYTM-217:

If an existing component definition is present in the workspace you have to complete in place updates, it is not a recreation (using trestle sdk tooling)

Yes, it will create a new component definition. I think the AC is out of the CPLYTM-217 scope. It seems for whole EPIC. Also, the Fourth AC is not for CPLYTM-217.

The rule and parameter information from yaml files in cac are present as OSCAL properties

If there is an existing component definition in the workplace, I have no the component's uuid and title mapping, it seems that the component element can't be obtained correctly. It's difficult to say the component exists.

I have thought about this issue again. It should be updated. But I don't know how to update it at that moment.

I added a new function to update the existing component definition.

qduanmu · 2024-12-27T06:49:37Z

trestlebot/cli/utils.py

@@ -35,3 +41,16 @@ def run_bot(pre_tasks: List[TaskBase], kwargs: Dict[Any, Any]) -> BotResults:
        ),
        dry_run=kwargs.get("dry_run", False),
    )
+
+
+def get_component_title(product_name: str, cac_path: str) -> str:


This function is for CaC content specially, utils module is used for common functions for cli commands. Since there are following work based on this to attach component props, how about moving to a transformation workflow?

Sounds good. How about move it to a new file trestlebot/transformers/cac_transformer.py?

Signed-off-by: Sophia Wang <[email protected]>

…xists Signed-off-by: Sophia Wang <[email protected]>

marcusburghardt

I have just minor comments but already tested locally and it is working fine.

marcusburghardt · 2025-01-06T10:47:20Z

trestlebot/cli/commands/sync_cac_content.py

+    product = kwargs["product"]
+    cac_content_root = kwargs["cac_content_root"]
+    component_title = get_component_title(product, cac_content_root)
+    component_description = kwargs["product"]


Not blocking but since we are reading the product properties, we could use the full_name as description. This can be done in a separate PR anyways.

I updated the component description, and it works. But it can't pass the test with python 3.8. The reason is that Python 3.8 and earlier used Tuple[str, str] from the typing module for type hinting.

I fixed the test error.

marcusburghardt · 2025-01-06T11:08:34Z

trestlebot/tasks/authored/compdef.py

+            with open(ofile, "r", encoding="utf-8") as f:
+                data = json.load(f)
+            for component in data["component-definition"]["components"]:
+                if component.get("title") == get_component_title(


Could it be compared with oscal_component.title to avoid calling this function twice?

Good catch. Updated.

marcusburghardt · 2025-01-06T11:10:28Z

trestlebot/tasks/authored/compdef.py

+                    logger.info("Update the exsisting component definition.")
+                    # Need to update props parts if the rules updated
+                    # Update the version and last modify time
+                    update_component_definition(ofile)


I assume this function is here without conditions just temporarily, correct? Or is it necessary to update version and timestamp whenever the command is called?

Yes, I think it should update the rules/parameters part according to the specific requirement. At that moment, I am not sure how to update that part, but the version and last-modified should be updated together with all the updates.

Signed-off-by: Sophia Wang <[email protected]>

marcusburghardt

LGTM. Thanks for the updates.

qduanmu · 2025-01-06T02:42:04Z

pyproject.toml

@@ -29,6 +29,7 @@ github3-py = "^4.0.1"
 python-gitlab = "^4.2.0"
 ruamel-yaml = "^0.18.5"
 pydantic = "^2.0.0"
+ssg = {git = "https://github.com/ComplianceasCode/content"}


From the ssg guide, it recommends to use a stable version, it can change unexpectedly thus unstable when installing from master. e.g.,
`https://github.com/ComplianceasCode/[email protected]

Thanks. The latest tag of content is v0.1.75, released on 2024 Nov 11. We need the latest SSG extension in the master. Let's keep it now. It could be updated when the new tag contains the latest changes.

qduanmu · 2025-01-06T03:07:44Z

trestlebot/cli/commands/sync_cac_content.py

+    authored_comp: AuthoredComponentDefinition = AuthoredComponentDefinition(
+        trestle_root=working_dir,
+    )
+    authored_comp.create_update_cac_compdef(


The create or update in this function is just in local, should these updates be pushed to remote like that in the compdef create command? I'd like to see your opinions here @jpower432 and Marcus.

I added the sync_cac_content_task to push the local change to the remote.

Signed-off-by: Sophia Wang <[email protected]>

huiwangredhat force-pushed the 217 branch 4 times, most recently from d5458f9 to 183b12f Compare December 25, 2024 08:32

huiwangredhat requested a review from qduanmu December 25, 2024 08:35

huiwangredhat force-pushed the 217 branch 2 times, most recently from 66cd46b to 1e9f047 Compare December 26, 2024 02:15

feat: populate cac content product name as component title

2ae3bb7

Signed-off-by: Sophia Wang <[email protected]>

huiwangredhat force-pushed the 217 branch from 1e9f047 to 2ae3bb7 Compare December 26, 2024 02:51

Add unitest for populate cac product nameto component title

4343fe4

Signed-off-by: Sophia Wang <[email protected]>

huiwangredhat force-pushed the 217 branch from 0b7c587 to 4343fe4 Compare December 26, 2024 07:03

huiwangredhat changed the title ~~feat: populate cac content product name as component title~~ CPLYTM-380 CPLYTM-381: populate cac content product name as component title Dec 26, 2024

huiwangredhat changed the title ~~CPLYTM-380 CPLYTM-381: populate cac content product name as component title~~ feat: populate cac content product name as component title Dec 26, 2024

qduanmu reviewed Dec 27, 2024

View reviewed changes

Move the ssg connections from utils to cac transformer

f3f26fc

Signed-off-by: Sophia Wang <[email protected]>

huiwangredhat force-pushed the 217 branch from 83f4539 to f3f26fc Compare December 27, 2024 08:37

gvauter changed the title ~~feat: populate cac content product name as component title~~ feat: CPLYTM-380 populate cac content product name as component title Jan 2, 2025

gvauter changed the title ~~feat: CPLYTM-380 populate cac content product name as component title~~ feat: cplytm-380 populate cac content product name as component title Jan 2, 2025

huiwangredhat force-pushed the 217 branch 4 times, most recently from 88422ca to dbaed45 Compare January 3, 2025 08:25

Add function to only update not recreate component definition if it e…

810483f

…xists Signed-off-by: Sophia Wang <[email protected]>

huiwangredhat force-pushed the 217 branch from dbaed45 to 810483f Compare January 3, 2025 08:41

qduanmu requested review from jpower432 and mprpic and removed request for mprpic January 6, 2025 02:37

marcusburghardt suggested changes Jan 6, 2025

View reviewed changes

Update the component description as the product full name

cfa689d

Signed-off-by: Sophia Wang <[email protected]>

huiwangredhat force-pushed the 217 branch from 5a4a99c to cfa689d Compare January 6, 2025 12:26

huiwangredhat changed the title ~~feat: cplytm-380 populate cac content product name as component title~~ CPLYTM-380 CPLYTM-381 feat: populate cac content product name as component title Jan 6, 2025

marcusburghardt approved these changes Jan 6, 2025

View reviewed changes

huiwangredhat changed the title ~~CPLYTM-380 CPLYTM-381 feat: populate cac content product name as component title~~ CPLYTM-380, CPLYTM-381 feat: populate cac content product name as component title Jan 6, 2025

qduanmu reviewed Jan 6, 2025

View reviewed changes

huiwangredhat changed the title ~~CPLYTM-380, CPLYTM-381 feat: populate cac content product name as component title~~ CPLYTM-380, CPLYTM-381-feat: populate cac content product name as component title Jan 6, 2025

huiwangredhat changed the title ~~CPLYTM-380, CPLYTM-381-feat: populate cac content product name as component title~~ CPLYTM-380-CPLYTM-381-populate cac content product name as component title Jan 6, 2025

huiwangredhat changed the title ~~CPLYTM-380-CPLYTM-381-populate cac content product name as component title~~ feat: populate cac content product name as component title Jan 6, 2025

huiwangredhat changed the title ~~feat: populate cac content product name as component title~~ feat: CPLYTM-380 populate cac content product name as component title Jan 6, 2025

huiwangredhat changed the title ~~feat: CPLYTM-380 populate cac content product name as component title~~ feat: populate cac content product name as component title for CPLYTM-380 and CPLYTM-381 Jan 6, 2025

add the sync cac content task to push local change to remote

f1e4148

Signed-off-by: Sophia Wang <[email protected]>

qduanmu approved these changes Jan 7, 2025

View reviewed changes

huiwangredhat merged commit 371ab6b into complytime:main Jan 7, 2025
10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: populate cac content product name as component title for CPLYTM-380 and CPLYTM-381 #402

feat: populate cac content product name as component title for CPLYTM-380 and CPLYTM-381 #402

huiwangredhat commented Dec 25, 2024 •

edited

Loading

qduanmu Dec 27, 2024

huiwangredhat Dec 27, 2024 •

edited

Loading

huiwangredhat Jan 2, 2025

huiwangredhat Jan 3, 2025

qduanmu Dec 27, 2024

huiwangredhat Dec 27, 2024

marcusburghardt left a comment

marcusburghardt Jan 6, 2025

huiwangredhat Jan 6, 2025

huiwangredhat Jan 6, 2025

marcusburghardt Jan 6, 2025

huiwangredhat Jan 6, 2025

marcusburghardt Jan 6, 2025

huiwangredhat Jan 6, 2025

marcusburghardt left a comment

qduanmu Jan 6, 2025

huiwangredhat Jan 7, 2025

qduanmu Jan 6, 2025

huiwangredhat Jan 7, 2025

feat: populate cac content product name as component title for CPLYTM-380 and CPLYTM-381 #402

feat: populate cac content product name as component title for CPLYTM-380 and CPLYTM-381 #402

Conversation

huiwangredhat commented Dec 25, 2024 • edited Loading

Description

Type of change

How has this been tested?

Checklist

Choose a reason for hiding this comment

huiwangredhat Dec 27, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marcusburghardt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marcusburghardt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

huiwangredhat commented Dec 25, 2024 •

edited

Loading

huiwangredhat Dec 27, 2024 •

edited

Loading