Introducing EmptyElement to avoid showing "invalid" elements on diagrams

[oleh-mykytyuk]

In the issue #222 was mentioned creation of the spurious "invalid" elements. Also, in the PR #223 tried to override the name of auto-generated elements.

This PR goal is to skip using "invalid" elements on DFD and SEQ diagrams. Such elements can be a problem when a lot of Finding used for overriding findings (add additional information to the threats). This approach is mentioned in the official documentation (README file), but leads to problems on the diagrams.

What is done:

• added EmptyElement (just to be used as non-optional property for Finding)
• skipped generation EmptyElement on DFD and SEQ diagrams (skip include EmptyElement in diagram generation)

[raphaelahrens]

I like the commit, since it clarifies the intend.

[raphaelahrens]

This seems backwards. I have not checked yet, but if this is correct it should be refactored to represent the intention better.

[oleh-mykytyuk]

Added description

[raphaelahrens]

Ahh after reading the code and your comment self._is_drawn is an indicator to mark if the element has already been drawn.
I read it as "is it supposed to be drawn" . So I was just confused by the name.

But now I wonder if it is necessary as a member variable at all and if it could be moved into a drawing function. But this would require a mayor change I guess, since the drawing is split between all the dfd methods. I mean they are very similar and have duplicated code.

[oleh-mykytyuk]

I can exclude this element from drawing procedure using if clause (like if not isinstance(e, EmptyElement) )...
Or introduce a variable and appropriate function/property like def is_visible()
Just tell me what is better to do in your opinion!

[raphaelahrens]

You can use isinstance with a tuple of types.

instance(e, (A,B,C))

https://docs.python.org/3/library/functions.html#isinstance

[oleh-mykytyuk]

Fixed

[oleh-mykytyuk]

Included this situation in tests

[izar]

hey, thanks for the PR. I am a bit full this week but will be taking a look asap.

[oleh-mykytyuk]

hey, thanks for the PR. I am a bit full this week but will be taking a look asap.

Greetings! Sorry for disturbing, but how about this PR?

[izar]

sorry, still prioritizing work, but i have not forgotten this!

[neilpvirtualitics]

Hey, thanks for working on this!
I am using pytm, and just noticed this 'element_invalid_" issue on my diagram, when using findings.

I've tried a couple of workarounds, but I still have two problems (with 1.3.1), the element_invalid_ issue, and the text of my "response" member that I add to my Findings, not appearing in the report. I added a response field to my template like this:

<details>
  <summary>   {{{{item.id}}}}  --  {{{{item.threat_id}}}}   --   {{{{item.description}}}}</summary>
  <h6> Targeted Element </h6>
  <p> {{{{item.target}}}} </p>
  <h6> Severity </h6>
  <p>{{{{item.severity}}}}</p>
  <h6>Example Instances</h6>
  <p>{{{{item.example}}}}</p>
  <h6>Mitigations</h6>
  <p>{{{{item.mitigations}}}}</p>
  <h6>References</h6>
  <p>{{{{item.references}}}}</p>
  <h6>Response</h6>
  <p>{{{{item.response}}}}</p>
  &emsp;
</details>

but in the html, that item.response is empty, even though I supply a response="""To Mitigate: foo""", in my Finding().

One of the workarounds I thought I'd try for the invalid element issue, is to redirect the model.py --dfd-colormap output to jq to try to use jq to edit-out the element_invalid_ elements, but this output is not json. It's close to json, but jq won't deal with it.

[izar]

ho @oleh-mykytiuk will you be able to address @neilpvirtualitics comment or should i review/merge as is ?

[oleh-mykytiuk]

Hey, thanks for working on this! I am using pytm, and just noticed this 'element_invalid_" issue on my diagram, when using findings.

I've tried a couple of workarounds, but I still have two problems (with 1.3.1), the element_invalid_ issue, and the text of my "response" member that I add to my Findings, not appearing in the report. I added a response field to my template like this:

<details>
  <summary>   {{{{item.id}}}}  --  {{{{item.threat_id}}}}   --   {{{{item.description}}}}</summary>
  <h6> Targeted Element </h6>
  <p> {{{{item.target}}}} </p>
  <h6> Severity </h6>
  <p>{{{{item.severity}}}}</p>
  <h6>Example Instances</h6>
  <p>{{{{item.example}}}}</p>
  <h6>Mitigations</h6>
  <p>{{{{item.mitigations}}}}</p>
  <h6>References</h6>
  <p>{{{{item.references}}}}</p>
  <h6>Response</h6>
  <p>{{{{item.response}}}}</p>
  &emsp;
</details>

but in the html, that item.response is empty, even though I supply a response="""To Mitigate: foo""", in my Finding().

One of the workarounds I thought I'd try for the invalid element issue, is to redirect the model.py --dfd-colormap output to jq to try to use jq to edit-out the element_invalid_ elements, but this output is not json. It's close to json, but jq won't deal with it.

Hi! This is another issue. I saw the same. It can be overridden by using the property example in the template and Finding. this is because not all fields are used during report generation.
e.g.:

fundings.py

finding_DO01_api_gw_used = Finding(
    threat_id="DO01",
    example="Optional API Gateway can be used to check and limit requests",
)

template.md:

## Findings

{elements:repeat:{{item.findings:if:

### {{item.name}}
{{item.findings:repeat:
---

{{{{item.description}}}}

| # | ThreatID | Severity               | Description              |Status                |
|:-:|:--------:|------------------------|:-------------------------|:----------------------|
| {{{{item.id}}}} | {{{{item.threat_id}}}} | {{{{item.severity}}}} | {{{{item.description}}}} | {{{{item.example}}}} |

**Mitigations**: {{{{item.mitigations}}}}

**References**: {{{{item.references}}}}

}}}}}

I may open another PR to fix this. This PR should not affect other issues or problems.

[oleh-mykytyuk]

ho @oleh-mykytiuk will you be able to address @neilpvirtualitics comment or should i review/merge as is ?

Hi @izar ! I replied ;) I think that this is another issue and should be fixed by another PR.