Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(mappings): Add Prowler example #78

Merged
merged 7 commits into from
Oct 23, 2024

Conversation

jfagoagas
Copy link
Contributor

@jfagoagas jfagoagas commented Sep 2, 2024

Context

Fix #75

Description

Add Prowler OCSF Detection Finding v1.3.0 example to the mappings folder.

Copy link
Contributor

@floydtree floydtree left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specific comments are inline.

A few notes:

  1. Can you run the transformed output through OCSF validator? I am seeing a few issues that the validator could have caught. Validator is available in the ocsf-server.
  2. In the samples dir, can you also add a prowler.raw file with the corresponding raw input record?

mappings/markdown/Prowler/samples/prowler.ocsf Outdated Show resolved Hide resolved
mappings/markdown/Prowler/samples/prowler.ocsf Outdated Show resolved Hide resolved
mappings/markdown/Prowler/samples/prowler.ocsf Outdated Show resolved Hide resolved
@jfagoagas
Copy link
Contributor Author

jfagoagas commented Sep 4, 2024

Specific comments are inline.

A few notes:

  1. Can you run the transformed output through OCSF validator? I am seeing a few issues that the validator could have caught. Validator is available in the ocsf-server.

First of all, I apologise if I'm doing something wrong

I'm checking it by calling https://schema.ocsf.io/api/validate with a POST request but I always receive the same error message even having that value in the mapping, as per https://github.com/ocsf/examples/pull/78/files#diff-b322e9b9688ddcf1d96e52975883fa04e35d8fb92468424beedb643e5d663454R114

➜ curl -X POST "https://schema.ocsf.io/api/validate" \
 -H "accept: application/json" \
 -d @prowler.json
{"error":"Missing class_uid"}

➜ jq .class_uid < prowler.json
2004

I tried with several mappings from this repo:

... and I always receive the same error, what am I doing wrong?

The same happens using the Detection Finding sample from https://schema.ocsf.io/sample/1.3.0/classes/detection_finding

P.S.: I've also tried with the api/v2/validate endpoint but the same still happens...

➜  ~ curl -s -X POST "https://schema.ocsf.io/api/v2/validate" \
 -H "accept: application/json" \
 -d "$(curl -s https://schema.ocsf.io/sample/1.3.0/classes/base_event)" \
 | jq
{
  "warnings": [],
  "errors": [
    {
      "error": "attribute_required_missing",
      "message": "Required attribute \"class_uid\" is missing.",
      "attribute": "class_uid",
      "attribute_path": "class_uid"
    }
  ],
  "error_count": 1,
  "warning_count": 0
}
➜  ~ jq .class_uid <<< "$(curl -s https://schema.ocsf.io/sample/1.3.0/classes/base_event)"
  1. In the samples dir, can you also add a prowler.raw file with the corresponding raw input record?

I thought of adding it here but that prowler.raw is not available for our Prowler users since it generates OCSF natively using an internal object which is then mapped to CSV, OCSF, ASFF, HTML, etc. Does it still make sense to include that prowler.raw?

@floydtree
Copy link
Contributor

Regarding your validation issue, you need to pass a content-type header in your curl. Try appending the following -H "content-type: application/json" to your curl. You should be able to see validation results.

One thing that I can see missing is, metadata.profiles[], you would need to populate this array with all the profiles that are used in the event. You'll see validation errors due to this. For e.g. you would cloud profile for this finding. Also, you would need to use the Datetime profile which allows usage of ISO 8601 timestamp strings.


Regarding, this being a natively produced event. Agree, it makes sense to not add a raw sample.

mappings/markdown/Prowler/README.md Outdated Show resolved Hide resolved
mappings/markdown/Prowler/README.md Outdated Show resolved Hide resolved
Signed-off-by: Pepe Fagoaga <[email protected]>
@jfagoagas
Copy link
Contributor Author

Regarding your validation issue, you need to pass a content-type header in your curl. Try appending the following -H "content-type: application/json" to your curl. You should be able to see validation results.

Totally my bad, I didn't see that I was using the accept header 🤦 , thanks!

One thing that I can see missing is, metadata.profiles[], you would need to populate this array with all the profiles that are used in the event. You'll see validation errors due to this. For e.g. you would cloud profile for this finding. Also, you would need to use the Datetime profile which allows usage of ISO 8601 timestamp strings.

Thank you very much for this, now I'm validating all the errors, the only thing I'm not being able to validate is all the timestamps even with the datetime profile.

Regarding, this being a natively produced event. Agree, it makes sense to not add a raw sample.
👌

@floydtree
Copy link
Contributor

Thank you very much for this, now I'm validating all the errors, the only thing I'm not being able to validate is all the timestamps even with the datetime profile.

The datetime profile adds new, shadow fields to the schema. These are any timestamp field suffixed with _dt, you will need to use those fields to represent timestamp strings.

Signed-off-by: Pepe Fagoaga <[email protected]>
@jfagoagas
Copy link
Contributor Author

jfagoagas commented Sep 6, 2024

Thank you very much for this, now I'm validating all the errors, the only thing I'm not being able to validate is all the timestamps even with the datetime profile.

The datetime profile adds new, shadow fields to the schema. These are any timestamp field suffixed with _dt, you will need to use those fields to represent timestamp strings.

Thank you for the insight, just added!

We have some warnings that we need to review but at least 0 errors using the validation API.

@jfagoagas
Copy link
Contributor Author

@floydtree everything was addressed, thank you for your review!

@zschmerber zschmerber merged commit db20813 into ocsf:main Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add DetectionFinding example from Prowler
4 participants