-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #110 from f5devcentral/matt-aug24
change-lab-order
- Loading branch information
Showing
8 changed files
with
279 additions
and
279 deletions.
There are no files selected for viewing
Binary file not shown.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,105 +1,28 @@ | ||
| Enable API traffic discovery | ||
| ============================ | ||
| Enable API code scanning discovery (under construction) | ||
| ======================================================= | ||
|
|
||
| In the previous section, we enabled ``API Validation`` in order to enforce protection on ``what we know`` from the OpenAPI Spec file. | ||
| But we kept the ``Fall Through Mode`` to ``Allow`` so that we do not break the application or impact business down when DevOps push a new version of the API, but SecOps are not ready or up to date. | ||
| F5 Solutions can protect API during the full API Develoment Lifecycle. It means F5 can detect and learn API endpoints when developers push the code into the repository. | ||
| This is call **API Code Scanning Discovery** | ||
|
|
||
| The ``API Discovery`` will provide visility for SecOps in order to see this ``Drift``. This Drift is the difference between ``what we know`` and ``what we see / what is consumed`` | ||
| The sentence application source code is available into our public Github repository : https://github.com/MattDierick/sentence-source-code | ||
|
|
||
| .. image:: ../pictures/slide-api-discovery.png | ||
| :align: center | ||
| :scale: 40% | ||
| Enable Code Base Integration | ||
| ---------------------------- | ||
|
|
||
| But OWASP Top10 requires also to provide visibility on PII (Personal Identifiable Information) in order to avoid Data Leakage. To do so, we will enable ``Sensitive Data Disvovery`` | ||
| * Goto Web App & API Protection > API Management > Code Base Ingration | ||
| * Add a new code base integration profile | ||
|
|
||
| Enable API Endpoint Discovery | ||
| ----------------------------- | ||
| * Name: ``github-sentence`` | ||
| * Code base: ``Github Integration`` | ||
| * Github Name: ``f5xclab`` | ||
| * GitHub Personal Access Token: paste and blindfold below Token | ||
|
|
||
| * Edit your Load Balancer again, go to API Protection and enable ``API Discovery`` (keep the default settings) | ||
| .. code-block:: bash | ||
| sdlkjflksdjglkdfshglkjdflgjldksfgjlksd | ||
| .. image:: ../pictures/enable-api-discovery.png | ||
| :align: left | ||
| :scale: 40% | ||
|
|
||
| Enable Sensitive Data Discovery | ||
| ------------------------------- | ||
|
|
||
| OWASP Top10 API requires to detect and discover sensitive datas in Requests and Responses. F5 Distributed Cloud supports this and provides a predefined list (+400) of known PII (Personal Identifiable Information), such as: | ||
|
|
||
| * credit card number | ||
| * US Social Security Number | ||
| * IP address | ||
|
|
||
| .. note:: By default, a list is already assigned to the Load Balancer | ||
|
|
||
| .. image:: ../pictures/default-pii.png | ||
| :align: left | ||
| :scale: 50% | ||
|
|
||
|
|
||
| But if you want to detect your own PII, such as: | ||
|
|
||
| * Country Social Security Number | ||
| * Mobile Phone Number | ||
| * Etc ... | ||
|
|
||
| You must create your own patterns. | ||
|
|
||
| Create custom Sensitive Data Discovery | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| * In Sensitive Data Discovery, select ``Custom`` | ||
| * Add a new item | ||
|
|
||
| * Give a name: custom-frenchies | ||
| * Select the Compliance Frameworks required for this API Application. We select ``PCI-DSS`` ``GDPR`` | ||
|
|
||
| .. note:: By selecting PCI-DSS and GDPR, all data patterns classified as PCI-DSS and GDPR will be added. | ||
| * But now, we want to add custom patterns to detect frenchy sensitive datas | ||
| * Configure ``Defined Custom Sensitive Data Types``, and add 2 items | ||
|
|
||
| * Name: ``france-ssn`` | ||
|
|
||
| * Data Type Rules: | ||
|
|
||
| * Value Pattern | ||
| * Regex Value : ``[12][0-9]{2}(0[1-9]|1[0-2])(2[AB]|[0-9]{2})[0-9]{3}[0-9]{3}([0-9]{2})`` | ||
|
|
||
| * Mark as Sensitive Data | ||
| * Mark as PII | ||
| * Relevant Compliance: ``GDPR`` | ||
|
|
||
| .. image:: ../pictures/pii-ssn.png | ||
| :align: left | ||
| :scale: 50% | ||
|
|
||
|
|
||
| * Name: ``france-mobile-phone`` | ||
|
|
||
| * Data Type Rules: | ||
|
|
||
| * Value Pattern | ||
| * Regex Value : ``^(?:(?:\+|00)33|0)\s*[1-9](?:[\s.-]*\d{2}){4}$`` | ||
|
|
||
| * Mark as Sensitive Data | ||
| * Mark as PII | ||
| * Relevant Compliance: ``GDPR`` | ||
|
|
||
| * Apply and Save your LB config | ||
|
|
||
|
|
||
| Run the traffic generator script | ||
| -------------------------------- | ||
|
|
||
| It is time to run a traffic generator script to populate the logs and the AI/ML engines. | ||
|
|
||
| * SSH or WEBSSH to the Jumphost | ||
| * Run this script into /home/ubuntu/api-protection-lab folder | ||
|
|
||
| .. code-block:: none | ||
| cd /home/ubuntu/api-protection-lab | ||
| bash api-all.sh sentence-re-$$makeId$$.workshop.emea.f5se.com | ||
| .. image:: ../pictures/code-based-repo.png | ||
| :align: left | ||
|
|
||
| * Save |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,111 +1,105 @@ | ||
| API Discovery outcomes | ||
| ====================== | ||
| Enable API traffic discovery | ||
| ============================ | ||
|
|
||
| .. note:: The "traffic discovery" scheduler runs on a random interval within a two hours time window and therefore it can take up to 2 hours (maximum) to see all results in the Dashboard for the "API Discovery outcomes" lab section. You can also continue with the next lab "Advanced Protection - "JWT validation and access control" (module 3) and continue here later. | ||
| In the previous section, we enabled ``API Validation`` in order to enforce protection on ``what we know`` from the OpenAPI Spec file. | ||
| But we kept the ``Fall Through Mode`` to ``Allow`` so that we do not break the application or impact business down when DevOps push a new version of the API, but SecOps are not ready or up to date. | ||
|
|
||
| Endpoint Discovery | ||
| ------------------ | ||
| The ``API Discovery`` will provide visility for SecOps in order to see this ``Drift``. This Drift is the difference between ``what we know`` and ``what we see / what is consumed`` | ||
|
|
||
| * Goto Web App & API Protection > Overview > Security > Dashboard | ||
| * Click on your Application Load Balancer | ||
| * Click on ``API Endpoints`` to see the endpoints in the the "Table" view. | ||
| .. image:: ../pictures/slide-api-discovery.png | ||
| :align: center | ||
| :scale: 40% | ||
|
|
||
| .. image:: ../pictures/api-endpoints-table.png | ||
| :align: left | ||
| :scale: 50% | ||
|
|
||
| Understand the API Discovery elements | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| On the top left corner, there are 3 important elements: | ||
|
|
||
| * **Inventory** : Endpoints known from the OpenAPI Spec file | ||
|
|
||
| * In our lab, there are 3 endpoints know (adjectives, animals, locations) | ||
| But OWASP Top10 requires also to provide visibility on PII (Personal Identifiable Information) in order to avoid Data Leakage. To do so, we will enable ``Sensitive Data Disvovery`` | ||
|
|
||
| * **Discovered** : Endpoints that the XC platform has discovered/learned from live traffic (known and unknown endpoints) | ||
| * **Shadow** : Endpoints that have been ``Discovered`` but are **NOT PART** of the ``Inventory`` | ||
| Enable API Endpoint Discovery | ||
| ----------------------------- | ||
|
|
||
| You can filter on ``Shadow`` only to show the ``/colors`` endpoint as a Shadow API. | ||
| * Edit your Load Balancer again, go to API Protection and enable ``API Discovery`` (keep the default settings) | ||
|
|
||
| .. image:: ../pictures/shadow.png | ||
| .. image:: ../pictures/enable-api-discovery.png | ||
| :align: left | ||
| :scale: 50% | ||
| :scale: 40% | ||
|
|
||
| Go deeper into the discovery | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| Enable Sensitive Data Discovery | ||
| ------------------------------- | ||
|
|
||
| * Click on the ``/colors`` shadow API endpoint. A pop-up will appear on the right side of the screen. | ||
| * You can see on the top right corner, 2 actions | ||
| OWASP Top10 API requires to detect and discover sensitive datas in Requests and Responses. F5 Distributed Cloud supports this and provides a predefined list (+400) of known PII (Personal Identifiable Information), such as: | ||
|
|
||
| * **API Protection rule** : if you want to block this endpoint. Let's say SecOps have this power to block unknown endpoints. | ||
| * credit card number | ||
| * US Social Security Number | ||
| * IP address | ||
|
|
||
| * **Rate Limiting** : if you want to Rate Limit this endpoint because SecOps don't have the full power and don't want to break the app. | ||
| .. note:: By default, a list is already assigned to the Load Balancer | ||
|
|
||
| * Click on the ``Discovered`` tab and navigate into the sub-menus. You will see all the details discovered by the platform. | ||
| .. image:: ../pictures/default-pii.png | ||
| :align: left | ||
| :scale: 50% | ||
|
|
||
| .. image:: ../pictures/discovered.png | ||
| :align: left | ||
| :scale: 50% | ||
|
|
||
| But if you want to detect your own PII, such as: | ||
|
|
||
| PII Discovery | ||
| ------------- | ||
| * Country Social Security Number | ||
| * Mobile Phone Number | ||
| * Etc ... | ||
|
|
||
| * Click on the ``/animals`` API endpoint. A pop-up will appear on the right side of the screen. | ||
| You must create your own patterns. | ||
|
|
||
| .. image:: ../pictures/pii-1.png | ||
| :align: left | ||
| :scale: 50% | ||
| Create custom Sensitive Data Discovery | ||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| * Click on the ``Discovered`` tab to show discovered sensitive data for requests and responses. | ||
| * In Sensitive Data Discovery, select ``Custom`` | ||
| * Add a new item | ||
|
|
||
| .. image:: ../pictures/pii-2.png | ||
| :align: left | ||
| :scale: 50% | ||
| * Give a name: custom-frenchies | ||
| * Select the Compliance Frameworks required for this API Application. We select ``PCI-DSS`` ``GDPR`` | ||
|
|
||
| .. warning:: Dataguard can obfuscate sensitive PII data in the response but currently not for custom created PII configurations. This feature is in the roadmap. OWASP Top 10 does not require to ``hide`` sensitive data. | ||
| .. note:: By selecting PCI-DSS and GDPR, all data patterns classified as PCI-DSS and GDPR will be added. | ||
|
|
||
| * But now, we want to add custom patterns to detect frenchy sensitive datas | ||
| * Configure ``Defined Custom Sensitive Data Types``, and add 2 items | ||
|
|
||
| Click on the ``Graph`` tab to show the API endpoints in a different view. | ||
| * Name: ``france-ssn`` | ||
|
|
||
| * Data Type Rules: | ||
|
|
||
| * Value Pattern | ||
| * Regex Value : ``[12][0-9]{2}(0[1-9]|1[0-2])(2[AB]|[0-9]{2})[0-9]{3}[0-9]{3}([0-9]{2})`` | ||
|
|
||
| .. image:: ../pictures/octopus.png | ||
| :align: left | ||
| :scale: 50% | ||
| * Mark as Sensitive Data | ||
| * Mark as PII | ||
| * Relevant Compliance: ``GDPR`` | ||
|
|
||
|
|
||
| Authentication Discovery | ||
| ------------------------ | ||
| .. image:: ../pictures/pii-ssn.png | ||
| :align: left | ||
| :scale: 50% | ||
|
|
||
| * Click on an endpoint with an ``Authenticated`` state, like **/api/locations** | ||
|
|
||
| .. image:: ../pictures/authenticated-endpoint.png | ||
| :align: left | ||
| :scale: 50% | ||
| * Name: ``france-mobile-phone`` | ||
|
|
||
| * Data Type Rules: | ||
|
|
||
| * Value Pattern | ||
| * Regex Value : ``^(?:(?:\+|00)33|0)\s*[1-9](?:[\s.-]*\d{2}){4}$`` | ||
|
|
||
| * Click on ``Discovered`` tab and check the Authentication details | ||
| * Mark as Sensitive Data | ||
| * Mark as PII | ||
| * Relevant Compliance: ``GDPR`` | ||
|
|
||
| .. image:: ../pictures/auth-discovery-new.png | ||
| :align: left | ||
| :scale: 50% | ||
| * Apply and Save your LB config | ||
|
|
||
| * Notice that the auth information collected from the OpenAPI Spec file differs from the discovered auth information. If both don't match, a "Security Posture" is raised. | ||
|
|
||
| .. image:: ../pictures/basic-auth.png | ||
| :align: left | ||
| :scale: 50% | ||
| Run the traffic generator script | ||
| -------------------------------- | ||
|
|
||
| AI/ML Security Posture | ||
| ---------------------- | ||
| It is time to run a traffic generator script to populate the logs and the AI/ML engines. | ||
|
|
||
| * Click on an endpoint with the highest ``Risk Score`` | ||
| * And click on the ``Security Posture`` tab | ||
| * Review the recommandations done by the AI/ML engine | ||
| * SSH or WEBSSH to the Jumphost | ||
| * Run this script into /home/ubuntu/api-protection-lab folder | ||
|
|
||
| .. image:: ../pictures/security-posture.png | ||
| :align: left | ||
| :scale: 50% | ||
| .. code-block:: none | ||
| * Click on the ``Evidence`` link to get more details about the logs who generated this security posture. | ||
| cd /home/ubuntu/api-protection-lab | ||
| bash api-all.sh sentence-re-$$makeId$$.workshop.emea.f5se.com | ||
| .. note:: Congratulation, your application is now protected by a modern engine enforcing (validating) what is provided by the developers, but also providing visibility for unkown traffic. |
Oops, something went wrong.