Merge branch 'master' into devel-workflows

.github/pull_request_template.md

@@ -0,0 +1,23 @@
+### Summary
+> Describe your changes.
+
+
+
+### Related issues or links
+> Include links to relevant issues or other pages.
+
+- https://github.com/lyft/cartography/issues/...
+
+
+### Checklist
+
+Provide proof that this works (this makes reviews move faster). Please perform one or more of the following:
+- [ ] Update/add unit or integration tests.
+- [ ] Include a screenshot showing what the graph looked like before and after your changes.
+- [ ] Include console log trace showing what happened before and after your changes.
+
+If you are changing a node or relationship:
+- [ ] Update the [schema](https://github.com/lyft/cartography/tree/master/docs/root/modules) and [readme](https://github.com/lyft/cartography/blob/master/docs/schema/README.md).
+
+If you are implementing a new intel module:
+- [ ] Use the NodeSchema [data model](https://lyft.github.io/cartography/dev/writing-intel-modules.html#defining-a-node).

.pre-commit-config.yaml

@@ -2,46 +2,44 @@ default_language_version:
   # force all unspecified python hooks to run python3
   python: python3
 repos:
-  - repo: https://github.com/pre-commit/pre-commit-hooks
+-   repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v4.6.0
     hooks:
-      - id: check-docstring-first
-      - id: check-executables-have-shebangs
-      - id: check-merge-conflict
-      - id: check-vcs-permalinks
-      - id: check-yaml
-      - id: debug-statements
-      - id: end-of-file-fixer
-      - id: trailing-whitespace
-      - id: detect-private-key
-      - id: check-added-large-files
-  - repo: https://github.com/pycqa/flake8
-    rev: 7.1.0
+    -   id: check-docstring-first
+    -   id: check-executables-have-shebangs
+    -   id: check-merge-conflict
+    -   id: check-vcs-permalinks
+    -   id: check-yaml
+    -   id: debug-statements
+    -   id: end-of-file-fixer
+    -   id: trailing-whitespace
+-   repo: https://github.com/pycqa/flake8
+    rev: 7.1.1
     hooks:
-      - id: flake8
-  - repo: https://github.com/pre-commit/mirrors-autopep8
+    -   id: flake8
+-   repo: https://github.com/pre-commit/mirrors-autopep8
     rev: v2.0.4
     hooks:
-      - id: autopep8
+    -   id: autopep8
         # disable a few rewrites which will cause autopep8 to reflow
-        args: [--in-place, "--ignore=E265,E501,W504"]
-  - repo: https://github.com/asottile/pyupgrade
-    rev: v3.16.0
+        args: [--in-place, '--ignore=E265,E501,W504']
+-   repo: https://github.com/asottile/pyupgrade
+    rev: v3.17.0
     hooks:
       - id: pyupgrade
         args: [--py36-plus]
-  - repo: https://github.com/asottile/add-trailing-comma
+-   repo: https://github.com/asottile/add-trailing-comma
     rev: v3.1.0
     hooks:
       - id: add-trailing-comma
         args: [--py36-plus]
-  - repo: https://github.com/asottile/reorder_python_imports
+-   repo: https://github.com/asottile/reorder_python_imports
     rev: v3.13.0
     hooks:
       - id: reorder-python-imports
         args: [--py3-plus]
-  - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v1.10.1
+-   repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v1.11.2
     hooks:
       - id: mypy
         additional_dependencies:

README.md

@@ -5,16 +5,14 @@ Cartography is a Python tool that consolidates infrastructure assets and the rel
 ![Visualization of RDS nodes and AWS nodes](docs/root/images/accountsandrds.png)
 
 ## Why Cartography?
-Cartography aims to enable a broad set of exploration and automation scenarios.  It is particularly good at exposing otherwise hidden dependency relationships between your service's assets so that you may validate assumptions about security risks.
+Cartography aims to enable a broad set of exploration and automation scenarios. It is particularly good at exposing otherwise hidden dependency relationships between your service's assets so that you may validate assumptions about security risks.
 
-Service owners can generate asset reports, Red Teamers can discover attack paths, and Blue Teamers can identify areas for security improvement.   All can benefit from using the graph for manual exploration through a web frontend interface, or in an automated fashion by calling the APIs.
+Service owners can generate asset reports, Red Teamers can discover attack paths, and Blue Teamers can identify areas for security improvement. All can benefit from using the graph for manual exploration through a web frontend interface, or in an automated fashion by calling the APIs.
 
-Cartography is not the only [security](https://github.com/dowjones/hammer) [graph](https://github.com/BloodHoundAD/BloodHound) [tool](https://github.com/Netflix/security_monkey) [out](https://github.com/vysecurity/ANGRYPUPPY) [there](https://github.com/duo-labs/cloudmapper), but it differentiates itself by being fully-featured yet generic and [extensible](https://lyft.github.io/cartography/dev/writing-analysis-jobs.html) enough to help make anyone better understand their risk exposure, regardless of what platforms they use.  Rather than being focused on one core scenario or attack vector like the other linked tools, Cartography focuses on flexibility and exploration.
+Cartography is not the only [security](https://github.com/dowjones/hammer) [graph](https://github.com/BloodHoundAD/BloodHound) [tool](https://github.com/Netflix/security_monkey) [out](https://github.com/vysecurity/ANGRYPUPPY) [there](https://github.com/duo-labs/cloudmapper), but it differentiates itself by being fully-featured yet generic and [extensible](https://lyft.github.io/cartography/dev/writing-analysis-jobs.html) enough to help make anyone better understand their risk exposure, regardless of what platforms they use. Rather than being focused on one core scenario or attack vector like the other linked tools, Cartography focuses on flexibility and exploration.
 
 You can learn more about the story behind Cartography in our [presentation at BSidesSF 2019](https://www.youtube.com/watch?v=ZukUmZSKSek).
 
-## Install and configure
-Start [here](https://lyft.github.io/cartography/install.html).
 
 ## Supported platforms
 
@@ -34,10 +32,52 @@ Start [here](https://lyft.github.io/cartography/install.html).
 - [Lastpass](https://lyft.github.io/cartography/modules/lastpass/index.html) - users
 - [BigFix](https://lyft.github.io/cartography/modules/bigfix/index.html) - Computers
 - [Duo](https://lyft.github.io/cartography/modules/duo/index.html) - Users, Groups, Endpoints
+- [Kandji](https://lyft.github.io/cartography/modules/kandji/index.html) - Devices
+- [SnipeIT](https://lyft.github.io/cartography/modules/snipeit/index.html) - Users, Assets
+
+
+## Philosophy
+Here are some points that can help you decide if adopting Cartography is a good fit for your problem.
+
+### What Cartography is
+- A simple Python script that pulls data from multiple providers and writes it to a Neo4j graph database in batches.
+- A powerful analysis tool that captures the current snapshot of the environment, building a uniquely useful inventory where you can ask complex questions such as:
+  - Which identities have access to which datastores?
+  - What are the cross-tenant permission relationships in the environment?
+  - What are the network paths in and out of the environment?
+  - What are the backup policies for my datastores?
+- Battle-tested in production by [many companies](#who-uses-cartography).
+- Straightforward to extend with your own custom plugins.
+- Provides a useful data-plane that you can build automation and CSPM (Cloud Security Posture Management) applications on top of.
+
+### What Cartography is not
+- A near-real time capability.
+  - Cartography is not designed for very fast updates. Cartography writes to the database in a batches (not streamed).
+  - Cartography is also limited by how most upstream sources only provide APIs to retrieve assets in a batched manner.
+- By itself, Cartography does not capture data changes over time.
+  - Although we do include a [drift detection](https://lyft.github.io/cartography/usage/drift-detect.html) feature.
+  - It's also possible to implement other processes in your Cartography installation to make this happen.
+
+
+## Install and configure
+
+### Trying out Cartography on a test machine
+Start [here](https://lyft.github.io/cartography/install.html) to set up a test graph and get data into it.
 
+### Setting up Cartography in production
+When you are ready to try it in production, read [here](https://lyft.github.io/cartography/ops.html) for recommendations on getting cartography spun up in your environment.
 
 ## Usage
-Start with our [tutorial](https://lyft.github.io/cartography/usage/tutorial.html). Our [data schema](https://lyft.github.io/cartography/usage/schema.html) is a helpful reference when you get stuck.
+
+### Querying the database directly
+
+![poweruser.png](docs/root/images/poweruser.png)
+
+Now that data is in the graph, you can quickly start with our [querying tutorial](https://lyft.github.io/cartography/usage/tutorial.html). Our [data schema](https://lyft.github.io/cartography/usage/schema.html) is a helpful reference when you get stuck.
+
+### Building applications around Cartography
+Directly querying Neo4j is already very useful as a sort of "swiss army knife" for security data problems, but you can also build applications and data pipelines around Cartography. View this doc on [applications](https://lyft.github.io/cartography/usage/applications.html).
+
 
 ## Community
 
@@ -74,6 +114,7 @@ and follow the instructions to sign the CLA.
 1. [MessageBird](https://messagebird.com)
 1. [Cloudanix](https://www.cloudanix.com/)
 1. [ZeusCloud](https://www.zeuscloud.io/)
+1. [Corelight](https://www.corelight.com/)
 1. {Your company here} :-)
 
 If your organization uses Cartography, please file a PR and update this list. Say hi on Slack too!

cartography/cli.py

@@ -541,6 +541,28 @@ def _build_parser(self):
                 'Required if you are using the Semgrep intel module. Ignored otherwise.'
             ),
         )
+        parser.add_argument(
+            '--snipeit-base-uri',
+            type=str,
+            default=None,
+            help=(
+                'Your SnipeIT base URI'
+                'Required if you are using the SnipeIT intel module. Ignored otherwise.'
+            ),
+        )
+        parser.add_argument(
+            '--snipeit-token-env-var',
+            type=str,
+            default=None,
+            help='The name of an environment variable containing token with which to authenticate to SnipeIT.',
+        )
+        parser.add_argument(
+            '--snipeit-tenant-id',
+            type=str,
+            default=None,
+            help='An ID for the SnipeIT tenant.',
+        )
+
         return parser
 
     def main(self, argv: str) -> int:
@@ -745,6 +767,26 @@ def main(self, argv: str) -> int:
         else:
             config.cve_api_key = None
 
+        # SnipeIT config
+        if config.snipeit_base_uri:
+            if config.snipeit_token_env_var:
+                logger.debug(
+                    "Reading SnipeIT API token from environment variable '%s'.",
+                    config.snipeit_token_env_var,
+                )
+                config.snipeit_token = os.environ.get(config.snipeit_token_env_var)
+            elif os.environ.get('SNIPEIT_TOKEN'):
+                logger.debug(
+                    "Reading SnipeIT API token from environment variable 'SNIPEIT_TOKEN'.",
+                )
+                config.snipeit_token = os.environ.get('SNIPEIT_TOKEN')
+            else:
+                logger.warning("A SnipeIT base URI was provided but a token was not.")
+                config.kandji_token = None
+        else:
+            logger.warning("A SnipeIT base URI was not provided.")
+            config.snipeit_base_uri = None
+
         # Run cartography
         try:
             return cartography.sync.run_with_config(self.sync, config)

cartography/config.py

@@ -111,6 +111,12 @@ class Config:
     :param duo_api_hostname: The Duo api hostname, e.g. "api-abc123.duosecurity.com". Optional.
     :param semgrep_app_token: The Semgrep api token. Optional.
     :type semgrep_app_token: str
+    :type snipeit_base_uri: string
+    :param snipeit_base_uri: SnipeIT data provider base URI. Optional.
+    :type snipeit_token: string
+    :param snipeit_token: Token used to authenticate to the SnipeIT data provider. Optional.
+    :type snipeit_tenant_id: string
+    :param snipeit_tenant_id: Token used to authenticate to the SnipeIT data provider. Optional.
     """
 
     def __init__(
@@ -170,6 +176,9 @@ def __init__(
         duo_api_secret=None,
         duo_api_hostname=None,
         semgrep_app_token=None,
+        snipeit_base_uri=None,
+        snipeit_token=None,
+        snipeit_tenant_id=None,
     ):
         self.neo4j_uri = neo4j_uri
         self.neo4j_user = neo4j_user
@@ -226,3 +235,6 @@ def __init__(
         self.duo_api_secret = duo_api_secret
         self.duo_api_hostname = duo_api_hostname
         self.semgrep_app_token = semgrep_app_token
+        self.snipeit_base_uri = snipeit_base_uri
+        self.snipeit_token = snipeit_token
+        self.snipeit_tenant_id = snipeit_tenant_id