Merge pull request #29 from netboxlabs/develop

release 🚚
netboxlabs · Dec 17, 2024 · a825e71 · a825e71
2 parents 0d0faba + 3f2fe51
commit a825e71
Show file tree

Hide file tree

Showing 5 changed files with 302 additions and 88 deletions.
diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml
@@ -197,7 +197,7 @@ jobs:
               "branches": "release",
               "repositoryUrl": "https://github.com/netboxlabs/orb-agent",
               "debug": "true",
-              "tagFormat": "/v${version}",
+              "tagFormat": "v${version}",
               "plugins": [
                 ["semantic-release-export-data"],
                 ["@semantic-release/commit-analyzer", {

diff --git a/README.md b/README.md
@@ -1,114 +1,79 @@
-# orb-agent
-Orb network observability agent, part of the NetBox ecosystem and developed by NetBox Labs
+# Orb Agent - the NetBox Discovery agent
+Orb agent is a component of the NetBox Discovery solution. It provides network discovery and observability capabilities and is developed by NetBox Labs.
+
+## Project status
+
+The Orb agent project is currently in the Public Preview stage. For details, please see [NetBox Labs Product and Feature Lifecycle](https://docs.netboxlabs.com/product_feature_lifecycle/). We actively welcome feedback to help us identify and prioritize bugs, features, and improvements.
 
 ## Getting Started
-1. Clone the orb-agent project to your local environment:
+To run `orb-agent`, first pull the Docker image from [Docker Hub](https://hub.docker.com/r/netboxlabs/orb-agent):
 
-```sh
-git clone https://github.com/netboxlabs/orb-agent.git
-```
 
-2. Enter the orb-agent directory and build the orb-agent docker image:
 ```sh
-make agent
+docker pull netboxlabs/orb-agent:latest
 ```
 
-3. Start the agent, passing your config file(s):
-```sh
- docker run -v /local/orb:/opt/orb/ netboxlabs/orb-agent:develop run -c /opt/orb/agent.yaml
-```
+## Orb Agent Configuration
+To run, the Orb agent requires a configuration file. This configuration file consists of three main sections: `Config Manager`, `Backends`, and `Policies`.
 
-## Config file samples
 
-### Device-discovery backend
+### Config Manager
+The `Config Manager` section specifies how Orb agent should retrieve it's configuration information. The configuration manager is responsible for processing the configuration to retrieve policies and pass them to the appropriate backend.
 
 ```yaml
 orb:
-  config_manager: 
+  config_manager:
     active: local
-  backends:
-    device_discovery:
-    common:
-      diode:
-        target: grpc://192.168.31.114:8080/diode
-        api_key: ${DIODE_API_KEY}
-        agent_name: agent01
-  policies:
-    device_discovery:
-      discovery_1:
-        config:
-          schedule: "0 */2 * * *"
-          defaults:
-            site: New York NY
-        scope:
-          - hostname: 10.90.0.50
-            username: admin
-            password: ${PASS}
+  ...
 ```
 
-Run command:
-```sh
- docker run -v /local/orb:/opt/orb/ \
- -e DIODE_API_KEY={YOUR_API_KEY} \
- -e PASS={DEVICE_PASSWORD} \
- netboxlabs/orb-agent:develop run -c /opt/orb/agent.yaml
-```
+Currently, only the `local` manager is supported, which retrieves policies from the local configuration file passed to the agent.
 
-#### Custom Drivers
-You can specify community or custom NAPALM drivers using the env variable `INSTALL_DRIVERS_PATH`. Ensure that the required files are placed in the mounted volume (`/opt/orb`).
+### Backends
+The `Backends` section specifies what Orb agent backends should be enabled. Each Orb agent backend offers specific discovery or observability capabilities and may require specific configuration information.  
 
-Mounted folder example:
-```sh
-/local/orb/
-├── agent.yaml
-├── drivers.txt
-├── napalm-mos/
-└── napalm-ros-0.3.2.tar.gz
+```yaml
+orb:
+  ...
+  backends:
+    network_discovery:
+    ...
 ```
+Only the `network_discovery` and `device_discovery` backends are currently supported. They do not require any special configuration.
+- [Device Discovery](./docs/backends/device_discovery.md) 
+- [Network Discovery](./docs/backends/network_discovery.md)
+### Commons
+A special `common` subsection under `Backends` defines configuration settings that are shared with all backends. Currently, it supports passing [diode](https://github.com/netboxlabs/diode) server settings to all backends.
 
-Example `drivers.txt`:
-```txt
-napalm-sros==1.0.2 # try install from pypi
-napalm-ros-0.3.2.tar.gz # try install from a tar.gz
-./napalm-mos # try to install from a folder that contains project.toml
+```yaml
+  backends:
+      ...
+      common:
+      diode:
+        target: grpc://192.168.0.22:8080/diode
+        api_key: ${DIODE_API_KEY}
+        agent_name: agent01
 ```
 
-Run command:
-```sh
- docker run -v /local/orb:/opt/orb/ \
- -e DIODE_API_KEY={YOUR_API_KEY} \
- -e PASS={DEVICE_PASSWORD} \
- -e INSTALL_DRIVERS_PATH=/opt/orb/drivers.txt \
- netboxlabs/orb-agent:develop run -c /opt/orb/agent.yaml
-```
-The relative path used by `pip install` is the folder that contains `.txt` file.
 
+### Policies
+The `Policies` section specifies what discovery policies should be passed to each backend. Policies define specific settings for discovery (such as scheduling and default properties) and the scope (targets). Backends can run multiple policies simultaneously, but for each backend all policies must have a unique name. These policies are defined in the `policies` section and are grouped under a subsection for each backend:
 
-### Network-discovery backend
-```yaml
+ ```yaml
 orb:
-  config_manager:
-    active: local
-  backends:
-    network_discovery:
-    common:
-      diode:
-        target: grpc://192.168.31.114:8080/diode
-        api_key: ${DIODE_API_KEY}
-        agent_name: agent02
+  ...
   policies:
+    device_discovery:
+      device_policy_1:
+        # see device_discovery section
     network_discovery:
-      policy_1:
-        config:
-          schedule: "0 */2 * * *"
-          timeout: 5
-        scope:
-          targets: [192.168.1.1/22, google.com]
-```
+      network_policy_1:
+       # see network_discovery section
+ ```
 
-Run command:
-```sh
- docker run -v /local/orb:/opt/orb/ \
- -e DIODE_API_KEY={YOUR_API_KEY} \
- netboxlabs/orb-agent:develop run -c /opt/orb/agent.yaml
-```
+ ## Configuration samples
+You can find sample configurations [here](./docs/config_samples.md) of how to configure Orb agent to run network and device discoveries.
+
+## Required Notice
+
+Copyright NetBox Labs, Inc.
diff --git a/docs/backends/device_discovery.md b/docs/backends/device_discovery.md
@@ -0,0 +1,76 @@
+# Device Discovery
+The device discovery backend leverages NAPALM to connect to network devices and collect network information.
+
+
+## Configuration
+The `device_discovery` backend does not require any special configuration, though overriding `host` and `port` values can be specified. The backend will use the `diode` settings specified in the `common` subsection to forward discovery results.
+
+
+```yaml
+orb:
+  backends:
+    common:
+      diode:
+        target: grpc://192.168.0.100:8080/diode
+        api_key: ${DIODE_API_KEY}
+        agent_name: agent01
+    device_discovery:
+      host: 192.168.5.11 #default 0.0.0.0
+      port: 8857 # default 8072
+
+```
+
+## Policy
+Device discovery policies are broken down into two subsections: `config` and `scope`. 
+
+### Config
+Config defines data for the whole scope and is optional overall.
+
+| Parameter | Type | Required | Description |
+|:---------:|:----:|:--------:|:-----------:|
+| schedule | cron format | no  |  If defined, it will execute scope following cron schedule time. If not defined, it will execute scope only once  |
+| defaults | map | no  |  key value pair that defines default values  |
+
+#### Defaults
+Current supported defaults:
+
+|  Key  |  Description  |
+|:-----:|:-------------:|
+| site  |  NetBox Site Name |
+
+### Scope
+The scope defines a list of devices that can be accessed and pulled data. 
+
+| Parameter | Type | Required | Description |
+|:---------:|:----:|:--------:|:-----------:|
+| hostname | string | yes  | Device hostname |
+| username | string | yes  | Device username  |
+| password | string | yes  | Device username's password |
+| optional_args | map | no  | NAPALM optional arguments defined [here](https://napalm.readthedocs.io/en/latest/support/#list-of-supported-optional-arguments) |
+| driver | string | no  |  If defined, try to connect to device using the specified NAPALM driver. If not, it will try all the current installed drivers |
+
+
+
+### Sample
+A sample policy including all parameters supported by the device discovery backend.
+```yaml
+orb:
+  ...
+  policies:
+    device_discovery:
+      discovery_1:
+        config:
+          schedule: "* * * * *"
+          defaults:
+            site: New York NY
+        scope:
+          - driver: ios
+            hostname: 192.168.0.5
+            username: admin
+            password: ${PASS}
+            optional_args:
+               canonical_int: True
+          - hostname: myhost.com
+            username: remote
+            password: 12345
+```
diff --git a/docs/backends/network_discovery.md b/docs/backends/network_discovery.md
@@ -0,0 +1,76 @@
+# Network Discovery
+The network discovery backend leverages NMAP to scan networks and discover IP information.
+
+
+## Configuration
+The `network_discovery` backend does not require any special configuration, though overriding `host` and `port` values can be specified. The backend will use the `diode` settings specified in the `common` subsection to forward discovery results.
+
+
+```yaml
+orb:
+  backends:
+    common:
+      diode:
+        target: grpc://192.168.0.100:8080/diode
+        api_key: ${DIODE_API_KEY}
+        agent_name: agent01
+    network_discovery:
+      host: 192.168.5.11 #default 0.0.0.0
+      port: 8863 # default 8072
+      log_level: ERROR #default INFO
+      log_format: JSON #default TEXT
+
+```
+
+## Policy
+Network discovery policies are broken down into two subsections: `config` and `scope`.
+
+### Config
+Config defines data for the whole scope and is optional overall.
+
+| Parameter | Type | Required | Description |
+|:---------:|:----:|:--------:|:-----------:|
+| schedule | cron format | no  |  If defined, it will execute scope following cron schedule time. If not defined, it will execute scope only once  |
+| defaults | map | no  |  key value pair that defines default values  |
+| timeout | int | no | Timeout in minutes for the nmap scan operation. The default value is 2 minutes.
+
+#### Defaults
+Current supported defaults:
+
+|  Key  |  Description  |
+|:-----:|:-------------:|
+| comments  |  NetBox Comments information to be added to discovered IP |
+| description  |  NetBox Description data to be added to discovered IP |
+
+### Scope
+The scope defines a list of targets to be scanned.
+
+| Parameter | Type | Required | Description |
+|:---------:|:----:|:--------:|:-----------:|
+| targets | list | yes  | The targets that NMAP will scan. These can be specified as IP addresses (192.168.1.1), IP ranges (192.168.1.10-20), IP subnets with mask (192.168.1.0/24) or resolvable domain names. |
+
+
+
+
+### Sample
+A sample policy including all parameters supported by the network discovery backend.
+```yaml
+orb:
+  ...
+  policies:
+    network_discovery:
+      discovery_1:
+        config:
+          schedule: "* * * * *"
+          timeout: 5
+          defaults:
+            comments: none
+            description: IP discovered by network discovery
+        scope:
+          targets: 
+            - 192.168.7.32
+            - 192.168.7.30-40 # IP range
+            - 192.168.7.0/24 # IP subnet
+            - google.com #dns lookup
+
+```