Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/add yamllint #1032

Merged
merged 1 commit into from
Jul 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
- while mounting volumes in docker compose explicitly mount them as read only or read write
- added `values.yaml` schema validation using `values.schema.json`
- released beta version of improved polling performance
- added `yamllint` validation for the `values.yaml` formatting
- added "in code" validation of groups and profiles

### Fixed
- fixed a bug with configuration from values.yaml not being transferred to the UI while migrating to SC4SNMP-UI
Expand Down
6 changes: 3 additions & 3 deletions charts/splunk-connect-for-snmp/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ splunk:

# name of the metrics index
metricsIndex: "netmetrics"

################################################################################
# Splunk Observability configuration
################################################################################
Expand All @@ -97,7 +97,7 @@ sim:

# Splunk Observability realm to send telemetry data to.
signalfxToken: ""
# Required for Splunk Observability (if `realm` is specified).
# Required for Splunk Observability (if `realm` is specified).
# Splunk Observability org access token.
signalfxRealm: ""

Expand Down Expand Up @@ -383,7 +383,7 @@ inventory:
name: ""

service:
annotations: { }
annotations: {}

# set CPU and Memory limits for an inventory pod
resources: {}
Expand Down
11 changes: 11 additions & 0 deletions docs/bestpractices.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,17 @@ microk8s kubectl delete job/snmp-splunk-connect-for-snmp-inventory -n sc4snmp
```
The upgrade command can be executed again.

### "The following profiles have invalid configuration" or "The following groups have invalid configuration" errors
Following errors are examples of wrong configuration:
```
The following groups have invalid configuration and won't be used: ['group1']. Please check indentation and keywords spelling inside mentioned groups configuration.
```
```
The following profiles have invalid configuration and won't be used: ['standard_profile', 'walk_profile']. Please check indentation and keywords spelling inside mentioned profiles configuration.
```
Errors above indicate, that the mentioned groups or profiles might have wrong indentation or some keywords were omitted or misspelled. Refer to [Configuring profiles](./configuration/configuring-profiles.md)
or [Configuring Groups](./configuration/configuring-groups.md) sections to check how the correct configuration should look like.

### Identifying Traps issues

#### Wrong IP or port
Expand Down
16 changes: 16 additions & 0 deletions docs/gettingstarted/sc4snmp-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,22 @@ or download it directly from Helm using the command `microk8s helm3 show values

It is recommended to start by completing the base template and gradually add additional configurations as needed.


The `values.yaml` file is validated using `JSON schema` built into `helm chart` and inside the code.
To ensure that your `values.yaml` follows formatting standards, you can use `yamllint`. In order to download
`yamllint` refer to the [installation instructions](https://yamllint.readthedocs.io/en/stable/quickstart.html#installing-yamllint).
Then create `custom-config.yamllint` file and add the following configuration:
```yaml
extends: default

rules:
line-length:
max: 80
level: warning
```
Configuration above can be found in the `examples` directory in SC4SNMP [GitHub repository](https://github.com/splunk/splunk-connect-for-snmp).
Next run `yamllint -c <path to custom-config.yamllint> <path to values.yaml>` command. Warnings can be ignored.

#### Install SC4SNMP

After the `values.yaml` creation, you can proceed with the SC4SNMP installation:
Expand Down
6 changes: 6 additions & 0 deletions examples/custom-config.yamllint
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
extends: default

rules:
line-length:
max: 80
level: warning
34 changes: 17 additions & 17 deletions examples/lightweight_installation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@ traps:
- homelab
replicaCount: 1
resources:
limits:
cpu: 100m
memory: 300Mi
requests:
cpu: 40m
memory: 256Mi
limits:
cpu: 100m
memory: 300Mi
requests:
cpu: 40m
memory: 256Mi
#loadBalancerIP: The IP address in the metallb pool
loadBalancerIP: ###TRAP_RECEIVER_IP###
worker:
Expand Down Expand Up @@ -54,12 +54,12 @@ worker:
scheduler:
logLevel: "INFO"
resources:
limits:
cpu: 40m
memory: 260Mi
requests:
cpu: 20m
memory: 180Mi
limits:
cpu: 40m
memory: 260Mi
requests:
cpu: 20m
memory: 180Mi
profiles: |
generic_switch:
frequency: 300
Expand All @@ -80,8 +80,8 @@ poller:
10.0.0.100,,3,,sc4snmp-hlab-sha-des,,1800,generic_switch,,
inventory:
resources:
limits:
cpu: 60m
memory: 300Mi
requests:
cpu: 20m
limits:
cpu: 60m
memory: 300Mi
requests:
cpu: 20m
2 changes: 1 addition & 1 deletion examples/o11y_values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,4 @@ poller:
inventory: |
address,port,version,community,secret,security_engine,walk_interval,profiles,smart_profiles,delete
54.82.4.248,,2c,public,,,4000,small_walk;IF_profile;ICMP_profile,,
54.82.4.249,,2c,public,,,1800,small_walk;IF_profile,,
54.82.4.249,,2c,public,,,1800,small_walk;IF_profile,,
2 changes: 1 addition & 1 deletion examples/polling_and_traps_v3.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,4 @@ poller:
- sc4snmp-hlab-sha-aes
inventory: |
address,port,version,community,secret,security_engine,walk_interval,profiles,smart_profiles,delete
54.82.4.248,,3,public,sc4snmp-hlab-sha-aes,,2000,switch_profile,,
54.82.4.248,,3,public,sc4snmp-hlab-sha-aes,,2000,switch_profile,,
2 changes: 1 addition & 1 deletion examples/polling_groups_values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ scheduler:
poller:
inventory: |
address,port,version,community,secret,security_engine,walk_interval,profiles,smart_profiles,delete
switch_group,,2c,public,,,2000,switch_profile,,
switch_group,,2c,public,,,2000,switch_profile,,
2 changes: 1 addition & 1 deletion examples/polling_values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,4 @@ poller:
inventory: |
address,port,version,community,secret,security_engine,walk_interval,profiles,smart_profiles,delete
54.82.4.248,,2c,public,,,4000,small_walk;IF_profile;ICMP_profile,,
54.82.4.249,,2c,public,,,1800,small_walk;IF_profile,,
54.82.4.249,,2c,public,,,1800,small_walk;IF_profile,,
142 changes: 142 additions & 0 deletions integration_tests/test_poller_integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -1268,6 +1268,148 @@ def test_wrong_profiles(self, request, setup_splunk):
assert metric_count == 0


@pytest.fixture(scope="class")
def setup_misconfigured_profiles(request):
"""
ajasnosz marked this conversation as resolved.
Show resolved Hide resolved
None of the profiles below should poll anything.
"""
trap_external_ip = request.config.getoption("trap_external_ip")
profiles = {
"no_varbinds_profile": {
"frequency": 7,
"varBinds": [],
"conditions": [
{"field": "IF-MIB.ifIndex", "operation": dq("gt"), "value": 20},
{
"field": "IF-MIB.ifDescr",
"operation": dq("equals"),
"value": dq("eth0"),
},
],
},
"no_operation_key_in_condition_profile": {
"frequency": 7,
"varBinds": [yaml_escape_list(sq("IF-MIB"), sq("ifOutDiscards"))],
"conditions": [
{"field": "IF-MIB.ifIndex", "operation": dq("lt"), "value": 20},
{
"field": "IF-MIB.ifDescr",
"value": [dq("test value 1"), dq("test value 2")],
},
],
},
"no_frequency_profile": {
"varBinds": [yaml_escape_list(sq("IF-MIB"), sq("ifOutDiscards"))],
"conditions": [
{"field": "IF-MIB.ifIndex", "operation": dq("equals"), "value": 200}
],
},
"no_patterns_profile": {
"frequency": 3,
"condition": {
"type": "field",
"field": "SNMPv2-MIB.sysDescr",
},
"varBinds": [
yaml_escape_list(sq("IP-MIB"), sq("icmpOutDestUnreachs"), 0),
yaml_escape_list(sq("IP-MIB"), sq("icmpOutEchoReps"), 0),
],
},
}

update_profiles(profiles)
update_file(
[
f"{trap_external_ip},1165,2c,public,,,600,no_varbinds_profile;no_operation_key_in_condition_profile;no_frequency_profile;no_patterns_profile,,",
],
"inventory.yaml",
)
upgrade_helm(["inventory.yaml", "profiles.yaml"])
time.sleep(120)
yield
update_file(
[
f"{trap_external_ip},1165,2c,public,,,600,no_varbinds_profile;no_operation_key_in_condition_profile;no_frequency_profile;no_patterns_profile,,t",
],
"inventory.yaml",
)
upgrade_helm(["inventory.yaml"])
time.sleep(120)


@pytest.mark.usefixtures("setup_misconfigured_profiles")
class TestMisconfiguredProfiles:
def test_wrong_profiles(self, request, setup_splunk):
time.sleep(20)
search_string = """| mpreview index=netmetrics | search profiles=no_varbinds_profile OR profiles=no_operation_key_in_condition_profile OR profiles=no_frequency_profile OR profiles=no_patterns_profile """
result_count, metric_count = run_retried_single_search(
setup_splunk, search_string, 2
)
assert result_count == 0
assert metric_count == 0


@pytest.fixture(scope="class")
def setup_misconfigured_groups(request):
trap_external_ip = request.config.getoption("trap_external_ip")
profiles = {
"routers_wrong_group_profile": {
"frequency": 7,
"varBinds": [yaml_escape_list(sq("IP-MIB"))],
},
"switches_wrong_group_profile": {
"frequency": 7,
"varBinds": [yaml_escape_list(sq("IP-MIB"))],
},
}
groups = {
"routers": [
{"address": trap_external_ip, "port": 1163},
{"address": trap_external_ip, "port": 1164, "wrong_key": 1},
],
"switches": [
{"addre": trap_external_ip},
{"address": trap_external_ip, "port": 1162},
],
}

update_profiles(profiles)
update_groups(groups)
update_file(
[
f"{trap_external_ip},1165,2c,public,,,600,single_profile,,",
f"routers,,2c,public,,,600,routers_wrong_group_profile,,",
f"switches,,2c,public,,,600,switches_wrong_group_profile,,",
],
"inventory.yaml",
)
upgrade_helm(["inventory.yaml", "profiles.yaml", "groups.yaml"])
time.sleep(120)
yield
update_file(
[
f"{trap_external_ip},1165,2c,public,,,600,single_profile,,t",
f"routers,,2c,public,,,600,routers_wrong_group_profile,,t",
f"switches,,2c,public,,,600,switches_wrong_group_profile,,t",
],
"inventory.yaml",
)
upgrade_helm(["inventory.yaml"])
time.sleep(100)


@pytest.mark.usefixtures("setup_misconfigured_groups")
class TestMisconfiguredGroups:
def test_wrong_groups(self, request, setup_splunk):
time.sleep(20)
search_string = """| mpreview index=netmetrics | search profiles=routers_wrong_group_profile OR profiles=routers_wrong_group_profile """
result_count, metric_count = run_retried_single_search(
setup_splunk, search_string, 2
)
assert result_count == 0
assert metric_count == 0


def run_retried_single_search(setup_splunk, search_string, retries):
for _ in range(retries):
result_count, metric_count = splunk_single_search(setup_splunk, search_string)
Expand Down
Loading
Loading