Initial release for integration DuckDB (#19238)

* create duckdb

* fix readme

* implementation v1

* implementation v2

* add tests

* test

* commit

* docker

* add tests

* bump to python3.12

* clean files

* rewrite tests

* tests

* tests

* commit

* commit

* commit

* commit

* commit

* commit

* ci

* asset

* add service check

* metadata

* labler

* licenses

* licenses

* tests

* remove file

* add e2e test

* base check

* lint

* remove service check

* commit

.codecov.yml

@@ -186,6 +186,10 @@ coverage:
         target: 75
         flags:
         - druid
+      DuckDB:
+        target: 75
+        flags:
+        - duckdb
       EKS_Fargate:
         target: 75
         flags:
@@ -974,6 +978,11 @@ flags:
     paths:
     - druid/datadog_checks/druid
     - druid/tests
+  duckdb:
+    carryforward: true
+    paths:
+    - duckdb/datadog_checks/duckdb
+    - duckdb/tests
   ecs_fargate:
     carryforward: true
     paths:

.github/workflows/config/labeler.yml

@@ -177,6 +177,8 @@ integration/dotnetclr:
 - dotnetclr/**/*
 integration/druid:
 - druid/**/*
+integration/duckdb:
+- duckdb/**/*
 integration/ecs_fargate:
 - ecs_fargate/**/*
 integration/eks_anywhere:

.github/workflows/test-all.yml

@@ -1154,6 +1154,26 @@ jobs:
       minimum-base-package: ${{ inputs.minimum-base-package }}
       pytest-args: ${{ inputs.pytest-args }}
     secrets: inherit
+  j9eb6aa6:
+    uses: ./.github/workflows/test-target.yml
+    with:
+      job-name: DuckDB
+      target: duckdb
+      platform: linux
+      runner: '["ubuntu-22.04"]'
+      repo: "${{ inputs.repo }}"
+      python-version: "${{ inputs.python-version }}"
+      standard: ${{ inputs.standard }}
+      latest: ${{ inputs.latest }}
+      agent-image: "${{ inputs.agent-image }}"
+      agent-image-py2: "${{ inputs.agent-image-py2 }}"
+      agent-image-windows: "${{ inputs.agent-image-windows }}"
+      agent-image-windows-py2: "${{ inputs.agent-image-windows-py2 }}"
+      test-py2: ${{ inputs.test-py2 }}
+      test-py3: ${{ inputs.test-py3 }}
+      minimum-base-package: ${{ inputs.minimum-base-package }}
+      pytest-args: ${{ inputs.pytest-args }}
+    secrets: inherit
   j562bfe5:
     uses: ./.github/workflows/test-target.yml
     with:

LICENSE-3rdparty.csv

@@ -22,6 +22,7 @@ cryptography,PyPI,BSD-3-Clause,Copyright (c) Individual contributors.
 cryptography,PyPI,PSF,Copyright (c) Individual contributors.
 ddtrace,PyPI,BSD-3-Clause,"Copyright 2016 Datadog, Inc."
 dnspython,PyPI,ISC,Copyright (C) Dnspython Contributors
+duckdb,PyPI,MIT,Copyright (c) Hannes Muehleisen
 flup,Vendor,BSD-3-Clause,Copyright (c) 2005 Allan Saddi. All Rights Reserved.
 flup-py3,Vendor,BSD-3-Clause,"Copyright (c) 2005, 2006 Allan Saddi <[email protected]> All rights reserved."
 foundationdb,PyPI,Apache-2.0,Copyright 2017 FoundationDB

README.md

@@ -54,4 +54,4 @@ For more information on integrations, please reference our [documentation][11] a
 [28]: https://img.shields.io/badge/typing-Mypy-blue.svg
 [29]: https://github.com/python/mypy
 [30]: https://img.shields.io/badge/license-BSD--3--Clause-9400d3.svg
-[31]: https://spdx.org/licenses/BSD-3-Clause.html
+[31]: https://spdx.org/licenses/BSD-3-Clause.html

agent_requirements.in

@@ -13,6 +13,7 @@ confluent-kafka==2.6.1
 cryptography==43.0.1
 ddtrace==2.10.6
 dnspython==2.6.1
+duckdb==1.1.1
 foundationdb==6.3.24
 hazelcast-python-client==5.4.0
 importlib-metadata==2.1.3; python_version < '3.8'

duckdb/CHANGELOG.md

@@ -0,0 +1,4 @@
+# CHANGELOG - DuckDB
+
+<!-- towncrier release notes start -->
+

duckdb/README.md

@@ -0,0 +1,60 @@
+# Agent Check: DuckDB
+
+## Overview
+
+This check monitors [DuckDB][1] through the Datadog Agent. 
+
+Include a high level overview of what this integration does:
+- What does your product do (in 1-2 sentences)?
+- What value will customers get from this integration, and why is it valuable to them?
+- What specific data will your integration monitor, and what's the value of that data?
+
+## Setup
+
+Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the [Autodiscovery Integration Templates][3] for guidance on applying these instructions.
+
+### Installation
+
+The DuckDB check is included in the [Datadog Agent][2] package.
+No additional installation is needed on your server.
+
+### Configuration
+
+1. Edit the `duckdb.d/conf.yaml` file, in the `conf.d/` folder at the root of your Agent's configuration directory to start collecting your duckdb performance data. See the [sample duckdb.d/conf.yaml][4] for all available configuration options.
+
+2. [Restart the Agent][5].
+
+### Validation
+
+[Run the Agent's status subcommand][6] and look for `duckdb` under the Checks section.
+
+## Data Collected
+
+### Metrics
+
+See [metadata.csv][7] for a list of metrics provided by this integration.
+
+### Events
+
+The DuckDB integration does not include any events.
+
+### Service Checks
+
+The DuckDB integration does not include any service checks.
+
+See [service_checks.json][8] for a list of service checks provided by this integration.
+
+## Troubleshooting
+
+Need help? Contact [Datadog support][9].
+
+
+[1]: **LINK_TO_INTEGRATION_SITE**
+[2]: https://app.datadoghq.com/account/settings/agent/latest
+[3]: https://docs.datadoghq.com/agent/kubernetes/integrations/
+[4]: https://github.com/DataDog/integrations-core/blob/master/duckdb/datadog_checks/duckdb/data/conf.yaml.example
+[5]: https://docs.datadoghq.com/agent/guide/agent-commands/#start-stop-and-restart-the-agent
+[6]: https://docs.datadoghq.com/agent/guide/agent-commands/#agent-status-and-information
+[7]: https://github.com/DataDog/integrations-core/blob/master/duckdb/metadata.csv
+[8]: https://github.com/DataDog/integrations-core/blob/master/duckdb/assets/service_checks.json
+[9]: https://docs.datadoghq.com/help/

duckdb/assets/configuration/spec.yaml

@@ -0,0 +1,34 @@
+name: DuckDB
+files:
+- name: duckdb.yaml
+  options:
+  - template: init_config
+    options:
+    - template: init_config/default
+  - template: instances
+    options:
+    - name: db_name
+      required: true
+      description: |
+        The database to connect to (file path).
+        NOTE: DuckDB databases are stored as files.
+      value:
+        example: "/path-to-file/my_database.db"
+        type: string
+    - name: connection_attempt
+      description: | 
+        The number of retries to connect to the database in case of failure
+      value:
+        type: integer
+        default: 3
+        display_default: 3
+    - template: instances/default
+      overrides:
+        min_collection_interval.description: |
+          This changes the collection interval of this check to avoid 
+          the risk of locking the database file.
+          If your database file is heavily used in write mode, you changes
+          further increase this value to minimise the monitoring acccess.
+        min_collection_interval.value.default: 60
+        min_collection_interval.value.example: 60
+        min_collection_interval.enabled: true

duckdb/assets/dashboards/duckdb_overview.json

@@ -0,0 +1,29 @@
+{
+    "title": "DuckDB Overview",
+    "description": "[[suggested_dashboards]]",
+    "widgets": [
+        {
+            "id": 6432334130190000,
+            "definition": {
+                "type": "image",
+                "url": "https://static.datadoghq.com/static/images/logos/duckdb_small.svg",
+                "sizing": "contain",
+                "margin": "md",
+                "has_background": false,
+                "has_border": false,
+                "vertical_align": "center",
+                "horizontal_align": "center"
+            },
+            "layout": {
+                "x": 0,
+                "y": 0,
+                "width": 2,
+                "height": 2
+            }
+        }
+    ],
+    "template_variables": [],
+    "layout_type": "ordered",
+    "notify_list": [],
+    "reflow_type": "fixed"
+}

duckdb/assets/service_checks.json

@@ -0,0 +1,2 @@
+
+[]

duckdb/changelog.d/1.added

@@ -0,0 +1 @@
+Initial Release

duckdb/datadog_checks/__init__.py

@@ -0,0 +1,4 @@
+# (C) Datadog, Inc. 2024-present
+# All rights reserved
+# Licensed under a 3-clause BSD style license (see LICENSE)
+__path__ = __import__('pkgutil').extend_path(__path__, __name__)  # type: ignore

duckdb/datadog_checks/duckdb/__about__.py

@@ -0,0 +1,4 @@
+# (C) Datadog, Inc. 2024-present
+# All rights reserved
+# Licensed under a 3-clause BSD style license (see LICENSE)
+__version__ = '0.0.1'

duckdb/datadog_checks/duckdb/__init__.py

@@ -0,0 +1,7 @@
+# (C) Datadog, Inc. 2024-present
+# All rights reserved
+# Licensed under a 3-clause BSD style license (see LICENSE)
+from .__about__ import __version__
+from .check import DuckdbCheck
+
+__all__ = ['__version__', 'DuckdbCheck']