From 15f400bd8fe854b8383982349135bb3e02d71ece Mon Sep 17 00:00:00 2001 From: Colton Padden Date: Sat, 14 Dec 2024 21:58:39 -0500 Subject: [PATCH 1/2] [docs-beta] migrate - Insights docs (export) --- .../features/insights/export-metrics.md | 145 ++++++++++++++++++ 1 file changed, 145 insertions(+) diff --git a/docs/docs-beta/docs/dagster-plus/features/insights/export-metrics.md b/docs/docs-beta/docs/dagster-plus/features/insights/export-metrics.md index 219eee2060f41..2a88305fa82b6 100644 --- a/docs/docs-beta/docs/dagster-plus/features/insights/export-metrics.md +++ b/docs/docs-beta/docs/dagster-plus/features/insights/export-metrics.md @@ -5,3 +5,148 @@ sidebar_position: 200 unlisted: true --- +Using a GraphQL API endpoint, you can export [Dagster+ Insights](/dagster-plus/insights) metrics from your Dagster+ instance. + +Refer to the [Available Insights metrics](/dagster-plus/insights#available-metrics) for a list of available metrics. + +## Prerequisites + +To complete the steps in this guide, you'll need: + +- A Dagster+ account +- Access to the [Dagster+ Insights feature](/dagster-plus/insights) +- A Dagster+ [user token](/dagster-plus/account/managing-user-agent-tokens#managing-user-tokens) +- Your deployment-scoped Dagster+ deployment URL. For example: `dagster-university.dagster.cloud/prod` + +## Before you start + +Before you start, note that: + +- Metrics are currently computed once per day +- We don't recommend frequently querying over large time ranges that may download a large amount of data. After an initial data load, we recommend loading data daily for the most recent week or less. + +## Using the API + +In this example, we're using the [GraphQL Python Client](/concepts/webserver/graphql-client) to export the Dagster credits metric for all assets for September 2023: + +```python +from datetime import datetime +from dagster_graphql import DagsterGraphQLClient + +ASSET_METRICS_QUERY = """ +query AssetMetrics($metricName: String, $after: Float, $before: Float) { + reportingMetricsByAsset( + metricsSelector: { + metricName: $metricName + after: $after + before: $before + sortAggregationFunction: SUM + granularity: DAILY + } + ) { + __typename + ... on ReportingMetrics { + metrics { + values + entity { + ... on ReportingAsset { + assetKey { + path + } + } + } + } + } + } +} + +""" + + +def get_client(): + url = "YOUR_ORG.dagster.cloud/prod" # Your deployment-scoped url + user_token = "YOUR_TOKEN" # A token generated from Organization Settings > Tokens + return DagsterGraphQLClient(url, headers={"Dagster-Cloud-Api-Token": user_token}) + + +if __name__ == "__main__": + client = get_client() + result = client._execute( + ASSET_METRICS_QUERY, + { + "metricName": "__dagster_dagster_credits", + "after": datetime(2023, 9, 1).timestamp(), + "before": datetime(2023, 10, 1).timestamp(), + }, + ) + + for asset_series in result["reportingMetricsByAsset"]["metrics"]: + print("Asset key:", asset_series["entity"]["assetKey"]["path"]) + print("Daily values:", asset_series["values"]) + +``` + +To use this example yourself, replace the values of `url` and `user_token` in this function: + +```python +def get_client(): + url = "YOUR_ORG.dagster.cloud/prod" # Your deployment-scoped url + user_token = "YOUR_TOKEN" # A token generated from Organization Settings > Tokens + return DagsterGraphQLClient(url, headers={"Dagster-Cloud-Api-Token": user_token}) +``` + +Refer to the [Reference section](#reference) for more info about the endpoints available in the GraphQL API. + +## Reference + +For the full GraphQL API reference: + +1. Navigate to `YOUR_ORG.dagster.cloud/prod/graphql`, replacing `YOUR_ORG` with your organization name. For example: `https://dagster-university.dagster.cloud/prod/graphql` +2. Click the **Schema** tab. + +### Available top-level queries + +```graphql +reportingMetricsByJob( + metricsFilter: JobReportingMetricsFilter + metricsSelector: ReportingMetricsSelector! +): ReportingMetricsOrError! + +reportingMetricsByAsset( + metricsFilter: AssetReportingMetricsFilter + metricsSelector: ReportingMetricsSelector! +): ReportingMetricsOrError! + +reportingMetricsByAssetGroup( + metricsFilter: AssetGroupReportingMetricsFilter + metricsSelector: ReportingMetricsSelector! +): ReportingMetricsOrError! +``` + +### Specifying metrics and time granularity + +Use `metricsSelector` to specify the metric name and time granularity: + +```graphql +input ReportingMetricsSelector { + after: Float # timestamp + before: Float # timestamp + metricName: String # see below for valid values + granularity: ReportingMetricsGranularity +} + +enum ReportingMetricsGranularity { + DAILY + WEEKLY + MONTHLY +} + +# The valid metric names are: +# "__dagster_dagster_credits" +# "__dagster_execution_time_ms" +# "__dagster_materializations" +# "__dagster_step_failures" +# "__dagster_step_retries" +# "__dagster_asset_check_errors" +# "__dagster_asset_check_warnings" +``` From d98373fa550c760461bdcda4b9e3ab258bed7a5f Mon Sep 17 00:00:00 2001 From: Colton Padden Date: Thu, 19 Dec 2024 13:41:55 -0500 Subject: [PATCH 2/2] todo links --- .../features/insights/export-metrics.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/docs/docs-beta/docs/dagster-plus/features/insights/export-metrics.md b/docs/docs-beta/docs/dagster-plus/features/insights/export-metrics.md index 2a88305fa82b6..1944b206519b8 100644 --- a/docs/docs-beta/docs/dagster-plus/features/insights/export-metrics.md +++ b/docs/docs-beta/docs/dagster-plus/features/insights/export-metrics.md @@ -5,17 +5,21 @@ sidebar_position: 200 unlisted: true --- -Using a GraphQL API endpoint, you can export [Dagster+ Insights](/dagster-plus/insights) metrics from your Dagster+ instance. +{/* Using a GraphQL API endpoint, you can export [Dagster+ Insights](/dagster-plus/insights) metrics from your Dagster+ instance. */} +Using a GraphQL API endpoint, you can export [Dagster+ Insights](/todo) metrics from your Dagster+ instance. -Refer to the [Available Insights metrics](/dagster-plus/insights#available-metrics) for a list of available metrics. +{/* Refer to the [Available Insights metrics](/dagster-plus/insights#available-metrics) for a list of available metrics. */} +Refer to the [Available Insights metrics](/todo) for a list of available metrics. ## Prerequisites To complete the steps in this guide, you'll need: - A Dagster+ account -- Access to the [Dagster+ Insights feature](/dagster-plus/insights) -- A Dagster+ [user token](/dagster-plus/account/managing-user-agent-tokens#managing-user-tokens) +{/* - Access to the [Dagster+ Insights feature](/dagster-plus/insights) */} +- Access to the [Dagster+ Insights feature](/todo) +{/* - A Dagster+ [user token](/dagster-plus/account/managing-user-agent-tokens#managing-user-tokens) */} +- A Dagster+ [user token](/todo) - Your deployment-scoped Dagster+ deployment URL. For example: `dagster-university.dagster.cloud/prod` ## Before you start @@ -27,7 +31,8 @@ Before you start, note that: ## Using the API -In this example, we're using the [GraphQL Python Client](/concepts/webserver/graphql-client) to export the Dagster credits metric for all assets for September 2023: +{/* In this example, we're using the [GraphQL Python Client](/concepts/webserver/graphql-client) to export the Dagster credits metric for all assets for September 2023: */} +In this example, we're using the [GraphQL Python Client](/todo) to export the Dagster credits metric for all assets for September 2023: ```python from datetime import datetime