Initial commit (v0.0.1) (#1)

* Initial commit (v0.0.1)

.gitignore

@@ -0,0 +1 @@
+.terraform

README.md

@@ -1,33 +1,98 @@
 [![New Relic Experimental header](https://github.com/newrelic/opensource-website/raw/master/src/images/categories/Experimental.png)](https://opensource.newrelic.com/oss-category/#new-relic-experimental)
 
-# [Name of Project] [build badges go here when available]
+# New Relic CloudWatch Metric Streams Terraform module
 
->[Brief description - what is the project and value does it provide? How often should users expect to get releases? How is versioning set up? Where does this project want to go?]
+Terraform module which creates the necessary resources for using CloudWatch Metric Streams integration with New Relic.
 
-## Installation
+This module will provision the following resources, including their necessary IAN permissions:
 
-> [Include a step-by-step procedure on how to get your code installed. Be sure to include any third-party dependencies that need to be installed separately]
+* IAM Role for linking your AWS account to New Relic One
+* CloudWatch Metric Stream
+* Kinesis Firehose Delivery Stream
 
-## Getting Started
->[Simple steps to start working with the software similar to a "Hello World"]
+Please note that this module is experimental, and if you find it useful, we would greatly appreciate your feedback!
 
-## Usage
->[**Optional** - Include more thorough instructions on how to use the software. This section might not be needed if the Getting Started section is enough. Remove this section if it's not needed.]
+## Terraform version
+
+Terraform v0.12 or above is required.
+
+## Providers
 
+| Name          | Version   |
+|---------------|-----------|
+| hashicorp/aws | >= 3.42.0 |
 
-## Building
+## Notes
 
->[**Optional** - Include this section if users will need to follow specific instructions to build the software from source. Be sure to include any third party build dependencies that need to be installed separately. Remove this section if it's not needed.]
+* Although this module provisions the IAM role and policy required for linking your AWS and New Relic accounts together, it does not do so automatically, and will have to be done either manually in New Relic UI, or using NerdGraph. See more details under "Getting Started" below.
 
-## Testing
+* Please make sure to use the correct New Relic collector endpoint for your account using the variable `newrelic_collector_endpoint`. The correct URL can be found [here](https://docs.newrelic.com/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-metric-stream/#manual-setup).
 
->[**Optional** - Include instructions on how to run tests if we include tests with the codebase. Remove this section if it's not needed.]
+## Getting Started
+
+First, you will need to include the module in your Terraform code. An example usage is documented below.
 
-## Support
+After your Terraform changes are applied, you will need to link each of your AWS accounts with your New Relic account.
+To do so:
 
-New Relic hosts and moderates an online forum where customers can interact with New Relic employees as well as other customers to get help and share best practices. Like all official New Relic open source projects, there's a related Community topic in the New Relic Explorers Hub. You can find this project's topic/threads here:
+* Go to [one.newrelic.com](https://one.newrelic.com/) **> Infrastructure > AWS**, click on **Add an AWS account**, then on **Use metric streams**, and follow the steps.
+* You may [automate this step with NerdGraph](https://docs.newrelic.com/docs/apis/nerdgraph/examples/nerdgraph-cloud-integrations-api-tutorial/#link-aws).
+
+## Usage
 
->Add the url for the support thread here
+Copy and paste into your Terraform configuration, insert the variables, and run `terraform init`:
+
+```hcl
+module "example-usage" {
+  source = "git::ssh://[email protected]/newrelic-experimental/terraform-cloudwatch-metric-streams?ref=tags/v0.0.1"
+
+  # Required variables
+  
+  newrelic_trusted_account_id = "<your-newrelic-account-id>"
+  newrelic_license_key        = "<your-newrelic-license-key>"
+  
+  # The URL can be found on: https://docs.newrelic.com/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-metric-stream/#manual-setup
+  # Typically, this changes based on your account region.
+  #   - For accounts in US: https://aws-api.newrelic.com/cloudwatch-metrics/v1
+  #   - For accounts in EU: https://aws-api.eu01.nr-data.net/cloudwatch-metrics/v1
+  newrelic_collector_endpoint = "<newrelic-collector-endpoint>"
+
+  # Optional variables
+
+  name_suffix = "-example"
+  name_prefix = "example-"
+
+  cloudwatch_metric_stream_include_filter = [
+    "AWS/EC2",
+    "AWS/S3",
+    "AWS/SQS",
+    "AWS/SageMaker"
+  ]
+}
+```
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:-----:|
+| newrelic\_trusted\_account\_id | This is your New Relic account id, and it is needed to only allow your account to assume the role needed for account linking. | `string` | - | yes |
+| newrelic\_license\_key | This is your New Relic ingest license key, and it is needed for Kinesis Firehose to successfully send metrics to your New Relic account. | `string` | - | yes |
+| newrelic\_collector\_endpoint | This is the New Relic collector endpoint. The URL changes based on your account region (US/EU), and can be found on https://docs.newrelic.com/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-metric-stream/#manual-setup. | `string` | - | yes |
+| name\_prefix | A prefix to prepend to the name of all resources created by this module. | `string` | empty ("") | no |
+| name\_suffix | A suffix to append to the name of all resources created by this module. | `string` | empty ("") | no |
+| cloudwatch\_metric\_stream\_include\_filter | List of namespaces to include from the CloudWatch Metric Stream. Mutually exclusive with `cloudwatch_metric_stream_exclude_filter`. | `list` | empty ([]) | no |
+| cloudwatch\_metric\_stream\_exclude\_filter | List of namespaces to exclude from the CloudWatch Metric Stream. Mutually exclusive with `cloudwatch_metric_stream_include_filter`. | `list` | empty ([]) | no |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| newrelic-one-external-role-arn | The ARN of the provisioned IAM role used for New Relic account linking. |
+| kinesis-firehose-stream-arn | The ARN of the provisioned Kinesis Firehose Delivery Stream. |
+| kinesis-firehose-stream-role-arn | The ARN of the provisioned IAM role used by Kinesis Firehose Delivery Stream. |
+| kinesis-firehose-stream-failed-data-s3-bucket-arn | The ARN of the provisioned S3 bucket used for storing data which Kinesis Firehose failed sending to New Relic. |
+| cloudwatch-metric-stream-arn | The ARN of the provisioned CloudWatch Metric Stream. |
+| cloudwatch-metric-stream-role-arn | The ARN of the provisioned IAM role used by CloudWatch Metric Stream. |
 
 ## Contributing
 We encourage your contributions to improve [project name]! Keep in mind when you submit your pull request, you'll need to sign the CLA via the click-through using CLA-Assistant. You only have to sign the CLA one time per project.
@@ -40,5 +105,4 @@ As noted in our [security policy](../../security/policy), New Relic is committed
 If you believe you have found a security vulnerability in this project or any of New Relic's products or websites, we welcome and greatly appreciate you reporting it to New Relic through [HackerOne](https://hackerone.com/newrelic).
 
 ## License
-[Project Name] is licensed under the [Apache 2.0](http://apache.org/licenses/LICENSE-2.0.txt) License.
->[If applicable: The [project name] also uses source code from third-party libraries. You can find full details on which libraries are used and the terms under which they are licensed in the third-party notices document.]
+Licensed under Apache 2.0, see [LICENSE](LICENSE) for full details.

data.tf

@@ -0,0 +1,95 @@
+# external role (to be assumed by New Relic One)
+
+data "aws_iam_policy_document" "newrelic-one-external-role-trust-policy" {
+  statement {
+    sid     = "AllowRoleAssumptionByNewRelicOne"
+    effect  = "Allow"
+    actions = ["sts:AssumeRole"]
+
+    principals {
+      type        = "AWS"
+      identifiers = ["754728514883"] # New Relic One account
+    }
+
+    condition {
+      test     = "StringEquals"
+      variable = "sts:ExternalId"
+      values   = [var.newrelic_trusted_account_id]
+    }
+  }
+}
+
+data "aws_iam_policy_document" "newrelic-one-external-role-budget-policy" {
+  statement {
+    sid     = "AllowViewBudget"
+    effect  = "Allow"
+    actions = ["budgets:ViewBudget"]
+
+    resources = ["*"]
+  }
+}
+
+# Kinesis Firehose Delivery Stream
+
+data "aws_iam_policy_document" "kinesis-firehose-stream-role-trust-policy" {
+  statement {
+    sid     = "AllowRoleAssumptionByKinesisFirehose"
+    effect  = "Allow"
+    actions = ["sts:AssumeRole"]
+
+    principals {
+      type        = "Service"
+      identifiers = ["firehose.amazonaws.com"]
+    }
+  }
+}
+
+data "aws_iam_policy_document" "kinesis-firehose-stream-role-s3-policy" {
+  statement {
+    sid    = "AllowFirehoseS3Access"
+    effect = "Allow"
+    actions = [
+      "s3:AbortMultipartUpload",
+      "s3:GetBucketLocation",
+      "s3:GetObject",
+      "s3:ListBucke",
+      "s3:ListBucketMultipartUpload",
+      "s3:PutObject"
+    ]
+
+    resources = [
+      aws_s3_bucket.kinesis-firehose-stream-failed-data.arn,
+      "${aws_s3_bucket.kinesis-firehose-stream-failed-data.arn}/*"
+    ]
+  }
+}
+
+# CloudWatch Metric Stream
+
+data "aws_iam_policy_document" "cloudwatch-metric-stream-role-trust-policy" {
+  statement {
+    sid     = "AllowRoleAssumptionByloudWatchMetricStream"
+    effect  = "Allow"
+    actions = ["sts:AssumeRole"]
+
+    principals {
+      type        = "Service"
+      identifiers = ["streams.metrics.cloudwatch.amazonaws.com"]
+    }
+  }
+}
+
+data "aws_iam_policy_document" "cloudwatch-metric-stream-role-firehose-policy" {
+  statement {
+    sid    = "AllowCloudWatchFirehoseAccess"
+    effect = "Allow"
+    actions = [
+      "firehose:PutRecord",
+      "firehose:PutRecordBatch"
+    ]
+
+    resources = [
+      aws_kinesis_firehose_delivery_stream.kinesis-firehose-stream.arn
+    ]
+  }
+}

main.tf

@@ -0,0 +1,109 @@
+# External role for linking your AWS account to New Relic
+
+resource "aws_iam_role" "newrelic-one-external-role" {
+  name               = "${var.name_prefix}newrelic-one-external-role${var.name_suffix}"
+  assume_role_policy = data.aws_iam_policy_document.newrelic-one-external-role-trust-policy.json
+
+  tags = {
+    Name = "${var.name_prefix}newrelic-one-external-role${var.name_suffix}"
+  }
+}
+
+resource "aws_iam_role_policy_attachment" "newrelic-one-external-role-ReadOnlyAccess-attachment" {
+  role       = aws_iam_role.newrelic-one-external-role.id
+  policy_arn = "arn:aws:iam::aws:policy/ReadOnlyAccess"
+}
+
+resource "aws_iam_role_policy" "newrelic-one-external-role-budget-policy" {
+  name   = "${var.name_prefix}NewRelicBudget${var.name_suffix}"
+  role   = aws_iam_role.newrelic-one-external-role.id
+  policy = data.aws_iam_policy_document.newrelic-one-external-role-budget-policy.json
+}
+
+# Kinesis Firehose Delivery Stream
+
+resource "aws_iam_role" "kinesis-firehose-stream-role" {
+  name               = "${var.name_prefix}cloudwatch-firehose-stream-role${var.name_suffix}"
+  assume_role_policy = data.aws_iam_policy_document.kinesis-firehose-stream-role-trust-policy.json
+
+  tags = {
+    Name = "${var.name_prefix}cloudwatch-firehose-stream-role${var.name_suffix}"
+  }
+}
+
+resource "aws_s3_bucket" "kinesis-firehose-stream-failed-data" {
+  bucket = "${var.name_prefix}cloudwatch-metric-stream-failed-data${var.name_suffix}"
+  acl    = "private"
+}
+
+resource "aws_iam_role_policy" "kinesis-firehose-stream-role-s3-policy" {
+  name   = "${var.name_prefix}KinesisFirehose-S3Access${var.name_suffix}"
+  role   = aws_iam_role.kinesis-firehose-stream-role.id
+  policy = data.aws_iam_policy_document.kinesis-firehose-stream-role-s3-policy.json
+}
+
+resource "aws_kinesis_firehose_delivery_stream" "kinesis-firehose-stream" {
+  name        = "${var.name_prefix}cloudwatch-metric-stream${var.name_suffix}"
+  destination = "http_endpoint"
+
+  s3_configuration {
+    role_arn           = aws_iam_role.kinesis-firehose-stream-role.arn
+    bucket_arn         = aws_s3_bucket.kinesis-firehose-stream-failed-data.arn
+    buffer_size        = 10  # MiB
+    buffer_interval    = 300 # seconds
+    compression_format = "GZIP"
+  }
+
+  http_endpoint_configuration {
+    url                = var.newrelic_collector_endpoint
+    name               = "New Relic - Metrics"
+    access_key         = var.newrelic_license_key
+    buffering_size     = 1  # MiB
+    buffering_interval = 60 # seconds
+    role_arn           = aws_iam_role.kinesis-firehose-stream-role.arn
+    s3_backup_mode     = "FailedDataOnly"
+    retry_duration     = 60 # seconds
+
+    request_configuration {
+      content_encoding = "GZIP"
+    }
+  }
+}
+
+# CloudWatch Metric Stream
+
+resource "aws_iam_role" "cloudwatch-metric-stream-role" {
+  name               = "${var.name_prefix}cloudwatch-metric-stream-role${var.name_suffix}"
+  assume_role_policy = data.aws_iam_policy_document.cloudwatch-metric-stream-role-trust-policy.json
+
+  tags = {
+    Name = "${var.name_prefix}cloudwatch-metric-stream-role${var.name_suffix}"
+  }
+}
+
+resource "aws_iam_role_policy" "cloudwatch-metric-stream-role-firehose-policy" {
+  name   = "${var.name_prefix}MetricStreams-FirehosePutRecords${var.name_suffix}"
+  role   = aws_iam_role.cloudwatch-metric-stream-role.id
+  policy = data.aws_iam_policy_document.cloudwatch-metric-stream-role-firehose-policy.json
+}
+
+resource "aws_cloudwatch_metric_stream" "cloudwatch-metric-stream" {
+  name          = "${var.name_prefix}newrelic-metric-stream${var.name_suffix}"
+  role_arn      = aws_iam_role.cloudwatch-metric-stream-role.arn
+  firehose_arn  = aws_kinesis_firehose_delivery_stream.kinesis-firehose-stream.arn
+  output_format = "opentelemetry0.7"
+
+  dynamic "include_filter" {
+    for_each = var.cloudwatch_metric_stream_include_filter
+    content {
+      namespace = include_filter["value"]
+    }
+  }
+
+  dynamic "exclude_filter" {
+    for_each = var.cloudwatch_metric_stream_exclude_filter
+    content {
+      namespace = exclude_filter["value"]
+    }
+  }
+}

outputs.tf

@@ -0,0 +1,32 @@
+# Output variables file
+# https://www.terraform.io/docs/configuration/modules.html#accessing-module-output-values 
+
+output "newrelic-one-external-role-arn" {
+  description = "The ARN of the provisioned IAM role used for New Relic account linking."
+  value       = aws_iam_role.newrelic-one-external-role.arn
+}
+
+output "kinesis-firehose-stream-arn" {
+  description = "The ARN of the provisioned Kinesis Firehose Delivery Stream."
+  value       = aws_kinesis_firehose_delivery_stream.kinesis-firehose-stream.arn
+}
+
+output "kinesis-firehose-stream-role-arn" {
+  description = "The ARN of the provisioned IAM role used by Kinesis Firehose Delivery Stream."
+  value       = aws_iam_role.kinesis-firehose-stream-role.arn
+}
+
+output "kinesis-firehose-stream-failed-data-s3-bucket-arn" {
+  description = "The ARN of the provisioned S3 bucket used for storing data which Kinesis Firehose failed sending to New Relic."
+  value       = aws_s3_bucket.kinesis-firehose-stream-failed-data.arn
+}
+
+output "cloudwatch-metric-stream-arn" {
+  description = "The ARN of the provisioned CloudWatch Metric Stream."
+  value       = aws_cloudwatch_metric_stream.cloudwatch-metric-stream.arn
+}
+
+output "cloudwatch-metric-stream-role-arn" {
+  description = "The ARN of the provisioned IAM role used by CloudWatch Metric Stream."
+  value       = aws_iam_role.cloudwatch-metric-stream-role.arn
+}