Skip to content

Commit

Permalink
docs: simplify usage file docs
Browse files Browse the repository at this point in the history
  • Loading branch information
@alikhajeh1
alikhajeh1 committed Nov 9, 2023
1 parent fbcda71 commit c7bc153
Show file tree
Hide file tree
Showing 5 changed files with 117 additions and 179 deletions.
2 changes: 1 addition & 1 deletion docs/faq.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ Yes! Please see the [self-hosting guide](/docs/cloud_pricing_api/self_hosted) fo

### Does Infracost need cloud credentials?

No. However, if you want Infracost to fetch usage data from your AWS account, you need to give it [read-only access](/docs/features/usage_based_resources/#credentials).
No! Infracost parses code and uses pricing data to estimate costs.

### Does the Infracost CLI send the Terraform plan to the Cloud Pricing API?

Expand Down
291 changes: 116 additions & 175 deletions docs/features/usage_based_resources.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,41 +6,56 @@ title: Usage-based resources
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

Infracost distinguishes the **price** of a resource from its **cost**. Price is a per-unit value published by cloud vendors. Cost is calculated by multiplying a resource's price by its usage. Infracost shows prices by default for usage-based resources such as AWS Lambda:
Infracost differentiates the **price** of a resource from its **cost**. The price is the per-unit value provided by cloud vendors, while the cost is calculated by multiplying the resource's price by its usage. By default, Infracost displays prices for usage-based resources like AWS S3 or Lambda, providing users with visibility into the relevant prices from the multitude available for their applications.

Furthermore, users have the option to provide usage estimates in a file to calculate costs. This feature not only simplifies rapid what-if analysis but also enables exploration of scenarios. For example, you can assess how the cost estimate changes if a Lambda function experiences a 2x increase in requests.

<Tabs
defaultValue="without-usage-file"
values={[
{label: 'Default output', value: 'without-usage-file'},
{label: 'Output with usage file', value: 'with-usage-file'},
]}>
<TabItem value="without-usage-file">

```
Name Quantity Unit Monthly Cost
Name Quantity Unit Monthly Cost
aws_lambda_function.hi
├─ Requests Cost depends on usage: $0.20 per 1M requests
└─ Duration Cost depends on usage: $0.0000166667 per GB-seconds
```
</TabItem>
<TabItem value="with-usage-file">

PROJECT TOTAL $0.00
```
Name Quantity Unit Monthly Cost
There are two options for showing costs instead of prices:
1. **Specify usage manually**: for doing what-if analysis. For example, what happens to the cost estimate if a Lambda function gets 2x more requests? Usage can be defined for a resource type (e.g. `aws_lambda_function`) or individual resources (e.g. `aws_lambda_function.hi`).
2. **Fetch usage from Cloud APIs**: provides visibility of usage-based costs in the terminal and CI/CD. This currently only works for AWS.
aws_lambda_function.hi
├─ Requests 100 1M requests $20.00
└─ Duration 12,500,000 GB-seconds $208.33
## Specify usage manually
PROJECT TOTAL $228.33
```
</TabItem>
</Tabs>

Instead of using cloud vendor cost calculators, spreadsheets or wiki pages, you can specify usage estimates in a file called [`infracost-usage.yml`](https://github.com/infracost/infracost/blob/master/infracost-usage-example.yml), which can be passed to Infracost using the `--usage-file` flag so it can calculate costs. This enables you to do quick **what-if analysis**; for example, what happens to the cost estimate if a Lambda function gets 2x more requests?
## Usage

This flag should not be confused with the `--config-file` option that is used to [configure](/docs/features/config_file) how Infracost runs.
Instead of using cloud vendor cost calculators or spreadsheets, you can specify usage estimates in an auto-generated file called `infracost-usage.yml`. The Infracost GitHub/GitLab App integrations will use this file automatically when it is placed at the repo root. The CLI can also use this file.

### 1. Generate usage file

Use the `--sync-usage-file` option to generate a new usage file or update an existing one. You must specify the location of the new or existing usage file using the `--usage-file` flag:
Assuming you have [installed](/docs/#1-install-infracost) the Infracost CLI, use the `--sync-usage-file` flag to generate a new usage file or update an existing one. You must specify the location of the new or existing usage file using the `--usage-file` flag:
```sh
infracost breakdown --sync-usage-file --usage-file infracost-usage.yml --path /code
```

This updates the usage file by:
1. Attempting to fetch usage data from cloud APIs and updating any fields with those values. [See here](#fetch-from-cloudwatch) for more information about which resources and fields are supported.
2. Adding any missing resources or fields as comments with a zero value.
3. Deleting any resources that are not used in the Terraform project.
This creates/updates the usage file by:
1. Adding any missing resources or fields as comments with a zero value.
2. Deleting any resources that are not used in the Terraform project.

When using the `--usage-file` flag with the `breakdown` or `output` commands, cost components with a 0 hourly/monthly quantity are not shown in table and HTML formats so the output is less noisy. These are included in the JSON format.
When using the `--usage-file` flag with the `breakdown` or `output` commands, cost components with a 0 hourly/monthly quantity are not shown so the output is less noisy. These are included in the JSON format.

### 2. Edit usage file

Expand All @@ -56,7 +71,7 @@ Edit the generated usage file with your usage estimates, for example a Lambda fu
### 3. Run with usage file
Run `infracost breakdown` or `infracost diff` with the usage file to see monthly cost estimates:
The Infracost GitHub/GitLab App integrations will use the `infracost-usage.yml` file automatically when it is placed at the repo root. You can also run `infracost breakdown` or `infracost diff` with the usage file to see monthly cost estimates:

```sh
infracost breakdown --path /code --usage-file infracost-usage.yml
Expand All @@ -70,13 +85,91 @@ Run `infracost breakdown` or `infracost diff` with the usage file to see monthly
PROJECT TOTAL $228.33
```

### Supported parameters

The **reference file [infracost-usage-example.yml](https://github.com/infracost/infracost/blob/master/infracost-usage-example.yml)** contains the list of all of the available parameters and their descriptions.
## Usage profiles

You can use the `resource_type_default_usage` section of the usage file and create separate files for different traffic profiles, e.g. low/medium/high shown below. This enables you to get a rough estimate for many projects quickly without defining usage values for each individual resource in those projects, by running `infracost breakdown --path /code --usage-file infracost-usage-medium.yml`.

#### Resource type defaults
In the future, Infracost Cloud will enable you do setup these usage profiles and use them across all repos in your organization.

Usage for a resource type, e.g. `aws_dynamodb_table`, can also be defined in the usage file. Resource type defaults apply to all resources of that type regardless of the module they reside in.
<Tabs
defaultValue="infracost-usage-low"
values={[
{label: 'infracost-usage-low.yml', value: 'infracost-usage-low'},
{label: 'infracost-usage-medium.yml', value: 'infracost-usage-medium'},
{label: 'infracost-usage-high.yml', value: 'infracost-usage-high'}
]}>
<TabItem value="infracost-usage-low">

```yml
version: 0.1
resource_type_default_usage:
aws_lambda_function:
monthly_requests: 1000000
request_duration_ms: 100
aws_dynamodb_table:
storage_gb: 5
monthly_write_request_units: 20
monthly_read_request_units: 40
aws_cloudwatch_log_group:
storage_gb: 10
monthly_data_ingested_gb: 10
monthly_data_scanned_gb: 10
```
</TabItem>
<TabItem value="infracost-usage-medium">

```yml
version: 0.1
resource_type_default_usage:
aws_lambda_function:
monthly_requests: 5000000
request_duration_ms: 200
aws_dynamodb_table:
storage_gb: 100
monthly_write_request_units: 50
monthly_read_request_units: 70
aws_cloudwatch_log_group:
storage_gb: 100
monthly_data_ingested_gb: 100
monthly_data_scanned_gb: 100
```
</TabItem>
<TabItem value="infracost-usage-high">

```yml
version: 0.1
resource_type_default_usage:
aws_lambda_function:
monthly_requests: 10000000
request_duration_ms: 200
aws_dynamodb_table:
storage_gb: 500
monthly_write_request_units: 90
monthly_read_request_units: 140
aws_cloudwatch_log_group:
storage_gb: 500
monthly_data_ingested_gb: 500
monthly_data_scanned_gb: 500
```
</TabItem>
</Tabs>

## Supported parameters

### Reference file

The [infracost-usage-example.yml](https://github.com/infracost/infracost/blob/master/infracost-usage-example.yml) reference file contains the list of all of the available parameters and their descriptions for all resource types. These parameters can be added to either a resource (e.g. `aws_dynamodb_table.mytable`) or a resource type (e.g. `aws_dynamodb_table`) using the resource type defaults mentioned below.

### Resource type defaults

Usage for a resource type, e.g. `aws_dynamodb_table`, can also be defined in the `resource_type_default_usage` section of the usage file. Resource type defaults apply to all resources of that type regardless of the module they reside in.

This is useful when you want to create traffic profiles such as lower/medium/high. Resource type defaults can be overridden on a per-resource basis (shown below); usage keys that are re-defined at a resource level override the default, and new usage keys are merged with the defaults.

Expand All @@ -94,7 +187,7 @@ resource_usage:
storage_gb: 50 # Overrides the default
```

#### Terraform modules
### Terraform modules

Usage for resources inside modules can be specified using the full path of the resource. This is the same value as Infracost outputs in the Name column, for example:

Expand All @@ -109,7 +202,7 @@ resource_usage:
request_duration_ms: 600
```

#### Resource arrays/maps
### Resource arrays/maps

The wildcard character `[*]` can be used for resource arrays (resources with [`count` meta-argument](https://www.terraform.io/docs/language/meta-arguments/count.html)) and resource maps (resources with [`for_each` meta-argument](https://www.terraform.io/docs/language/meta-arguments/for_each.html)), such as AWS CloudWatch Log Groups. Infracost will apply the usage values individually to each element of the array/map (they all get the same values). If both an array element such as `this[0]` (or map element such as `this["foo"]`) and `[*]` are specified for a resource, only the array/map element's usage will be applied to that resource. This enables you to define default values using `[*]` and override specific elements using their index or key.

Expand Down Expand Up @@ -188,7 +281,7 @@ When wildcard entries exist in the usage file and `--sync-usage-file` is used:
</TabItem>
</Tabs>

#### EC2 reserved instances
### EC2 reserved instances

What-if anlaysis can be done on AWS EC2 Reserved Instances (RI) using the usage file. The RI type, term and payment option can be defined as shown below, to quickly get a monthly cost estimate. This works with `aws_instance` as well as `aws_eks_node_group` and `aws_autoscaling_group` as they also create EC2 instances. Let us know how you'd like Infracost to show the upfront costs by [creating a GitHub issue](https://github.com/infracost/infracost/issues/).

Expand All @@ -201,155 +294,3 @@ What-if anlaysis can be done on AWS EC2 Reserved Instances (RI) using the usage
reserved_instance_term: 1_year # Term for Reserved Instances. Can be: 1_year, 3_year.
reserved_instance_payment_option: all_upfront # Payment option for Reserved Instances. Can be: no_upfront, partial_upfront, all_upfront.
```

## Usage profiles

You can use the `resource_type_default_usage` section of the usage file and create separate files for different traffic profiles, e.g. low/medium/high shown below. This enables you to get a rough estimate for many projects quickly without defining usage values for each individual resource in those projects, by running `infracost breakdown --path /code --usage-file infracost-usage-medium.yml`.

<Tabs
defaultValue="infracost-usage-low"
values={[
{label: 'infracost-usage-low.yml', value: 'infracost-usage-low'},
{label: 'infracost-usage-medium.yml', value: 'infracost-usage-medium'},
{label: 'infracost-usage-high.yml', value: 'infracost-usage-high'}
]}>
<TabItem value="infracost-usage-low">

```yml
version: 0.1
resource_type_default_usage:
aws_lambda_function:
monthly_requests: 1000000
request_duration_ms: 100
aws_dynamodb_table:
storage_gb: 5
monthly_write_request_units: 20
monthly_read_request_units: 40
aws_cloudwatch_log_group:
storage_gb: 10
monthly_data_ingested_gb: 10
monthly_data_scanned_gb: 10
```
</TabItem>
<TabItem value="infracost-usage-medium">

```yml
version: 0.1
resource_type_default_usage:
aws_lambda_function:
monthly_requests: 5000000
request_duration_ms: 200
aws_dynamodb_table:
storage_gb: 100
monthly_write_request_units: 50
monthly_read_request_units: 70
aws_cloudwatch_log_group:
storage_gb: 100
monthly_data_ingested_gb: 100
monthly_data_scanned_gb: 100
```
</TabItem>
<TabItem value="infracost-usage-high">

```yml
version: 0.1
resource_type_default_usage:
aws_lambda_function:
monthly_requests: 10000000
request_duration_ms: 200
aws_dynamodb_table:
storage_gb: 500
monthly_write_request_units: 90
monthly_read_request_units: 140
aws_cloudwatch_log_group:
storage_gb: 500
monthly_data_ingested_gb: 500
monthly_data_scanned_gb: 500
```
</TabItem>
</Tabs>

## Fetch from cloud APIs

:::tip
We recommend using the new [Actual Costs](/docs/infracost_cloud/actual_costs) feature instead.
:::

We **experimented** with fetching usage data from AWS CloudWatch/cloud APIs, which provides you with visibility of usage-based costs in the terminal and CI/CD. However, a better approach is to fetch this data from the [AWS Cost and Usage Reports](/docs/infracost_cloud/actual_costs).
```
infracost breakdown --path /code --sync-usage-file --usage-file /tmp/ignore.yml
```
See the docs for the [usage file](/docs/features/usage_based_resources/#specify-usage-manually) if you're interested in editing it manually.
This currently only works for AWS and enables you to quickly see the last 30-day usage (and costs) for the following resources:
- **aws_dynamodb_table**: storage_gb, monthly_read_request_units and monthly_write_request_units
- **aws_lambda_function**: monthly_requests and request_duration_ms
- **aws_s3_bucket**:
- Standard storage class: storage_gb, monthly_tier_1_requests, monthly_tier_2_requests, monthly_select_data_scanned_gb and monthly_select_data_returned_gb
- Intelligent tiering storage class: frequent_access_storage_gb, infrequent_access_storage_gb, archive_access_storage_gb and deep_archive_storage_gb
- Other storage classes: storage_gb
- **aws_instance**, **aws_autoscaling_group**, **aws_eks_node_group**: operating_system (based on the AMI, detected as one of: linux, windows, suse, rhel)
- **aws_autoscaling_group** and **aws_eks_node_group**: instances. If unable to fetch the last 30-day average from CloudWatch this will fetch the current instance count from the AWS API instead.
### Credentials
This functionality uses the AWS credentials from the default AWS credential provider chain. To set or override these use the `AWS_PROFILE`, `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables. These can also be set in a [config-file](/docs/features/config_file/) for multi-project setups.
If you're using AssumeRole, something like the following should work but only for one AWS account. Terraform/Terragrunt projects that use multiple AWS accounts are not supported with this method.
```shell
aws sts assume-role --role-arn <arn> --role-session-name <session name>
# extract the AccessKeyId, SecretAccessKey and SessionToken
export AWS_ACCESS_KEY_ID=<AccessKeyId>
export AWS_SECRET_ACCESS_KEY=<SecretAccessKey>
export AWS_SESSION_TOKEN=<SessionToken>
infracost breakdown --path /code --sync-usage-file --usage-file /tmp/ignore.yml
```

Your AWS credentials need the following IAM permissions for this to work. These are likely to be already defined if you're using the same AWS credentials that you use for generating your Terraform plan JSON file. The following will be updated as we add support for more resources.

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "InfracostSyncUsage",
"Effect": "Allow",
"Action": [
"ec2:DescribeImages",
"eks:DescribeNodegroup",
"dynamodb:DescribeTable",
"autoscaling:DescribeAutoScalingGroups",
"s3:GetMetricsConfiguration",
"cloudwatch:GetMetricStatistics"
],
"Resource": "*"
}
]
}
```

### See usage costs in CI/CD

The following workaround can be used in the terminal or CI/CD systems so a usage file does not need to be created in advance. The `/tmp/ignore.yml` file can simply be ignored; in the future we might make this an optional flag, so usage data can be fetched from CloudWatch/cloud APIs without the need for a usage file.
```sh
# Generate an Infracost JSON file, including usage-based costs
infracost breakdown --path /code \
--sync-usage-file --usage-file /tmp/ignore.yml \
--format json --out-file infracost-base.json

# Show a breakdown in text format, from the Infracost JSON file
infracost output --path infracost-base.json --format table

# Post a comment using the Infracost JSON file.
# Run `infracost comment --help` to see the other required flags.
infracost comment github --path infracost-base.json ...
```
1 change: 0 additions & 1 deletion docs/guides/actions_migration.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ Changing your workflow to work with the parse HCL option requires the following
1. Remove the Terraform and Terragrunt dependencies:
- Delete any `hashicorp/setup-terraform` or `autero1/action-terragrunt` steps as Infracost now parses the HCL code directly, so it does not depend on these.
- Delete any step that runs `terraform` or `terragrunt`, e.g. "terraform init", "terraform plan" and "terraform show" are no longer needed.
- If you are not using the [fetch usage from CloudWatch](/docs/features/usage_based_resources/#fetch-from-cloudwatch) feature, delete any steps that set cloud credentials.

2. Bump the version of the `infracost/actions/setup` action from `v1` to `v2`:

Expand Down
1 change: 0 additions & 1 deletion docs/guides/azure_devops_migration.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ Changing your workflow to work with the parse HCL option requires the following
1. Remove the Terraform and Terragrunt dependencies:
- Delete any `TerraformInstaller` steps as Infracost now parses the HCL code directly, so it does not depend on these.
- Delete any stages and jobs that runs `terraform` or `terragrunt`, e.g. "terraform init", "terraform plan" and "terraform show" are no longer needed.
- If you are not using the [fetch usage from CloudWatch](/docs/features/usage_based_resources/#fetch-from-cloudwatch) feature, delete any steps that set cloud credentials.

2. Bump the version of the InfracostSetup task from `0` to `1`:

Expand Down
Loading

0 comments on commit c7bc153

Please sign in to comment.