Resources are not sensitive to provider's default project ID changes

[sylviamoss]

Description

Resources are not sensitive to the provider's default project ID changes. A plan modifier is already implemented and used by the hcp_notifications_webhook resource.

https://github.com/hashicorp/terraform-provider-hcp/blob/main/internal/provider/modifiers/project.go#L26
https://github.com/hashicorp/terraform-provider-hcp/blob/main/internal/provider/webhook/resource_notifications_webhook.go#L195

Affected Resource(s)

The majority of resources in the provider do not implement a modifier to identify default project ID changes.

Terraform Configuration Files

terraform {
  required_providers {
    hcp = {
       source  = "hashicorp/hcp"
       version = ">=0.65.0"
    }
  }
}

provider "hcp" {
  project_id = "<project_id>"
}

resource "hcp_vault_secrets_app" "example" {
  app_name    = "example-app-name"
  description = "My new app!"
}

Steps to Reproduce

1. terraform apply
2. Change the default project id
3. terraform plan/apply

Expected Behavior

After the provider's default project ID is changed, any resource inheriting the ID should be recreated under the new project.

Actual Behavior

After the provider's default project ID is changed, the plan/apply will fail to read the resource or depending on the resource not identify the changes.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[manish-hashicorp]

After the provider's default project ID is changed, the plan/apply will fail to read the resource or depending on the resource not identify the changes.

@sylviamoss Does the configured credential have access to both old and new project? Also, can you give example of kind of error shown?

[sylviamoss]

@manish-hashicorp There are several possible scenarios. For this problem to be silent the credential has to have access to both projects, but if it doesn't, the resource using the old project will fail to read instead of destroying and creating.

I ran an example to get the output for different use cases.

In the following examples I am using the configuration:

terraform {
  required_providers {
    hcp = {
      source  = "hashicorp/hcp"
      version = ">=0.65.0"
    }
  }
}
variable "project_id" {
  description = "The project id."
  type        = string
}
provider "hcp" {
  project_id = var.project_id
}

resource "hcp_service_principal" "example" {
  name   = "example-sp"
}

1. Org credentials with access to both projects. First, create the service principal in the provider's default project.

tf-help git:(main) ✗ terraform apply -var="project_id=ac8e2f32-37de-492f-9a2d-51c3b9a03089"

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # hcp_service_principal.example will be created
  + resource "hcp_service_principal" "example" {
      + name          = "example-sp"
      + parent        = (known after apply)
      + resource_id   = (known after apply)
      + resource_name = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

hcp_service_principal.example: Creating...
hcp_service_principal.example: Creation complete after 0s [name=example-sp]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Then re-apply changing the provider's default project.

➜  tf-help git:(main) ✗ terraform apply -var="project_id=82950878-75b8-464a-bb02-8d07f6083085"
hcp_service_principal.example: Refreshing state... [name=example-sp]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
➜  tf-help git:(main) ✗ 

The service principal continues in the old project and is not created in the new one.

[sylviamoss]

Still using org credentials, now I added hcp_project_iam_binding.

provider "hcp" {
  project_id = var.project_id
}

resource "hcp_service_principal" "sp" {
  name   = "example-sp-new-name"
}

resource "hcp_project_iam_binding" "example" {
  principal_id = hcp_service_principal.sp.resource_id
  role         = "roles/contributor"
}

Creating them using provider's default project

tf-help git:(main) ✗ terraform apply -var="project_id=ac8e2f32-37de-492f-9a2d-51c3b9a03089"

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # hcp_project_iam_binding.example will be created
  + resource "hcp_project_iam_binding" "example" {
      + principal_id = (known after apply)
      + role         = "roles/contributor"
    }

  # hcp_service_principal.sp will be created
  + resource "hcp_service_principal" "sp" {
      + name          = "example-sp-new-name"
      + parent        = (known after apply)
      + resource_id   = (known after apply)
      + resource_name = (known after apply)
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

hcp_service_principal.sp: Creating...
hcp_service_principal.sp: Creation complete after 0s [name=example-sp-new-name]
hcp_project_iam_binding.example: Creating...
hcp_project_iam_binding.example: Creation complete after 1s

Re-applying without changing the configuration. hcp_project_iam_binding seems to be sensitive to provider's default project, but since hcp_service_principal is not:

tf-help git:(main) ✗ terraform apply -var="project_id=82950878-75b8-464a-bb02-8d07f6083085"
hcp_service_principal.sp: Refreshing state... [name=example-sp-new-name]
hcp_project_iam_binding.example: Refreshing state...

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # hcp_project_iam_binding.example will be created
  + resource "hcp_project_iam_binding" "example" {
      + principal_id = "example-sp-new-name-023501@ac8e2f32-37de-492f-9a2d-51c3b9a03089"
      + role         = "roles/contributor"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

hcp_project_iam_binding.example: Creating...
╷
│ Error: failed to retrieve organization IAM policy
│ 
│   with hcp_project_iam_binding.example,
│   on main.tf line 98, in resource "hcp_project_iam_binding" "example":
│   98: resource "hcp_project_iam_binding" "example" {
│ 
│ [PUT /resource-manager/2019-12-10/projects/{id}/iam-policy][400] ProjectService_SetIamPolicy default  &{Code:9 Details:[] Message:failed to set IAM policy for project: not all service principals
│ in the policy are members of the org or the project}

[manish-hashicorp]

Closed with #808.

[sylviamoss]

Hey @manish-hashicorp, PR #808 doesn't fix all the affected resources but only some of them. I will reopen the issue. Let me know if you think otherwise.