`digitalocean_app` redeploys even though no `env` variables changed #782

jameyg · 2022-01-26T15:41:57Z

Bug Report

Describe the bug

The digitalocean_app resource thinks that the env arguments are changing on every Terraform apply. It seems to only happen when the env are of type "SECRET" rather than "GENERAL".

Affected Resource(s)

digitalocean_app

Expected Behaviour

It should not redeploy the app.

Actual Behaviour

It redeployed the app.

Steps to Reproduce

Once I have the below app deployed to Digital Ocean, if I run terraform apply all of the environment variables with the type "SECRET" show as being changed and it forces a rebuild.

Terraform Configuration Files


resource "digitalocean_app" "app" {
  spec {
    name   = var.project_name
    region = var.instance_region
    domain {
      name = var.api_domain_name
      type = "PRIMARY"
      zone = "[redacted].com"
    }

    env {
      key   = "DJANGO_SETTINGS_MODULE"
      value = var.django_settings_module
      type  = "GENERAL"
    }
    env {
      key   = "PYTHONPATH"
      value = "./[redacted]"
      type  = "GENERAL"
    }
    env {
      key   = "DATABASE_URL"
      value = "$${db.DATABASE_URL}"
      scope = "RUN_TIME"
      type  = "SECRET"
    }
    env {
      key   = "DOMAIN_NAME"
      value = var.api_domain_name
      type  = "GENERAL"
    }
    env {
      key   = "BUCKET_NAME"
      value = digitalocean_spaces_bucket.bucket.name
      type  = "GENERAL"
    }
    env {
      key   = "AWS_S3_ENDPOINT_URL"
      value = "https://${var.bucket_region}.digitaloceanspaces.com"
      type  = "GENERAL"
    }
    env {
      key   = "AWS_ACCESS_KEY_ID"
      value = var.spaces_access_id
      type  = "SECRET"
    }
    env {
      key   = "AWS_SECRET_ACCESS_KEY"
      value = var.spaces_secret_key
      type  = "SECRET"
    }
    env {
      key   = "SENTRY_DSN"
      value = var.sentry_dsn
      type  = "SECRET"
    }

    service {
      name               = "api"
      environment_slug   = "python"
      instance_count     = 1
      instance_size_slug = var.api_instance_size
      http_port          = 8000
      run_command        = "gunicorn --worker-tmp-dir /dev/shm [redacted].wsgi"

      github {
        repo           = "[redacted]"
        branch         = "main"
        deploy_on_push = true
      }
    }

    job {
      name               = "setup"
      kind               = "POST_DEPLOY"
      environment_slug   = "python"
      instance_count     = 1
      instance_size_slug = "basic-xxs"
      run_command        = "python manage.py createcachetable; python manage.py migrate"
      github {
        repo           = "[redacted]"
        branch         = "main"
        deploy_on_push = true
      }
    }

    database {
      name       = "db"
      engine     = "PG"
      production = var.database_is_production
    }
  }
}

I'm running Terraform v1.1.4 and v2.17.0 of this library.

References

This seems to be the same issue:
#514

The two most recent commenters after the issue closed seem to be experiencing a similar issue to me.

The text was updated successfully, but these errors were encountered:

hstaudacher · 2022-03-20T15:55:16Z

I also ran into this issue. Is there an eta when this can be fixed?

driedel1987 · 2022-06-05T07:51:25Z

Any updates here ?

zafai · 2022-06-23T07:47:13Z

Any new information?
We are facing this issue and new it solved, please.
Thank you

scotchneat · 2022-06-23T19:21:52Z

Thank you all for bringing this issue to our attention.
@jameyg can you confirm that the env vars with type = "SECRET" in your config have respective encrypted values being passed in (typically through a tfvars file).

The fix mentioned in the #514 was to Suppress diff when env type is GENERAL. However, if the values for any SECRET vars are not the same encrypted values returned by the API, it will result in a diff.

I will attempt to reproduce this issue to investigate further.

driedel1987 · 2022-07-04T07:36:32Z

@scotchneat are you able to reproduce the problem and has an idea when a fix could be releases ?

scotchneat · 2022-07-07T05:21:03Z

@driedel1987 yes.

When using the env vars with type = "SECRET" as I mentioned in my previous comment, app platform will encrypt and store the secret, then returns the encrypted value in the response. Terraform then stores the encrypted value in state.

The problem is, if the same plain-text value sent in the initial apply is sent on any subsequent apply, the values won't match the encrypted value stored in the state so it creates a diff. This is expected behavior and not a bug in the provider.

I realize this scenario isn't properly covered in the documentation. I'll see if I can elaborate on this in the documentation and probably an example config

driedel1987 · 2022-07-07T07:10:15Z

@scotchneat thanks for your answer. But I have the same behavior if i only use “general” as type in my app (pulumi/pulumi-digitalocean#308).

I am using the following code to roll out my app/worker in digital ocean:

func RegisterApp(UserID, EncryptionPassPhrase, AwsAccessKeyID, AwsGENERALAccessKey, AwsRegion, SlackWebhookDomain, SlackTeam, SlackService, SlackToken, ShedulerImageTag, CloudFlareDomain, AppAccessKeyID, AppBotToken, AppClientID, AppClientGENERAL, AppSpaceEndpoint, AppGENERALAccessKey, AppServerIP, AppPort, CustomerDBHost, CustomerDBPort, AppCA, AppImageTag, DashboardCorsDomain string, di *models.DatabaseUserInformation, ai *models.AppUserInformation) (pulumi.RunFunc, error) {
	return func(ctx *pulumi.Context) error {
		App, err := digitalocean.NewApp(ctx, UserID, &digitalocean.AppArgs{
			Spec: &digitalocean.AppSpecArgs{
				Name:        pulumi.String(UserID),
				Region:      pulumi.String("fra"),
				DomainNames: &digitalocean.AppSpecDomainNameArray{digitalocean.AppSpecDomainNameArgs{Name: pulumi.String(ai.Name + CloudFlareDomain)}},
				Workers: &digitalocean.AppSpecWorkerArray{&digitalocean.AppSpecWorkerArgs{
					Name:             pulumi.String("golang-worker"),
					Image:            &digitalocean.AppSpecWorkerImageArgs{RegistryType: pulumi.String("DOCR"), Repository: pulumi.String("sheduled-job"), Tag: pulumi.String(ShedulerImageTag)},
					InstanceCount:    pulumi.Int(1),
					InstanceSizeSlug: pulumi.String("basic-xxs"),
					Alerts: &digitalocean.AppSpecWorkerAlertArray{&digitalocean.AppSpecWorkerAlertArgs{Rule: pulumi.String("CPU_UTILIZATION"), Operator: pulumi.String("GREATER_THAN"), Value: pulumi.Float64(85), Window: pulumi.String("FIVE_MINUTES")},
						&digitalocean.AppSpecWorkerAlertArgs{Rule: pulumi.String("MEM_UTILIZATION"), Operator: pulumi.String("GREATER_THAN"), Value: pulumi.Float64(85), Window: pulumi.String("FIVE_MINUTES")},
						&digitalocean.AppSpecWorkerAlertArgs{Rule: pulumi.String("RESTART_COUNT"), Operator: pulumi.String("GREATER_THAN"), Value: pulumi.Float64(5), Window: pulumi.String("FIVE_MINUTES")}},
					Envs: &digitalocean.AppSpecWorkerEnvArray{
						&digitalocean.AppSpecWorkerEnvArgs{
							Key:   pulumi.String("DB_HOST"),
							Scope: pulumi.String("RUN_AND_BUILD_TIME"),
							Type:  pulumi.String("GENERAL"),
							Value: pulumi.String(CustomerDBHost),
						},
						&digitalocean.AppSpecWorkerEnvArgs{
							Key:   pulumi.String("DB_PORT"),
							Scope: pulumi.String("RUN_AND_BUILD_TIME"),
							Type:  pulumi.String("GENERAL"),
							Value: pulumi.String(CustomerDBPort),
						},
...............

It creates the app/worker successfully in digital ocean without using any type of secret.

But if I try an update directly after the creation of the app, Pulumi recognizes changes which are not present. I am using the same function for updating as I am using for creation (see above).

The Preview shows the following output in terminal for example: It seems that Pulumi wants to change the order or something like that. If I execute the preview, it runs in a timeout.

 ~ spec: {
          ~ services: [
              ~ [0]: {
                      ~ envs  : [
                          ~ [0]: {
                                  ~ key  : "CUSTOMERID" => "CORS_DOMAIN"
                                  ~ scope: "RUN_AND_BUILD_TIME" => "RUN_AND_BUILD_TIME"
                                  ~ value: [secret] => "http://localhost:3000"
                                }
                          ~ [1]: {
                                  ~ key  : "BOT_TOKEN" => "CLIENT_ID"
                                  ~ scope: "RUN_AND_BUILD_TIME" => "RUN_AND_BUILD_TIME"
                                  ~ value: [secret] => "Value"
                                }
                          ~ [2]: {
                                  ~ key  : "DB_HOST" => "AWS_SES_ACCESS_KEY_ID"
                                  ~ scope: "RUN_AND_BUILD_TIME" => "RUN_AND_BUILD_TIME"
                                  ~ value: [secret] => "Value"
                                }
                          ~ [3]: {
                                  ~ key  : "DB_PORT" => "AWS_SES_GENERAL_ACCESS_KEY"
                                  ~ scope: "RUN_AND_BUILD_TIME" => "RUN_AND_BUILD_TIME"
                                  ~ value: [secret] => "Value"
                                }
                          ~ [4]: {
                                  ~ key  : "EncryptionPassPhrase" => "AWS_SES_REGION"
                                  ~ scope: "RUN_AND_BUILD_TIME" => "RUN_AND_BUILD_TIME"
                                  ~ value: [secret] => "eu-central-1"

scotchneat · 2022-07-08T22:46:22Z

Thanks for the details @driedel1987.

I realize you're sharing only a snippet of the code and the diff output, but what stands out in the planned change is that the env vars listed in your code are supposed to be changed to other values (e.g. "DB_HOST" => "AWS_SES_ACCESS_KEY_ID"). This makes me think the pulumi library has some dependency on the order of those values. From my testing, plain terraform doesn't diff when env vars order changes.

However, since that's beyond the function or control of this provider, I'd like to avoid confusing the experience from the initial request in this issue with the behavior/interaction between pulumi and the DO Terraform provider. In the pulumi issue you referenced, @stack72 points to this issue being related to the behavior experienced in the pulumi library. @stack72 could you please share more details about the interaction? If it is beyond the experience described by the initial reporter of this issue, I think it makes more sense that you create a new issue so the topic can be specific to the pulumi > digitalocean provider integration.

voxivoid · 2022-07-20T08:28:35Z

are there any workarounds for the SECRET env vars always triggering changes? every time I apply my terraform it causes the app being deployed even though nothing changed

jameyg · 2022-10-19T11:26:27Z

@scotchneat Sorry I didn't see the notifications for this come through.

I ended up changing everything to "general" rather than "secret" which almost fixed my issue but not completely. I have three environments (dev/staging/prod) powered by the same module and for some reason the staging environment always retriggers a change and deployment stating that something in the env block has changed. However, if that were the case then both the other two environments should also be triggering a change and deployment.

I'm not sure what files or information to attach but I'm using Terraform without any abstractions beyond it running in a Docker container locally.

finn-christiansen · 2022-12-08T16:02:00Z

Any updates or new workarounds on this? I am still facing this issue.

StanScates · 2023-09-11T04:03:10Z

Looks like a duplicate of #869

danaelhe · 2023-09-13T15:55:56Z

We've pushed up an example that offers a workaround in #1030

We are aware this workaround adds a few more manual steps, but it should mitigate the issue enough until we find a better solution. Please feel free to offer feedback in that PR.

brainrake · 2024-11-05T16:28:07Z

Let's keep this issue open until it is fixed!

jameyg added the bug label Jan 26, 2022

jameyg changed the title ~~digitalocean_app redeploys even though no env variables changed~~ digitalocean_app redeploys even though no env variables changed Jan 27, 2022

stack72 mentioned this issue May 30, 2022

Env Issue on https://www.pulumi.com/registry/packages/digitalocean/api-docs/app/#create pulumi/pulumi-digitalocean#308

Open

scotchneat removed the bug label Jul 7, 2022

ChiefMateStarbuck added the Needs Investigation label Mar 28, 2023

danaelhe closed this as completed Sep 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`digitalocean_app` redeploys even though no `env` variables changed #782

`digitalocean_app` redeploys even though no `env` variables changed #782

jameyg commented Jan 26, 2022 •

edited

Loading

hstaudacher commented Mar 20, 2022

driedel1987 commented Jun 5, 2022

zafai commented Jun 23, 2022

scotchneat commented Jun 23, 2022

driedel1987 commented Jul 4, 2022 •

edited

Loading

scotchneat commented Jul 7, 2022

driedel1987 commented Jul 7, 2022 •

edited

Loading

scotchneat commented Jul 8, 2022

voxivoid commented Jul 20, 2022

jameyg commented Oct 19, 2022

finn-christiansen commented Dec 8, 2022

StanScates commented Sep 11, 2023

danaelhe commented Sep 13, 2023

brainrake commented Nov 5, 2024

digitalocean_app redeploys even though no env variables changed #782

digitalocean_app redeploys even though no env variables changed #782

Comments

jameyg commented Jan 26, 2022 • edited Loading

Bug Report

Describe the bug

Affected Resource(s)

Expected Behaviour

Actual Behaviour

Steps to Reproduce

hstaudacher commented Mar 20, 2022

driedel1987 commented Jun 5, 2022

zafai commented Jun 23, 2022

scotchneat commented Jun 23, 2022

driedel1987 commented Jul 4, 2022 • edited Loading

scotchneat commented Jul 7, 2022

driedel1987 commented Jul 7, 2022 • edited Loading

scotchneat commented Jul 8, 2022

voxivoid commented Jul 20, 2022

jameyg commented Oct 19, 2022

finn-christiansen commented Dec 8, 2022

StanScates commented Sep 11, 2023

danaelhe commented Sep 13, 2023

brainrake commented Nov 5, 2024

`digitalocean_app` redeploys even though no `env` variables changed #782

`digitalocean_app` redeploys even though no `env` variables changed #782

jameyg commented Jan 26, 2022 •

edited

Loading

driedel1987 commented Jul 4, 2022 •

edited

Loading

driedel1987 commented Jul 7, 2022 •

edited

Loading