Introducing a Default ForceNew field causes replacements

[t0yv0]

What happened?

When a Terraform provider introduces a new field that is Optional, has a Default, and is ForceNew, the change does not break Terraform programs but does break Pulumi programs.

Example

For example consider this:
pulumi/pulumi-aws#4653

Terraform provider introduced a new field:

			"api_type": {
				Type:             schema.TypeString,
				Optional:         true,
				ValidateDiagFunc: enum.Validate[awstypes.GraphQLApiType](),
				ForceNew:         true,
				Default:          awstypes.GraphQLApiTypeGraphql,
			},

This provider upgrade is a no-op in Terraform, but is a replacement in Pulumi.

Upgrading AWS versions 6.50.1 -> 6.56.0:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const api = new aws.appsync.GraphQLApi("api", {
    authenticationType: "API_KEY",
}, {
    ignoreChanges: ["apiType"],
});

{
  "method": "/pulumirpc.ResourceProvider/Diff",
  "request": {
    "id": "tqsvg6lljndjdb5blhd5zqig3e",
    "urn": "urn:pulumi:dev::pu::aws:appsync/graphQLApi:GraphQLApi::api",
    "olds": {
      "additionalAuthenticationProviders": [],
      "arn": "arn:aws:appsync:us-east-1:616138583583:apis/tqsvg6lljndjdb5blhd5zqig3e",
      "authenticationType": "API_KEY",
      "enhancedMetricsConfig": null,
      "id": "tqsvg6lljndjdb5blhd5zqig3e",
      "introspectionConfig": "ENABLED",
      "lambdaAuthorizerConfig": null,
      "logConfig": null,
      "name": "api-9e56f1b",
      "openidConnectConfig": null,
      "queryDepthLimit": 0,
      "resolverCountLimit": 0,
      "uris": {
        "GRAPHQL": "https://bahu3gx3kje77akdzd5qxhn2jm.appsync-api.us-east-1.amazonaws.com/graphql",
        "REALTIME": "wss://bahu3gx3kje77akdzd5qxhn2jm.appsync-realtime-api.us-east-1.amazonaws.com/graphql"
      },
      "userPoolConfig": null,
      "visibility": "GLOBAL",
      "xrayEnabled": false
    },
    "news": {
      "__defaults": [
        "apiType",
        "introspectionConfig",
        "name",
        "queryDepthLimit",
        "resolverCountLimit",
        "visibility"
      ],
      "apiType": "GRAPHQL",
      "authenticationType": "API_KEY",
      "introspectionConfig": "ENABLED",
      "name": "api-9e56f1b",
      "queryDepthLimit": 0,
      "resolverCountLimit": 0,
      "visibility": "GLOBAL"
    },
    "oldInputs": {
      "__defaults": [
        "introspectionConfig",
        "name",
        "queryDepthLimit",
        "resolverCountLimit",
        "visibility"
      ],
      "authenticationType": "API_KEY",
      "introspectionConfig": "ENABLED",
      "name": "api-9e56f1b",
      "queryDepthLimit": 0,
      "resolverCountLimit": 0,
      "visibility": "GLOBAL"
    }
  },
  "response": {
    "replaces": [
      "apiType"
    ],
    "stables": [
      "visibility"
    ],
    "changes": "DIFF_SOME",
    "diffs": [
      "apiType"
    ],
    "detailedDiff": {
      "apiType": {
        "kind": "ADD_REPLACE"
      }
    },
    "hasDetailedDiff": true
  },
  "metadata": {
    "kind": "resource",
    "mode": "client",
    "name": "aws"
  }
}

Output of pulumi about

CLI          
Version      3.130.0
Go Version   go1.22.6
Go Compiler  gc

Plugins
KIND      NAME    VERSION
resource  aws     6.56.0
resource  awsx    2.16.1
resource  docker  4.5.7
resource  docker  3.6.1
language  nodejs  unknown

Host     
OS       darwin
Version  14.6.1
Arch     arm64

This project is written in nodejs: executable='/Users/anton/bin/node' version='v18.18.2'

Current Stack: anton-pulumi-corp/pu/dev

TYPE                               URN
pulumi:pulumi:Stack                urn:pulumi:dev::pu::pulumi:pulumi:Stack::pu-dev
pulumi:providers:aws               urn:pulumi:dev::pu::pulumi:providers:aws::default_6_50_1
aws:appsync/graphQLApi:GraphQLApi  urn:pulumi:dev::pu::aws:appsync/graphQLApi:GraphQLApi::api


Found no pending operations associated with dev

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/anton-pulumi-corp
User           anton-pulumi-corp
Organizations  anton-pulumi-corp, moolumi, demo, pulumi
Token type     personal

Dependencies:
NAME            VERSION
@pulumi/aws     6.56.0
@pulumi/awsx    2.16.1
@pulumi/pulumi  3.136.1
@types/node     18.19.56
typescript      5.6.3

Pulumi locates its logs in /var/folders/gd/3ncjb1lj5ljgk8xl5ssn_gvc0000gn/T/com.apple.shortcuts.mac-helper// by default

Additional context

I have not fully chased down why this happens, but I suspect that the problem is that the Node SDK applies defaults. There is a chance that if we removed the default from the Node SDK, then the bridge could figure out the correct plan. This needs verifying though.

Another curious bit is that the bridge figures out that this is a default value (__defaults). Possibly worth checking if this behavior regressed with PlanResourceChange rollout.

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).