Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Terragrunt run-all evaluates locals of not included modules when --terragrunt-strict-include is used #2860

Closed
MichalKoziorowski-TomTom opened this issue Dec 19, 2023 · 8 comments · May be fixed by #3012
Closed

Terragrunt run-all evaluates locals of not included modules when --terragrunt-strict-include is used #2860

MichalKoziorowski-TomTom opened this issue Dec 19, 2023 · 8 comments · May be fixed by #3012
Assignees
@denis256
Labels
bug Something isn't working stale Stale

Comments

@MichalKoziorowski-TomTom
Copy link

MichalKoziorowski-TomTom commented Dec 19, 2023 •
edited by denis256
Loading

Terragrunt run-all evaluates locals of not included modules when --terragrunt-strict-include is used
@MichalKoziorowski-TomTom MichalKoziorowski-TomTom added the bug Something isn't working label Dec 19, 2023
@MichalKoziorowski-TomTom MichalKoziorowski-TomTom changed the title Terragrunt evaluates locals of not included modules when --terragrunt-strict-include is used Terragrunt run-all evaluates locals of not included modules when --terragrunt-strict-include is used Dec 19, 2023
@denis256
Copy link
Member

denis256 commented Jan 2, 2024

For reference:
https://github.com/gruntwork-io/terragrunt/blob/v0.54.12/configstack/module.go#L59
https://github.com/denis256/terragrunt-tests/tree/master/issue-2860

@egorksv
Copy link

egorksv commented Mar 11, 2024

Any news on this? It breaks the entire deployment as in our case staging has no key to production sops file and vice versa

@denis256 denis256 moved this to To do in Terragrunt Roadmap Mar 19, 2024
@denis256 denis256 mentioned this issue Mar 19, 2024
Draft
4 tasks
@denis256 denis256 added the terragrunt label Mar 19, 2024 — with Linear
@MichalKoziorowski-TomTom
Copy link
Author

Hi. Are there any news with PR for this bug? I have really ugly workarounds in my TG modules because of this (filling not needed env variables with fake values in unrelated modules).

@egorksv
Copy link

egorksv commented Apr 10, 2024 •
edited
Loading

https://gist.github.com/egorksv/65ab1a1ab8bc5cfdb7b3a6330e90b5af

Workaround script to force include specific modules

group/*.hcl are empty hcl files that are used to logically group modules

@MichalKoziorowski-TomTom
Copy link
Author

MichalKoziorowski-TomTom commented Apr 10, 2024 •
edited
Loading

I just started to see "new" version of this bug.

  • If not included module evaluates locals, deployment fails.
  • If not included module evaluates inputs, I see errors but deployment continues.

In effect, I need to to provide to terragrunt env variables of modules which I don't want to deploy.

I wanted only my "datacollector" application to be redeployed, and I've got errors about cluster that was not expected to be touched:

['terragrunt', 'run-all', 'apply', '--terragrunt-working-dir=/infrastructure/kubernetes/terragrunt', '--terragrunt-strict-include', '--terragrunt-include-module-prefix', '--terragrunt-log-level=info', '--terragrunt-non-interactive', '--terragrunt-include-dir=applications/datacollector']
time=2024-04-10T13:44:17Z level=info msg=The stack at /infrastructure/kubernetes/terragrunt will be processed in the following order for command apply:
Group 1
- Module /infrastructure/kubernetes/terragrunt/applications/datacollector


time=2024-04-10T13:44:23Z level=error msg=Error: Error in function call

 prefix=[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
time=2024-04-10T13:45:21Z level=error msg=  on /infrastructure/kubernetes/terragrunt/cluster/azure_tags/terragrunt.hcl line 16:
 prefix=[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
time=2024-04-10T13:45:21Z level=error msg=  16:   deployer            = "${get_env("DEPLOYER")}"
 prefix=[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
time=2024-04-10T13:45:21Z level=error msg=
 prefix=[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
time=2024-04-10T13:45:21Z level=error msg=Call to function "get_env" failed: EnvVarNotFoundError: Required environment
variable DEPLOYER - not found.

 prefix=[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
time=2024-04-10T13:45:21Z level=warning msg=Error reading partial config for dependency azure_tags: /infrastructure/kubernetes/terragrunt/cluster/azure_tags/terragrunt.hcl:16,28-36: Error in function call; Call to function "get_env" failed: EnvVarNotFoundError: Required environment variable DEPLOYER - not found. prefix=[/infrastructure/kubernetes/terragrunt/cluster/dns_zone] 
time=2024-04-10T13:45:59Z level=info msg=Downloading Terraform configurations from file:///infrastructure/kubernetes/terraform into /infrastructure/kubernetes/terragrunt/applications/datacollector/.terragrunt-cache/PrxYmuk-qgovdK4RDn3zHprxfsU/MuvRh0xLYIb1VzakUyVr62McSpg prefix=[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
[/infrastructure/kubernetes/terragrunt/applications/datacollector] Initializing the backend...
[/infrastructure/kubernetes/terragrunt/applications/datacollector] 
[/infrastructure/kubernetes/terragrunt/applications/datacollector] Successfully configured the backend "azurerm"! Terraform will automatically
[/infrastructure/kubernetes/terragrunt/applications/datacollector] use this backend unless the backend configuration changes.
[/infrastructure/kubernetes/terragrunt/applications/datacollector] Initializing modules...
[/infrastructure/kubernetes/terragrunt/applications/datacollector] - azure_tags in ../../modules/azure_tags
[/infrastructure/kubernetes/terragrunt/applications/datacollector]

TG version: v0.55.19

@amirasyraf
Copy link

amirasyraf commented Apr 22, 2024 •
edited
Loading

Hi, regarding the PR #3012

Just a question, would it also address dependency evaluation of the terragrunt stack, and not just locals evaluation?

For my use case, I append a .skip to the path name to environments or stacks we intend to skip. e.g. rename uat/ to uat.skip/. I use --terragrunt-include-dir along with --terragrunt-strict-include to specify exactly which stacks to execute. My pipeline checks for this path and handles it automatically.

However, since Terragrunt currently does not fully respect the --terragrunt-include-dir and --terragrunt-strict-include, when another stack which uat.skip/ depends on gets executed (e.g. networkhub), it will fail because the stacks inside uat.skip/ are still trying to evaluate the original path uat/.

e.g. uat.skip/tgw depends on networkhub/tgw. So when running terragrunt run-all plan --terragrunt-strict-include --terragrunt-include-dir networkhub/tgw, it still evaluates uat.skip/tgw - in my case causing a failure as the stacks under uat.skip are looking for the original uat path.

It really should fully respect the strict flags and not even evaluate it in the first place

@MichalKoziorowski-TomTom
Copy link
Author

Hi @denis256.
I see that there is a draft PR attached to this issue. Do you maybe plan to have this finished?

@denis256 denis256 self-assigned this Jul 10, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 9, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for raising this issue.

@github-actions github-actions bot added the stale Stale label Oct 9, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@denis256 denis256

Labels
bug Something isn't working stale Stale
Projects
Terragrunt Roadmap
Status: To do
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add skip evaluation of locals for strict included modules gruntwork-io/terragrunt
5 participants
@ZachGoldberg @denis256 @amirasyraf @egorksv @MichalKoziorowski-TomTom