How to run Atlantis plan in directories modified by pre workflow run?

[mimozell]

Hi,

I have a cdktf project written in TypeScript. The way things work is that infrastructure code is written in TS and then with the cdktf synth command you synthesize the code into terraform json code (the generated files will be cdk.tf.json). In order to plan, Atlantis looks only at the generated files, not the ones written in TS. The issue that I am facing is that I would like to skip the need to run that command and have Atlantis run it instead. So for a developer the flow should be the following:

1. Write infrastructure in TS
2. Create a new PR with the changes.

Then, Atlantis should run cdktf synth to generate the files and run terraform plan on them.

The issue is that: "The output of cdktf synth has to be committed to the pull request". This means that even my pre-workflow hook that looks like this:

  pre_workflow_hooks:
    - run: npm i && cdktf get && cdktf synth

will only run after someone already ran cdktf synth and pushed the changes to the PR.
Is there any way to solve this with Atlantis? Some way to run scripts when a PR is created, independently of the files in it?

Kinda thinking of using pre-commit hooks for this, but it's not ideal because there are cases where I would want people not belonging to the team to make PRs in the repo and they might not have the stack required for cdktf installed locally.

[nitrocode]

I was trying to solve this myself and I was able to get cdk synth in a pre workflow to generate the fjles but I could not get atlantis to pick up the changes unless I committed them. I didnt have the time to debug it.

Have you tried it with the pre workflow? Did you get the same results? If so, please troubleshoot further so we can get a fix in place and then we can make committing the json hcl optional.

[nitrocode]

@mimozell do you mind if we thread our comments here to encapsulate the discussions?

[mimozell]

How did you get the pre_workflow hook to run? In my case, without pushing the cdk.tf.json files Atlantis doesn't run anything. Once they are pushed, then the pre_workflow_hooks is triggered.

I can debug further as long as I can get the hook to run without having pushed the json files 😄

[nitrocode]

I used a custom image and ran cdktf synth in the pre_workflow_hooks. Running the pre_workflow_hook was easy but atlantis did not auto discover unless the files were committed.

[mimozell]

I got the synth to run by allowing planning on .ts files, but got the following error:

╷running "/usr/local/bin/terraform plan -input=false -refresh -out \"/home/atlantis/.atlantis/repos/<some repo details>/default/stacks/default.tfplan\"" in "/home/atlantis/.atlantis/repos/<some repo details>/default/stacks": exit status 1

│ Error: No configuration files
│ 
│ Plan requires configuration to be present. Planning without a configuration
│ would mark everything for destruction, which is normally not what is
│ desired. If you would like to destroy everything, run plan with the
│ -destroy option. Otherwise, create a Terraform configuration file (.tf
│ file) and try again.

Looks like either the synth didn't go correctly or it is not seeing the result of it. Will try to investigate further.

[mimozell]

Not using workspaces and no atlantis.yml file.

[mimozell]

In the logs I see the following messages:

{
    "level": "debug",
    "ts": "2022-11-21T08:33:47.615Z",
    "caller": "events/pre_workflow_hooks_command_runner.go:48",
    "msg": "pre-hooks configured, running...",
    "json": {
        "repo": "<name>",
        "pull": "38"
    }
}

{
    "level": "debug",
    "ts": "2022-11-21T08:33:47.615Z",
    "caller": "events/pre_workflow_hooks_command_runner.go:54",
    "msg": "got workspace lock",
    "json": {
        "repo": "<repo name>",
        "pull": "38"
    }
}

but there is no output of logs from the commands in the hook. And after that the plan fails with the message above.

When manually doing cdktf synth and pushing the change Atlantis always plans fine, but for some reason when it is doing that itself in the hooks it is not seeing the result and looking in the /default/stacks/ directory instead of the directories that contain the cdk.tf.json files.

Would there be some easy way to run this scenario locally?

[mimozell]

Okay, so from some testing today I see that Atlantis is trying to plan in the directories that have changed files. In this case, I have a .ts file in a directory that has no terraform configuration and that's why we see the error above.

Atlantis is not looking at the diff that is generated after it runs cdktf synth. I'm guessing that is because the diff isn't in the PR? So even if it is locally (in the Atlantis container), it doesn't matter as it's not in the PR. Is there some way to modify the behaviour without Atlantis committing the change produced by running cdktf synth back into the PR?

[nitrocode]

See the issue #2693 I wrote up from this discussion.

It's because the way our VCS framework is setup, atlantis is hitting the VCS API to figure out what has changed instead of looking at the changed files between the base branch and the PR branch. PRs are welcome.

[mimozell]

Thanks! I was thinking that even when we put this fix in place, there will still be errors with planning because the trigger for the plan is changes to .ts files and then Atlantis also tries to plan in the directory containing the changed file and it fails as there will never be any terraform configuration in dirs other then the cdktf.out ones.

[nitrocode]

Currently there is no way to modify the directories in a pre workflow run UNLESS a file in those directories has already been modified in the PR. This is because atlantis reads the VCS's API to retrieve the list of files modified instead of using a native local git command.

I've documented this limitation in #2693 for now.