Reinitialize if module version requirement changed

[jgrumboe]

What problem are you facing?

I wanted to switch from gruntwork/terraform-validate pre-commit hook to pre-commit-terraform/terraform_validate. So far so good, but when I changed a module version in my code I got the following error:

Terraform validate.......................................................Failed
- hook id: terraform_validate
- exit code: 1

Validation failed: XXXXXXX/production

Error: Module version requirements have changed

  on main.tf line 17, in module "base":
  17:   source                            = "app.terraform.io/XXXXXX/base/aws"

The version requirements have changed since this module was installed and the
installed version (2.0.10-pre) is no longer acceptable. Run "terraform init"
to install all modules required by this configuration.

I never got such an error with the gruntwork hook and I figured out that the gruntwork hook just always run terraform init -backend=false.
The terraform_validate hook only runs terraform init if there's no .terraform directory --> https://github.com/antonbabenko/pre-commit-terraform/blob/master/terraform_validate.sh#L93

How could pre-commit-terraform help solve your problem?

Honestly, I'm a fan of not running terraform init all the time, as it makes pre-commit checks faster overall.
On the other side I don't like to think about manually deleting .terraform and terraform.lock.hcl whenever I want to change to a different module version.

It would be great of terraform_validate would be aware of module version changes and rerun terraform init if needed.

[antonbabenko]

Hi Johannes,

Terraform does not give us a way to know what exactly is wrong: Provider or module version changed? Not initialized modules? The wrong version of Terraform? Or something else?

The easiest way for users right now is to remove .terraform folders before running hooks.

In the long run, we may implement force-init, as described here - #224

Closing for now.

[antonbabenko]

https://github.com/antonbabenko/pre-commit-terraform#terraform_validate - there is a function to remove .terraform folders.

[jgrumboe]

Hi Anton,

Thanks for the quick feedback and the link (I saw this one). I agree that Terraform gives you no hint currently.
Maybe it's worth thinking about a retry-mechanism after terraform validate failed, which runs the remove-folder (including lock-file) command, terraform init and retries terraform validate - just once. Would still be faster than force-init all the time.

[antonbabenko]

As a user, I agree. But as a developer, it is still tricky to try to guess if the error is retriable or a hard failure. Doing regexp matches on error messages to decide if it should retry can be rather error-prone (Terraform can change error messages any time).

[jgrumboe]

Yes, you're totally right that regexp matches on error messages aren't a good idea.
Could we be more confident in looking at the git-changes of the commits? For example every version = x.y.z change in the current staged changes would result in running terraform init.
What do you think?

[antonbabenko]

pre-commit does not know what is "git changes". It is often executed without relation to history, not right before doing a commit (despite the name - pre + commit). So, this won't work.

[jgrumboe]

Ok, didn't knew that, I was thinking of running git diff --cached with some grep inside the terraform_validate.sh. But never mind.

Last one (and then I'm really quiet, I promise 🤞 ):
I just figured out that the terraform providers (https://www.terraform.io/docs/cli/commands/providers.html) command could be a good indicator, if we need to run terraform init.
It checks if the already downloaded providers and modules are still aligned to the code - if not there's a simple error message and return code != 0.

So, line 93 of terraform_validate.sh could be changed to

if ! terraform providers >/dev/null 2>&1; then

and would always trigger a terraform init if providers or modules are not initialized correctly of whatever reason.
🤐

[antonbabenko]

Thanks for the suggestion!

If terraform providers will be able to catch such errors then we can do it as you describe.

I have just updated #224 issue.

I am not sure when or who will be able to work on it. Feel free to contribute if you have time.