Configurable recursive inspection

[wata727]

Introduction

When we introduced the --recursive option in #1622, I thought we would soon need a way to control behavior on a per-workspace basis, but we omitted it for simplicity. Specifically, I'm assuming something like the following:

• Enable different rules for root and child modules
• Enable different plugins for each service provider
• Apply different configs to each team that owns a workspace

Currently the only way to do this is to put .tflint.hcl in each workspace directory. Recursive inspection checks after --chdiring to each workspace directory, so if there is a configuration file in each directory, it will be loaded.

However, this introduces a new issue of configuration file redundancy. Placing almost the same .tflint.hcl in every workspace directory is a pain.

Proposal

There are several solutions to this problem.

1. Configuration cascading

See also #1536
See also https://eslint.org/docs/latest/use/configure/configuration-files#cascading-and-hierarchy

Reduce redundancy by walking up directories and merging configurations. This allows you to simply place the minimum required configuration in each workspace directory.

The challenge with this approach is that it becomes difficult to understand the final configuration that will be applied. If a configuration file located in a directory above that you are not aware of unintentionally affects the inspection, you may be confused as to why it is not working as intended. ESLint introduces the root key as a solution to this issue. See https://eslint.org/blog/2022/08/new-config-system-part-1/

Another challenge is that whether this solution works effectively depends on your directory structure. For example, if you have a directory structure like the one below and enable different rules for root modules and child modules, it will be difficult to effectively use cascading.

├── aws
│   ├── modules
│   ├── product1
│   │   ├── dev
│   │   ├── modules  <-- child module rules
│   │   ├── prod     <-- root module rules
│   │   ├── qa
│   │   └── staging
│   ├── product2
│   │   ├── dev
│   │   ├── modules  <-- child module rules
│   │   ├── prod     <-- root module rules
│   │   ├── qa
│   │   └── staging
│   └── product3
├── azure
└── google

2. Recursive inspection config

See also #1355 (comment)

Extend .tflint.hcl to add recursion inspection config like the following:

modules = [
  {
    path   = "aws/**"
    config = "tflint/aws/root_module.hcl"
  },
  {
    path   = "google/**"
    config = "tflint/google/root_module.hcl"
  }
  {
    path   = "aws/modules/**"
    config = "tflint/aws/child_module.hcl"
  }
  {
    path   = "google/modules/**"
    config = "tflint/google/child_module.hcl"
  }
  {
    path   = "aws/legacy_product/production"
    config = "tflint/legacy.hcl"
  }
]

This is inspired by the "flat config" idea mentioned in the ESLint blog. See https://eslint.org/blog/2022/08/new-config-system-part-2/
This solution eliminates the implicitness of configuration cascading and provides flexibility over directory structures.

The challenge with this approach is that the configuration files are located away from workspace directories. In a large repository, top-level configuration files become bloated and pulled out of workspace contexts. This can result in many orphaned configuration files that cannot be managed.

Another challenge is that you won't be able to perform the same inspection any other way than by recursive inspection. During development, you'll probably want to inspect your workspace with `tflint --chdir', but applying the same configuration requires some tweaking.

3. Others

An idea that has not yet been discovered.

Side notes

I intended to have a discussion of this issue in #1536, but I noticed many users are not aware that a workaround for configurable recursion inspection is by placing a .tflint.hcl in each workspace directory. This issue is being created to make such users aware of this issue.

Please note that not all problems are solved with one solution. For example, #1920 is a request for customization of recursive inspection, but since the purpose is to filter directories to be inspected according to file changes, it is probably not appropriate to solve it by configuration file. We welcome use cases for this issue, but please be sure to share the background you want to configure.

As another topic, the concept of Stacks introduced in Terraform may be worth noting. This introduces a configuration file *.tfstack.hcl for multiple Terraform root modules.

References

• Add --recursive option #1622
• Git config like .tflint.hcl files combination #1536
• https://eslint.org/docs/latest/use/configure/configuration-files#cascading-and-hierarchy
• https://eslint.org/blog/2022/08/new-config-system-part-1/
• Recursive inspection (again) #1355 (comment)
• https://eslint.org/blog/2022/08/new-config-system-part-2/