List stacks in hidden directories

[moatorres]

Hello,

I'm wondering if there's any workaround to list stacks within hidden directories (e.g. .foo/stacks/...). I'm assuming that terramate follows unix-like behaviour, where:

https://en.wikipedia.org/wiki/Hidden_file_and_hidden_directory

Any file or folder that starts with a dot character (for example, /home/user/.config), commonly called a dot file or dotfile, is to be treated as hidden – that is, the ls command does not display them unless the -a or -A flags (ls -a or ls -A) are used

From the docs:

https://terramate.io/docs/cli/reference/configuration/#configuration-files

Terramate files can be found in any non-hidden directory of a Terramate project and all non-hidden files in a single directory will be handled as the concatenation of all of them in a single file, forming a single configuration.

At first I thought that terramate was ignoring directories listed on .gitignore or had its own .tmignore file where we could list directories/files that should be ignored and, more importantly, list directories that should not be ignored (e.g. !.foo/) during traversals.

As an example, in the following structure the stack-a will not be picked up. However, even though dist is being ignored by Git, stack-b will be picked up.

# .gitignore
dist
node_modules

.foo/
└── stacks/
    └── stack-a/
        ├── .terraform/
        ├── cdk.tf.json
        └── stack.tm.hcl <-- will not be picked up
dist/
└── stacks/
    └── stack-b/
        ├── .terraform/
        ├── cdk.tf.json
        └── stack.tm.hcl <-- will be picked up

Although the behaviour of not ignoring directories listed on .gitignore looks slightly unconventional to me, I'm mostly interested on listing stacks within hidden directories. I've tried to use the import block on a root-level teramate.tm.hcl file to force stack-a to be picked up, but unfortunately stacks can't be imported as source. 😞

I'd be happy to open a pull-request to add support for .tmignore files or a mechanism that would allow us to make terramate aware of stacks within hidden directories, if anyone finds it useful. Meanwhile, any known workaround would be very welcome.

Thanks!

[i4ki]

Hi @moatorres

We did plan to add .tmignore support but the details have not been designed yet.
About listing stacks inside dot directories, what is the use case for that?

Sorry for the late reply...

[moatorres]

No worries.

Main use case would be grouping CDKTF output files and Terramate stack definitions into a single folder that can be cached and recreated/rehydrated dynamically (alongside other non-versioned infrastructure assets).

On a side note, is there a way to remove unused stacks or simply delete them (not necessarily destroying) via Terramate Cloud?

I've got a bunch of test/junk stacks that have been destroyed and are still lying over there. I've attempted to sync in the hope that they'd be thrown away (since they've been destroyed) but to no avail.

[soerenmartius]

On a side note, is there a way to remove unused stacks or simply delete them (not necessarily destroying) via Terramate Cloud?

We are just adding a new feature allowing you to archive stacks. Meaning that stacks won't pop up in Terramate Cloud anymore unless data gets pushed again. Do you use Terramate Cloud? I'd love to learn more about your use-case!

[moatorres]

Thank you for reaching out, @i4ki and @soerenmartius!

We are just adding a new feature allowing you to archive stacks.

Cool! 🎉🚀

@soerenmartius, yes! I've been using it mainly for test purposes so far. I really like the ability to navigate through resources and inspect drifts visually. I haven't managed to integrate it with GitHub (yet), but I intend to.

I've been researching/trying different tools in the IaC problem-space and Terramate just clicked with me. Although there are some redundancies when using CDKTF and NX (e.g. change detection is being handled in another layer), Terramate has proven really useful when we want to interact with generated stacks.

Being able to run commands on multiple stacks simultaneously is quite handy since cdktf CLI can only execute commands over one stack at a time. One possible workaround is creating targets on each project (e.g. diff) and executing them via nx.

(I've just joined Terramate's Discord server, by the way)

[soerenmartius]

Awesome thanks for the feedback @moatorres! Feel free to get in touch with us via email or discord in case we can help you further also!