Allow per-module terraform fixtures

[juliocc]

This PR introduces "terraform fixtures" which allow defining common terraform resources (or modules) that are deployed together with a given example. This allows:

• Describing all the required infrastructure for a given example
• Reusing common infrastructure used by more than one example (e.g. instance groups in the load balancing modules)
• Not polluting examples with repetitive boilerplate

[wiktorn]

I really love this one! One comment to consider - in README.md, maybe instead of putting fixtures at the end, let's keep them after examples and before TFDOC, and name the section something like Definitions of the modules used in the examples to make it easy discoverable.

[ludoo]

Julio, as we discussed I would much prefer to centralize fixtures instead of having a lot of them scattered around modules, which will become

• one more cognitive hurdle for any module changes
• a maintenance nightmare eventually as the provider, resources, and our understanding and use of fixtures evolve

So while this is nice, I am starting to strongly feel that we're focusing too much on the tooling and too little on the actual code. :) So unless we have a way of centralziing fixtures not at the module level, but at the repo level, this is a -1 from me. :)

BTW I also completely share your PoV in a different PR that with e2e, examples are starting to become cryptic. You and I both know the primary use of examples is to show module usage not to run e2e tests, and if usability starts becoming impaired we will need to start removing tooling. Can we find a way of reminding this to all our collaborators?

Edit: it's a fine balance, but now we're tipping too far on the tooling side and losing our initial focus.

@wiktorn for visibility

[wiktorn]

BTW I also completely share your PoV in a different PR that with e2e, examples are starting to become cryptic. You and I both know the primary use of examples is to show module usage not to run e2e tests, and if usability starts becoming impaired we will need to start removing tooling. Can we find a way of reminding this to all our collaborators?

I totally agree on the purpose of the examples, though I like to extend this definition, not to only show the module usage per se, but how it is used in the context of other modules.

Indeed, maybe now we are leaning too much over developer comfort over usability of examples. I think that by following a naming convention we can achieve both goals:

• Use defined module names in fixtures, that derive from module names in the repository. I.e define module "net-address" for modules/net-address. If we need more instances of modules, use -a, -b, etc. suffixes.

The result is, that it is easy to comprehend, which module output goes where on the other's module inputs.

The other thing we can do is to generate links in README.md to fixtures used, so it is easy to glue full example. WDYT?

[ludoo]

The other thing we can do is to generate links in README.md to fixtures used, so it is easy to glue full example. WDYT?

Yep, I mentioned it above and it seems a great idea and really easy to do via tfdoc. Links or even better expandables, I liked the one you and Julio showed a few days ago.

[juliocc]

First of all this is an initial draft to start the discussion before we invest more time in all the e2e. Nothing here is set in stone and we can change (or discard) things as needed.

Julio, as we discussed I would much prefer to centralize fixtures instead of having a lot of them scattered around modules, which will become

I rushed this PR a bit because of #1913 and didn't fully document the two options I implemented:

• Inlining the fixture directly in the README file using the tftest-fixture directive. This method keeps the code close the example but also becomes a bit unmaintainable IMO
• Dropping a terraform file fixtures in tests/fixtures and then reference it from examples. This method is a bit cleaner requires extra tooling to make fixtures easy to discover.

I started with the first method and then brainstorming with @ludoo we came up with the second idea. Both have pros and cons and I'm still not sure which one is better yet.

the primary use of examples is to show module usage

Agreed 100%. That's why I opened this PR so we can start discussing. My intention was to ping you both on Monday but you both were faster than me ;)

The other thing we can do is to generate links in README.md to fixtures used, so it is easy to glue full example. WDYT?

@ludoo and I discussed something similar. Let's first decide if we want fixtures at all and then we can figure out how to make them easy to discover.

[wiktorn]

@juliocc I allowed myself to move the test to global file, and change naming of the module.

Other notable changes:

• while preparing test directory, copy symlinks instead of copying of the contents (should reduce the IO during the tests, as we will not copy repo multiple times)
• create a fixtures folder with symlinks to root directory and variables.tf, to make the development of fixtures easier. My assumption is that here, it won't be a big problem for Windows-based developers

I'd love to get this merged soon, so others can base their work on fixtures.

EDIT: actually, creating symlinks instead of deep-copy makes tests run 20+% slower than with deep copy. Reverted those changes and left a comment for future self.