Add ability to refer to other project service accounts in Project Factory

[wiktorn]

• Allow interpolation of service accounts created for other projects, by specifying:

  iam:
    "roles/....":
    - ${project}/${service_account}

• Allow lookup of project service accounts for shared_vpc_service_config.network_users (also other projects, just to keep the code the same, though it doesn't make much sense)
• Allow lookup of project and other projects service accounts for bucket IAM
• Add support for extra_dirs for tfvars tests
• Add tfvars tests for project_factory

---

Checklist

I applicable, I acknowledge that I have:

• Read the contributing guide
• Ran terraform fmt on all modified files
• Regenerated the relevant README.md files using tools/tfdoc.py
• Made sure all relevant tests pass

Breaking Changes

`modules/project`: move input variable `service_agents_config.services_enabled` to `project_reuse.project_attributes.services_enabled`

[ludoo]

I like this approach, it's lean and tries incremental matches. I would put own service accounts first, then other projects, then context.

I don't really grasp your TODO though, can you explain it?

[wiktorn]

The idea is to feed all "local" context into local.context variable:

diff --git a/modules/project-factory/main.tf b/modules/project-factory/main.tf
index 736bfb093..1e5416684 100644
--- a/modules/project-factory/main.tf
+++ b/modules/project-factory/main.tf
@@ -20,21 +20,25 @@ locals {
   context = {
     folder_ids = merge(
       var.factories_config.context.folder_ids,
       local.hierarchy
     )
     iam_principals = merge(
       var.factories_config.context.iam_principals,
       {
         for k, v in module.automation-service-accounts :
         k => v.iam_email
-      }
+      },
+      {
+        for k, v in module.service-accounts :
+        k => v.iam_email
+      },
     )
   }
 }
@@ -112,29 +116,23 @@ module "projects-iam" {
   iam = {
     for k, v in lookup(each.value, "iam", {}) : k => [
       for vv in v : try(
-        # other projects service accounts
-        module.service-accounts[vv].iam_email,
-        # other automation service account
-        local.context.iam_principals[vv],
-        # project service accounts
-        module.service-accounts["${each.key}/${vv}"].iam_email,
-        # automation service account
+        # project service account
         local.context.iam_principals["${each.key}/${vv}"],
-        # other context
+        # other projects and external context
         local.context.iam_principals[vv],
         # passthrough
         vv
       )
     ]
   }

And replicate above change in other 2 iam_*. This will simplify the code and I fail to find a case, where it will give different result.

Question is what to do about buckets:

module "buckets" {
...
  iam = {
    for k, v in each.value.iam : k => [
      for vv in v : try(
        # other projects service accounts
        module.service-accounts[vv].iam_email,
        module.service-accounts["${each.value.project}/${vv}"].iam_email,
        var.factories_config.context.iam_principals[vv],
        vv
      )
    ]
  }

Should this use exactly the same construct, as we use for projects-iam, I do not follow why IaC service accounts are excluded here.

[ludoo]

The idea is to feed all "local" context into local.context variable:

Ok I get it and it makes sense. :)

Should this use exactly the same construct, as we use for projects-iam, I do not follow why IaC service accounts are excluded here.

There's a very delicate balance with dependency cycles. If it's excluded it's usually because I ran into a cycle. I broke up the project module to avoid some, but at some point we need to draw a line. :)

Try, and see what you get. We need a "golden set" of patterns in YAML to test against, I don't think we have one for the new approach of using in-project buckets and service accounts. I would not merge without testing though, I can help you compile a set of YAMLs tomorrow if you want.

[ludoo]

🎉

[juliocc]

Nice even number