Add cloud dataplex module

[prabhaarya]

Terraform code for creating Dataplex lake, zone and asset in GCP project.

[ludoo]

Thanks for this, can you make sure tests pass?

[lcaggio]

Thanks for working on this! Dataplex is a great addition to the repo. As mentioned in pur offline sync, I would shape this module similar to the BQ-dataset one:

• name: dataplex-lake
• add IAM support
• support multiple zone definition (unspecified, raw, curated)
• support multiple asset definition and mapping to the zone
• support different type of assets
• support multi-regions

If you do not implement all in this PR, I would make a TODO section in the README.

[prabhaarya]

Thanks for working on this! Dataplex is a great addition to the repo. As mentioned in pur offline sync, I would shape this module similar to the BQ-dataset one:

• name: dataplex-lake
• add IAM support
• support multiple zone definition (unspecified, raw, curated)
• support multiple asset definition and mapping to the zone
• support different type of assets
• support multi-regions

If you do not implement all in this PR, I would make a TODO section in the README.

Thanks for nice feedback and help. Sure, let me add TODO section in README :)

[juliocc]

Hi @prabhaarya. Can you switch you test to use an example in the README of your module?. It's much easier.

The details are explained in the contributing guide.

[lcaggio]

Hi @prabhaarya, I am under the impression that supporting multiple zone should be added in this PR.

How would you use this module in your terraform deployment?

My understanding is that a typical deployment would be one lake with 2 zones (RAW and CURATED) with attached assets. As of now, this module would create one lake with one zone that could be RAW or CURATED but not both.

[prabhaarya]

Hi @prabhaarya, I am under the impression that supporting multiple zone should be added in this PR.

How would you use this module in your terraform deployment?

My understanding is that a typical deployment would be one lake with 2 zones (RAW and CURATED) with attached assets. As of now, this module would create one lake with one zone that could be RAW or CURATED but not both.

I agree with you, Lorenzo. It currently supports either zone type but not both.

IMO, having both zones when creating Dataplex lakes is not a must; it can be an enhanced module feature.

[lcaggio]

Thanks! some comments.

[lcaggio]

Some minor changes and we are good to go with V1!

[lcaggio]

Thanks for the update, thanks for supporting already multiple zone types in this version.

[ludoo]

@prabhaarya @lcaggio can you open a separate PR to link the module from the top-level and section-level READMEs?