initial base module implementation for Dataplex AutoDQ #1543
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
modules/dataplex-autodq/variables.tf
Outdated
| */ | ||
|
|
||
| variable "data" { | ||
| description = "The data source for DataScan. The input variable 'data' should be an object containing a single key-value pair that can be one of: * `entity`: The Dataplex entity that represents the data source (e.g. BigQuery table) for DataScan, of the form: `projects/{project_number}/locations/{locationId}/lakes/{lakeId}/zones/{zoneId}/entities/{entityId}`. * `resource`: The service-qualified full resource name of the cloud resource for a DataScan job to scan against. The field could be: BigQuery table of type 'TABLE' for DataProfileScan/DataQualityScan format, e.g: `//bigquery.googleapis.com/projects/PROJECT_ID/datasets/DATASET_ID/tables/TABLE_ID`." |
There was a problem hiding this comment.
variable descriptions are usually one-liners :)
There was a problem hiding this comment.
haha ok :) will shorten
modules/dataplex-autodq/variables.tf
Outdated
| variable "execution_trigger" { | ||
| description = "The input variable 'execution_trigger' specifies when a scan should be triggered. If not specified, the default is `on_demand`, which means the scan will not run until the user calls `dataScans.run` API. Alternatively, you can set `execution_trigger` to `scheduled` scheduled to run periodically based on a cron schedule expression." | ||
| type = object({ | ||
| on_demand = optional(object({})) |
There was a problem hiding this comment.
this could also be done differently: the variable is execution_schedule with string type, and defaults to null which implies on demand.
There was a problem hiding this comment.
Yes this is a good point. I didn't realize the default behaviour for the official API was on_demand until after I've done everything else. I will refactor this according to your suggestion
There was a problem hiding this comment.
Left a couple minor comments, feel free to resolve them once done (and feel free to ignore the variable name/type change). When you are ready just merge.
Be careful in the future when pushing commits with the wrong email, had to rewrite commit history and force approve the CLA check. :)
modules/dataplex-autodq/README.md
Outdated
| @@ -0,0 +1,273 @@ | |||
| # Dataplex AutoDQ | |||
There was a problem hiding this comment.
Can you please link the module from the top-level repo README, and the modules folder README?
There was a problem hiding this comment.
And could you rename the module as cloud-dataplex-autodq, or rename the existing cloud-dataplex as dataplex?
There was a problem hiding this comment.
The official name is Dataplex not Cloud Dataplex. I will rename the other module accordingly.
|
Thanks Ludo for your feedback. I think I've addressed all of your feedback. Thank you for also fixing my CLA checks. I noticed I'm using the wrong email address in the first commit :/ sorry but I think I will need to do another git push --force. |
* Renamed output.tf in net-vlan-attachment
|
This is getting messy. I'm going to close this PR and reopen with a different branch. |
REST Resource: projects.locations.dataScans:
https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.dataScans#DataScan
Raw terraform resources:
https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/dataplex_datascan