Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] Only allow authorized_paths to be updated in the options field of databricks_catalog #4517

Merged
merged 7 commits into from
Feb 25, 2025

Conversation

mgyucht
Copy link
Contributor

@mgyucht mgyucht commented Feb 21, 2025

Changes

There was a misunderstanding with the Unity Catalog API in connection with #4476. Apparently, only authorized_paths is eligible for update in the UC Catalog API. Otherwise, any other changes to options must trigger resource recreation.

This PR customizes the logic of databricks_catalog to do this. To be specific:

  • CustomizeDiff marks the options field as ForceNew if there is a key other than authorized_paths which was updated.
  • The Update command removes any options that aren't authorized_paths. This might be the case in the future if other options are added for HMS catalogs that can be set at create time but not updated.
  • The Create command removes any options from the UpdateCatalog RPC.

Tests

Added two integration tests to verify the behavior of databricks_catalog update.

  • Updated TestUcAccCatalogUpdate to verify that adding options with a key other than authorized_paths to a catalog defined without any options causes the catalog to be recreated.
  • Added TestUcAccCatalogHmsConnectionUpdate which tests the lifecycle of HMS catalogs, including updating the authorized_paths option, enforcing that the catalog resource is never deleted during the lifecycle of the test.

@mgyucht mgyucht requested review from a team as code owners February 21, 2025 10:09
@mgyucht mgyucht requested review from rauchy and removed request for a team February 21, 2025 10:09
@mgyucht mgyucht requested a review from alexott February 21, 2025 15:15
Copy link

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/terraform

Inputs:

  • PR number: 4517
  • Commit SHA: 73036ab122bbb107516343aa48f940f62f67abbd

Checks will be approved automatically on success.

@mgyucht mgyucht added this pull request to the merge queue Feb 25, 2025
Merged via the queue into main with commit 3900690 Feb 25, 2025
13 checks passed
@mgyucht mgyucht deleted the fix/handle-catalog-options branch February 25, 2025 19:48
tanmay-db added a commit that referenced this pull request Feb 26, 2025
### New Features and Improvements

 * Add support for cluster logs delivery to UC Volumes ([#4492](#4492)).
 * Expose more attributes for `databricks_connection` resource ([#4502](#4502)).
 * add `databricks_dashboards` data source ([#4521](#4521)).

### Bug Fixes

 * Delete `databricks_sql_endpoint` that failed to start ([#4520](#4520)).
 * Fix filling of missing attributes in `databricks_quality_monitor` ([#4519](#4519)).
 * Mark `default_catalog_name` attribute in `databricks_metastore_assignment` as deprecated ([#4522](#4522)).
 * Only allow `authorized_paths` to be updated in the `options` field of `databricks_catalog` ([#4517](#4517)).
 * Populate `partitions` when reading `databricks_sql_table` ([#4486](#4486)).
 * Remove configuration drift when configuring `databricks_connection` to built-in Hive Metastore ([#4505](#4505)).

### Documentation

 * Improve documentation for databricks_cluster and databricks_clusters ([#4506](#4506)).
 * Small fix to docs for data source `databricks_cluster` ([#4510](#4510)).
 * mark `databricks_volume` and `databricks_credential` as GA ([#4524](#4524)).

### Internal Changes

 * Upgrade `staticcheck` to compatible with Go 1.24 ([#4507](#4507)).
 * Use InputGitAuthor when tagging a commit ([#4501](#4501)).
 * fix `TestUcAccDataSourceShares` flakiness ([#4526](#4526)).
 * switch `TestMwsAccWorkspacesTokenUpdate` to BYOVPC ([#4525](#4525)).

### Dependency Updates

 * Bump github.com/hashicorp/terraform-plugin-framework from 1.14.0 to 1.14.1 ([#4515](#4515)).
 * Bump github.com/hashicorp/terraform-plugin-framework-validators from 0.16.0 to 0.17.0 ([#4511](#4511)).
 * Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.36.0 to 2.36.1 ([#4512](#4512)).
 * Bump to 1.24.0 ([#4508](#4508)).

### Exporter

 * Explicitly abort execution via `panic` if list of users can't be fetched ([#4500](#4500)).
 * Fix matching on `workspace_path` and refactoring ([#4504](#4504)).
deco-sdk-tagging bot added a commit that referenced this pull request Feb 26, 2025
## Release v1.67.0

### New Features and Improvements

 * Add support for cluster logs delivery to UC Volumes ([#4492](#4492)).
 * Expose more attributes for `databricks_connection` resource ([#4502](#4502)).
 * Add `databricks_dashboards` resource ([#4521](#4521))

### Bug Fixes

 * Populate `partitions` when reading `databricks_sql_table` ([#4486](#4486)).
 * Fix configuration drift when configuring `databricks_connection` to builtin Hive Metastore ([#4505](#4505)).
 * Only allow `authorized_paths` to be updated in the `options` field of `databricks_catalog` ([#4517](#4517)).
 * Mark `default_catalog_name` attribute in `databricks_metastore_assignment` as deprecated ([#4522](#4522))
 * Delete `databricks_sql_endpoint` that failed to start ([#4520](#4520))
 * Fix filling of missing attributes in `databricks_quality_monitor` ([#4519](#4519)).

### Documentation

 * Update `databricks_cluster` and `databricks_clusters` data source documentation ([#4506](#4506)).

### Exporter

 * Explicitly abort execution via `panic` if list of users can't be fetched ([#4500](#4500)).
 * Fix matching on `workspace_path` and refactoring ([#4504](#4504)).

### Internal Changes

 * Bump golang version to 1.24.0 ([#4508](#4508)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants