Use the Go SDK for `databricks_mws_workspaces` #4633

mgyucht · 2025-04-10T16:50:44Z

Changes

Currently, many resources in the mws package are implemented manually in the Terraform provider. Refactoring these to use the Go SDK reduces the amount of code that needs to be maintained in the TF provider and also ensures that new changes are released automatically. This PR should be nearly identical in terms of behavior, but there are some small variations because of subtle limitations in the Go SDK. For example, we may need to fetch the workspace an additional time using the Go SDK because WaitGetWorkspaceRunning does not return the fetched workspace in a terminal failure state.

Refactoring and Improvements:

Refactored databricks_mws_workspaces resource and datasource, as well as databricks_mws_ncc_binding, to use the Go SDK.

Caching Mechanisms:

Introduced caching for workspace clients in DatabricksClient to improve performance and reduce redundant API calls. Added cachedWorkspaceClient, cachedWorkspaceClients, and cachedAccountClient fields along with associated methods. [1] [2] [3]

Test Case Updates:

Updated TestDataSourceMwsWorkspaces to use the new Go SDK mocks for testing, replacing HTTP fixtures with mock client responses. [1] [2] [3]
Refactored TestMwsAccWorkspaces_TokenUpdate to include additional checks and new token update scenarios.
Modified TestResourceNccBindingCreate and TestResourceNccBindingUpdate to use mock clients for workspace updates. [1] [2]

Test Framework Enhancements:

Added support for multiple mock workspace clients in the ResourceFixture struct and validation logic. [1] [2] [3]

Tests

Tests run locally. Verifying with integration tests.

TODO

Onboard client.go changes to code generation.
Add back unit tests for token changes

alexott · 2025-04-10T17:08:17Z

common/client.go is marked as // Code generated from OpenAPI specs by Databricks SDK Generator. DO NOT EDIT. - so it will be overwritten?

mgyucht · 2025-04-10T19:35:00Z

@alexott good observation, that will have to move to universe.

Copilot

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

alexott

lgtm. just few comments

alexott · 2025-04-11T11:11:48Z

mws/resource_mws_workspaces.go

+// wait for DNS caches to refresh, as sometimes we cannot make
+// API calls to new workspaces immediately after it's created
+func verifyWorkspaceReachable(ctx context.Context, w *databricks.WorkspaceClient) error {
+	return retries.New[struct{}](retries.WithRetryFunc(func(err error) bool {


Will we need to close this old PR? #2957

Done, nice catch!

alexott · 2025-04-11T11:42:31Z

mws/resource_mws_workspaces.go

 				}
 			}
 			s["account_id"].Sensitive = true
+			common.CustomizeSchemaPath(s, "account_id").SetRequired()


Why can't we get it from account-level provider?

I'm just preserving the existing behavior & limiting the number of changes in this PR, but I think this can/should be removed in a follow-up PR.

I've created #4638 to track it

tanmay-db · 2025-04-11T11:42:21Z

common/client.go

 	cachedWorkspaceClient *databricks.WorkspaceClient
-	cachedAccountClient   *databricks.AccountClient
-	mu                    sync.Mutex
+	// cachedWorkspaceClients is a map of workspace clients for each workspace ID


Nice to have comments

tanmay-db · 2025-04-11T11:52:42Z

mws/resource_mws_workspaces.go

+				}
+				log.Printf("[ERROR] Deleting failed workspace: %s", err)
+				if derr := a.Workspaces.Delete(ctx, provisioning.DeleteWorkspaceRequest{WorkspaceId: wait.Response.WorkspaceId}); derr != nil {
+					return fmt.Errorf("workspace creation failed: %w; failed workspace cleanup failed: %w", err, derr)


failed workspace cleanup failed

should this be workspace cleanup failed?

This was the existing message, but I think the point here is that the workspace creation failed, resulting in a failed workspace, and the cleanup of that failed workspace also failed.

tanmay-db · 2025-04-11T12:12:00Z

mws/resource_mws_workspaces.go

-	}
-	return json.Marshal(workspaceCreationRequest)
+	provisioning.Workspace
+	CustomerManagedKeyID string `json:"customer_managed_key_id,omitempty"` // just for compatibility, will be removed


Regarding removal, should we mark it deprecated?

It is marked as deprecated, see line 201.

tanmay-db · 2025-04-11T12:12:53Z

mws/resource_mws_workspaces.go

 		}
+		return false
+	})).Wait(ctx, func(ctx context.Context) error {
+		ctx, cancel := context.WithTimeout(ctx, 10*time.Second)


Might be good to abstract out this timeout on top so it easy to read where the timeouts are defined when navigating code.

alexott · 2025-04-11T13:32:01Z

It also would be useful to update documentation - new fields and computed attributes

alexott · 2025-04-11T13:43:59Z

mws/resource_mws_workspaces.go

@@ -521,23 +200,15 @@ func ResourceMwsWorkspaces() common.Resource {
 			s["is_no_public_ip_enabled"].DiffSuppressFunc = func(k, old, new string, d *schema.ResourceData) bool {
 				return old != ""
 			}
+			s["is_no_public_ip_enabled"].Default = true


We should handle a case of is_no_public_ip_enabled = false. It's marked as omitempty, so it won't be set in the payload

github-actions · 2025-04-14T12:35:19Z

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/terraform

Inputs:

PR number: 4633
Commit SHA: e0d9a7444815c5753f78c04397af3daaf000aea9

Checks will be approved automatically on success.

## Release v1.73.0 ### New Features and Improvements * Add a new settings resource `databricks_disable_legacy_dbfs_setting` ([#4605](#4605)) * Support for `lifetime` attribute in `databricks_service_principal_secret` resource ([#4633](#4633)). * Mark `databricks_sql_dashaboard` as deprecated ([#4641](#4641)) ### Bug Fixes * Increase the default HTTP timeout to 65 seconds. ### Documentation * Document `user_api_scopes` in `databricks_app` resource and data sources ([#4614](#4614)) * Document new fields in `databricks_model_serving` resource ([#4615](#4615)) ### Exporter * Add export of `databricks_mws_network_connectivity_config` and `databricks_mws_ncc_private_endpoint_rule` ([#4613](#4613)) ### Internal Changes * Refactor `databricks_mws_workspaces` resource and datasource, as well as `databricks_mws_ncc_binding`, to use the Go SDK ([#4633](#4633)). * Add `TestMwsAccGcpWorkspaces` and `TestMwsAccGcpByovpcWorkspaces` to flaky test ([#4624](#4624)). * Bump Go SDK to latest release ([#4645](#4645))

…igration of resource to Go SDK (#4652) ## Changes  Reverts change in #4633 and set computed for workspace_status and vpc_status until they are fixed internally. ## Tests  N/A - [ ] `make test` run locally - [ ] relevant change in `docs/` folder - [ ] covered with integration tests in `internal/acceptance` - [ ] using Go SDK - [ ] using TF Plugin Framework

mgyucht added 3 commits April 10, 2025 11:49

Use Go SDK for databricks_mws_workspaces

8e74642

work

a753d9b

some work

ec78f7e

mgyucht requested review from a team as code owners April 10, 2025 16:50

mgyucht requested review from tanmay-db and removed request for a team April 10, 2025 16:50

mgyucht temporarily deployed to test-trigger-is April 10, 2025 16:50 — with GitHub Actions Inactive

mgyucht temporarily deployed to test-trigger-is April 10, 2025 16:52 — with GitHub Actions Inactive

work

bd96365

mgyucht temporarily deployed to test-trigger-is April 10, 2025 19:33 — with GitHub Actions Inactive

fmt

dd41a24

mgyucht temporarily deployed to test-trigger-is April 10, 2025 19:34 — with GitHub Actions Inactive

mgyucht temporarily deployed to test-trigger-is April 10, 2025 19:36 — with GitHub Actions Inactive

mgyucht added 2 commits April 10, 2025 22:12

work

7e6bdea

Merge branch 'main' into fix/use-go-sdk-for-databricks_mws_workspaces

15d1a03

mgyucht temporarily deployed to test-trigger-is April 10, 2025 20:18 — with GitHub Actions Inactive

mgyucht temporarily deployed to test-trigger-is April 10, 2025 20:20 — with GitHub Actions Inactive

fix test

40265e4

mgyucht temporarily deployed to test-trigger-is April 11, 2025 05:37 — with GitHub Actions Inactive

mgyucht temporarily deployed to test-trigger-is April 11, 2025 05:38 — with GitHub Actions Inactive

work

50bb279

mgyucht temporarily deployed to test-trigger-is April 11, 2025 07:56 — with GitHub Actions Inactive

mgyucht temporarily deployed to test-trigger-is April 11, 2025 07:57 — with GitHub Actions Inactive

mgyucht changed the title ~~[WIP] Use the Go SDK for databricks_mws_workspaces~~ Use the Go SDK for databricks_mws_workspaces Apr 11, 2025

mgyucht added 2 commits April 11, 2025 11:13

Fix schema diffs

78c0eb5

work

e7f2dfd

mgyucht temporarily deployed to test-trigger-is April 11, 2025 09:15 — with GitHub Actions Inactive

fix

a399087

alexott requested a review from Copilot April 11, 2025 10:54

Copilot AI reviewed Apr 11, 2025

View reviewed changes

empty commit

cebb9f3

mgyucht temporarily deployed to test-trigger-is April 11, 2025 12:00 — with GitHub Actions Inactive

mgyucht temporarily deployed to test-trigger-is April 11, 2025 12:02 — with GitHub Actions Inactive

alexott approved these changes Apr 11, 2025

View reviewed changes

tanmay-db approved these changes Apr 11, 2025

View reviewed changes

alexott reviewed Apr 11, 2025

View reviewed changes

This was referenced Apr 14, 2025

Support creation of simple workspace in Terraform #4636

Closed

[Internal] Remove the need for explicit account_id in databricks_mws_workspaces #4638

Open

some fixes

74685dc

mgyucht temporarily deployed to test-trigger-is April 14, 2025 09:55 — with GitHub Actions Inactive

mgyucht temporarily deployed to test-trigger-is April 14, 2025 09:56 — with GitHub Actions Inactive

mgyucht added 2 commits April 14, 2025 13:31

work

77621c6

Merge branch 'main' into fix/use-go-sdk-for-databricks_mws_workspaces

ae2f4fd

mgyucht temporarily deployed to test-trigger-is April 14, 2025 11:58 — with GitHub Actions Inactive

add tests

e0d9a74

mgyucht temporarily deployed to test-trigger-is April 14, 2025 12:34 — with GitHub Actions Inactive

mgyucht temporarily deployed to test-trigger-is April 14, 2025 12:36 — with GitHub Actions Inactive

mgyucht added this pull request to the merge queue Apr 14, 2025

Merged via the queue into main with commit f6cd4a5 Apr 14, 2025
12 checks passed

mgyucht deleted the fix/use-go-sdk-for-databricks_mws_workspaces branch April 14, 2025 13:23

tanmay-db mentioned this pull request Apr 16, 2025

[Internal] Bug fixed in databricks_mws_workspaces resource due to migration of resource to Go SDK #4652

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use the Go SDK for `databricks_mws_workspaces` #4633

Use the Go SDK for `databricks_mws_workspaces` #4633

mgyucht commented Apr 10, 2025 •

edited

Loading

alexott commented Apr 10, 2025

mgyucht commented Apr 10, 2025

Copilot AI left a comment

alexott left a comment

alexott Apr 11, 2025

mgyucht Apr 14, 2025

alexott Apr 11, 2025

mgyucht Apr 14, 2025

alexott Apr 14, 2025

tanmay-db Apr 11, 2025

tanmay-db Apr 11, 2025

mgyucht Apr 14, 2025

tanmay-db Apr 11, 2025

mgyucht Apr 14, 2025

tanmay-db Apr 11, 2025

alexott commented Apr 11, 2025

alexott Apr 11, 2025

github-actions bot commented Apr 14, 2025

Use the Go SDK for databricks_mws_workspaces #4633

Use the Go SDK for databricks_mws_workspaces #4633

Conversation

mgyucht commented Apr 10, 2025 • edited Loading

Changes

Refactoring and Improvements:

Caching Mechanisms:

Test Case Updates:

Test Framework Enhancements:

Tests

TODO

alexott commented Apr 10, 2025

mgyucht commented Apr 10, 2025

Copilot AI left a comment

Choose a reason for hiding this comment

alexott left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexott commented Apr 11, 2025

Choose a reason for hiding this comment

github-actions bot commented Apr 14, 2025

Use the Go SDK for `databricks_mws_workspaces` #4633

Use the Go SDK for `databricks_mws_workspaces` #4633

mgyucht commented Apr 10, 2025 •

edited

Loading