google_dataflow_flex_template_job doesn't persist all arguments

[elitongadotti]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to a user, that user is claiming responsibility for the issue.
• Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.10.5

• provider registry.terraform.io/hashicorp/google v6.19.0
• provider registry.terraform.io/hashicorp/google-beta v6.19.0

Affected Resource(s)

google_dataflow_flex_template_job

Terraform Configuration

resource "google_dataflow_flex_template_job" "my_job" {
  provider = google-beta
  name     = "test-job-123"
  region   = var.region
  project  = var.project

  container_spec_gcs_path = local.template_path

  subnetwork = local.network_path

  ip_configuration = "WORKER_IP_PRIVATE"
  on_delete = "drain"
  enable_streaming_engine = "true"

  additional_experiments = [
    "enable_data_sampling",
    "use_runner_v2"
  ]
  max_workers = 3

  parameters = {
    input_subscription = "projects/${var.project}/subscriptions/test-123123"
  }
}

Debug Output

No response

Expected Behavior

No changes. Your infrastructure matches the configuration. is expected when nothing is changed in the resource at all.

Actual Behavior

Arguments like max_workers, machine_type and sdk_container_image (see the docs here) are not considered when plan runs for a second time. The plan command always ignores those arguments and so the resource is always planned to be changed over and over again.

Steps to reproduce

1. terraform plan setting those arguments
2. terraform plan for a second/third/whatever time, with the same configuration. No changes at all.

Important Factoids

No response

References

No response

[KubiNewb]

I have the same problem. I am not changing the configuration, yet Terraform still shows some changes. The expected result should be 0 to create, 0 to change, and 0 to destroy. Upgrading the provider for Google Beta also did not help resolve the issue.

However, I noticed one thing regarding the templates that are used. Specifically, the predefined templates provided by Google work, and the error does not appear there—unless that was just a coincidence. But the templates that we define ourselves contain this bug.

Has anyone already solved this issue?

Below I am attaching a screenshot of the problem:

[FilipKubawskiOkta]

Hi @shuyama1 , can we consider making these fields Computed: false? I wasn't able to reproduce this on any Google provided templates same as above

Specifically, the predefined templates provided by Google work

Which raises the question, what's the difference in the API response returning these values between the different templates? Another solution would be to add some conditions into the read logic based on that diff.

@elitongadotti if you could provide a plan with TF_LOG=DEBUG we should be able to narrow down the problem.

[elitongadotti]

Hi @shuyama1 , can we consider making these fields Computed: false? I wasn't able to reproduce this on any Google provided templates same as above

Specifically, the predefined templates provided by Google work

Which raises the question, what's the difference in the API response returning these values between the different templates? Another solution would be to add some conditions into the read logic based on that diff.

@elitongadotti if you could provide a plan with TF_LOG=DEBUG we should be able to narrow down the problem.

Hi.

I've seen that a PR was already opened towards this issue, so I assume no logs are needed 👍🏼

[FilipKubawskiOkta]

I've seen that a PR was already opened towards this issue, so I assume no logs are needed 👍🏼

yup, i was able to reproduce this behaviour and compared the 2 API payloads. Added a fallback data source for these fields because the API responses are different for some reason.