Skip to content

mongodbatlas_cloud_provider_snapshot_backup_policy restore_window_days (optional value) is being set even when omitted in resource config #290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bastiandg opened this issue Aug 12, 2020 · 6 comments
Labels

Comments

@bastiandg
Copy link

Terraform CLI and Terraform MongoDB Atlas Provider Version

$ terraform version
Terraform v0.12.29

Your version of Terraform is out of date! The latest version
is 0.13.0. You can update by downloading from https://www.terraform.io/downloads.html

Terraform Configuration File

resource "mongodbatlas_cloud_provider_snapshot_backup_policy" "backup" {
  count        = var.module_enabled ? 1 : 0
  project_id   = var.project_id
  cluster_name = mongodbatlas_cluster.cluster[0].name

  reference_hour_of_day    = var.backup_hour_of_day
  reference_minute_of_hour = var.backup_minute_of_hour

  policies {
    id = mongodbatlas_cluster.cluster[0].snapshot_backup_policy.0.policies.0.id

    policy_item {
      id                 = mongodbatlas_cluster.cluster[0].snapshot_backup_policy.0.policies.0.policy_item.0.id
      frequency_interval = 6
      frequency_type     = "hourly"
      retention_unit     = "days"
      retention_value    = 1
    }
    policy_item {
      id                 = mongodbatlas_cluster.cluster[0].snapshot_backup_policy.0.policies.0.policy_item.1.id
      frequency_interval = 1
      frequency_type     = "daily"
      retention_unit     = "days"
      retention_value    = 7
    }
    policy_item {
      id                 = mongodbatlas_cluster.cluster[0].snapshot_backup_policy.0.policies.0.policy_item.2.id
      frequency_interval = 6
      frequency_type     = "weekly"
      retention_unit     = "weeks"
      retention_value    = 4
    }
    policy_item {
      id                 = mongodbatlas_cluster.cluster[0].snapshot_backup_policy.0.policies.0.policy_item.3.id
      frequency_interval = 40
      frequency_type     = "monthly"
      retention_unit     = "months"
      retention_value    = 12
    }
  }
}

Steps to Reproduce

  1. omit restore_window_days in mongodbatlas_cloud_provider_snapshot_backup_policy configuration
  2. terraform init
  3. terraform apply

Expected Behavior

Successful apply

Actual Behavior

Error: error updating a Cloud Provider Snapshot Backup Policy: PATCH https://cloud.mongodb.com/api/atlas/v1.0/groups/e15fbeb63f3cfd2ead8422f3/clusters/clusterName/backup/schedule: 400 (request "Bad Request") The restore window days should be a positive number.

  on modules/mongodb-atlas-cluster/main.tf line 17, in resource "mongodbatlas_cloud_provider_snapshot_backup_policy" "backup":
  17: resource "mongodbatlas_cloud_provider_snapshot_backup_policy" "backup" {

Debug Output

module.mongodb_atlas_cluster.mongodbatlas_cloud_provider_snapshot_backup_policy.backup[0]: Creating...
2020/08/12 12:25:06 [DEBUG] module.mongodb_atlas_cluster.mongodbatlas_cloud_provider_snapshot_backup_policy.backup[0]: applying the planned Create change
2020/08/12 12:25:06 [TRACE] GRPCProvider: ApplyResourceChange
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: 2020/08/12 12:25:06 [DEBUG] MongoDB Atlas API Request Details:
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: ---[ REQUEST ]---------------------------------------
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: PATCH /api/atlas/v1.0/groups/1785443e7d5d6b30b285184d/clusters/clusterName/backup/schedule HTTP/1.1
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: Host: cloud.mongodb.com
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: User-Agent: terraform-provider-mongodbatlas
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: Content-Length: 659
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: Accept: application/json
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: Content-Type: application/json
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: Accept-Encoding: gzip
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: 
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: {
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:  "referenceHourOfDay": 18,
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:  "referenceMinuteOfHour": 14,
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:  "restoreWindowDays": 0,
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:  "updateSnapshots": false,
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:  "policies": [
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:   {
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:    "id": "5cf49b07f86027e294189354",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:    "policyItems": [
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:     {
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "id": "2a95498be628d34f02e4ced0",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "frequencyInterval": 6,
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "frequencyType": "hourly",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "retentionUnit": "days",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "retentionValue": 1
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:     },
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:     {
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "id": "5b2db29524e33b1bac84b737",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "frequencyInterval": 1,
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "frequencyType": "daily",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "retentionUnit": "days",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "retentionValue": 7
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:     },
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:     {
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "id": "5d4aa0cdc227802fe7b0d380",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "frequencyInterval": 6,
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "frequencyType": "weekly",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "retentionUnit": "weeks",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "retentionValue": 4
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:     },
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:     {
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "id": "7f229e4f08e1feda37b206a7",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "frequencyInterval": 40,
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "frequencyType": "monthly",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "retentionUnit": "months",
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:      "retentionValue": 12
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:     }
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:    ]
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:   }
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:  ]
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: }
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: 
2020-08-12T12:25:06.799Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: -----------------------------------------------------
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: 2020/08/12 12:25:07 [DEBUG] MongoDB Atlas API Response Details:
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: ---[ RESPONSE ]--------------------------------------
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: HTTP/2.0 400 Bad Request
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: Content-Length: 153
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: Content-Type: application/json
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: Date: Wed, 12 Aug 2020 12:25:07 GMT
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: Strict-Transport-Security: max-age=31536000
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: X-Frame-Options: DENY
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: X-Mongodb-Service-Version: gitHash=355529ebeb4f22c49593071be946bb2e7c33fe95; versionString=v20200721
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: 
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: {
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:  "detail": "The restore window days should be a positive number.",
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:  "error": 400,
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:  "errorCode": "INVALID_RESTORE_WINDOW",
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:  "parameters": [],
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4:  "reason": "Bad Request"
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: }
2020-08-12T12:25:07.408Z [DEBUG] plugin.terraform-provider-mongodbatlas_v0.6.3_x4: -----------------------------------------------------

Additional Context

As seen in the debug output the restoreWindowDays are sent to atlas even though it was not configured. Explicitly setting restore_window_days = null results in the same behavior.

@themantissa themantissa changed the title Creating mongodbatlas_cloud_provider_snapshot_backup_policy fails when optional parameter restore_window_days is omitted mongodbatlas_cloud_provider_snapshot_backup_policy restore_window_days (optional value) is being set even when omitted in resource config Aug 13, 2020
@themantissa
Copy link
Collaborator

Thank you @bastiandg, it is indeed an optional value and should be not sent. And what is being sent is out of a valid range.

@themantissa
Copy link
Collaborator

@bastiandg we've released a preview version that should allow you to test if this fixes the issue. If you can confirm we'd appreciate it.
https://github.com/mongodb/terraform-provider-mongodbatlas/releases/tag/0.6.4

@bastiandg
Copy link
Author

Hey @themantissa and @PacoDw thanks for implementing the fix.

The first thing I tried was to update an existing cluster with restore_window_days set to one. Sadly it doesn't recognize the change.

------------------------------------------------------------------------

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

The next thing I tried was to completely recreate the cluster backup policy. And the apply itself works without an issue, but oddly enough the restore_window_days is set to 7. When I then change it to another value the diff looks like this:

  # module.mongodb_atlas_cluster.mongodbatlas_cloud_provider_snapshot_backup_policy.backup[0] will be updated in-place
  ~ resource "mongodbatlas_cloud_provider_snapshot_backup_policy" "backup" {
        cluster_id               = "ba594c0b878790c0eceaafa7"
        cluster_name             = "dev"
        id                       = "2EhDr5C7ccmfh8yL2FvJydUb92PsbjY6G1VLMIuqSGIUbX3KSB8s6Nndcoj0YNWR"
        next_snapshot            = "2020-08-31T06:14:53Z"
        project_id               = "ba594c0b878790c0eceaafa7"
        reference_hour_of_day    = 18
        reference_minute_of_hour = 14
      ~ restore_window_days      = 7 -> 1
        update_snapshots         = false

My best guess is that the API from Atlas sets a default value without the provider picking that up properly.

@themantissa
Copy link
Collaborator

@bastiandg thank you, this is helpful. Before you did the first try did you refresh all state? I ask because I think part of the problem here may be how we had to implement the cloud backup policy. As you note on the second part of your comment above, Atlas sets a default value - not just for restore_window_days but as soon as Cloud Backup is enabled there is a policy/schedule created by default within Atlas. If you look at the underlying API documentation you can see that Cloud Backup Policy/Schedule has two methods - GET and PATCH. There is no POST as you create a Cloud Backup Policy when you enable backup and not DELETE because you remove it by turning backup off. So here's what we did in the provider to get around this:
-When you create or modify a cluster with cloud backup enabled we go ahead and import in the default policy.
-This allows one to either just leave the policy as is and not manage it or add a resource in Terraform to manage it further.
So it may be we aren't adequately explaining this in the documentation. Would you agree?

Most importantly thought, were you now able to at least set restore_window_days to 1? Did that second apply work?

@bastiandg
Copy link
Author

I didn't run an explicit terraform refresh before the apply. But I tried it just now and there is no change. So the odd behavior is:

  1. Set an explicit value for restore_window_days
  2. remove the value (essentially setting restore_window_days to null)
  3. terraform refresh
  4. terraform apply

-> No change is recognized.

I don't mind the slight inconsistency here, as it is a very rare edge case, that is also easy to resolve by re-adding the value.


Thanks for the explanation on the API in the background! Beside what you said, most of my confusion was my own fault. The documentation says:

restore_window_days - (Optional) Number of days back in time you can restore to with point-in-time accuracy. Must be a positive, non-zero integer.

What I read into it was: "If you don't set this value, 0 will be assumed".

So I think, yes for both cases above, it could be helpful to add information on which values are used when. Something like: In case it has been configured before, the configuration is left untouched. If a new backup policy is created, the default value of 7 is configured.


Changing the implicitly configured restore_window_days be explicitly changing it to a different value, worked without any issue.


Coming back to the original issue, it is fixed and this can be closed. Thanks a lot for your work and your explanations.

@themantissa
Copy link
Collaborator

@bastiandg thank you for the helpful response! Definitely realizing we have some room for improvement here in how we document backup, hopefully we can tackle soon but very glad the original issue is fixed! I'll close this issue but let us know if anything else comes up. Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants