google_project resource in DELETE_REQUESTED state causes terraform plan error

[ewbankkit]

I successfully create a project via Terraform:

resource "google_project" "service" {
  project_id      = "test-service3-id"
  name            = "test-service3-name"
  org_id          = ...
  billing_account = ...
}

and then shut down the project in the GCP console. The project transitions to DELETE_REQUESTED state.
I then run terraform plan and get an error:

* google_project.service: google_project.service: Error reading billing account for project "projects/test-service3-id": googleapi: Error 404: Requested entity was not found., notFound

Running with logging on:

2017-09-22T17:56:04.319-0400 [DEBUG] plugin.terraform-provider-google: ---[ REQUEST ]---------------------------------------
2017-09-22T17:56:04.319-0400 [DEBUG] plugin.terraform-provider-google: GET /v1/projects/test-service3-id?alt=json HTTP/1.1
2017-09-22T17:56:04.319-0400 [DEBUG] plugin.terraform-provider-google: Host: cloudresourcemanager.googleapis.com
2017-09-22T17:56:04.319-0400 [DEBUG] plugin.terraform-provider-google: User-Agent: google-api-go-client/0.5 (linux amd64) Terraform/0.10.0-dev
2017-09-22T17:56:04.319-0400 [DEBUG] plugin.terraform-provider-google: Accept-Encoding: gzip
2017-09-22T17:56:04.321-0400 [DEBUG] plugin.terraform-provider-google: 
2017-09-22T17:56:04.321-0400 [DEBUG] plugin.terraform-provider-google: -----------------------------------------------------
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: 2017/09/22 17:56:04 [DEBUG] Google API Response Details:
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: ---[ RESPONSE ]--------------------------------------
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: HTTP/2.0 200 OK
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: Alt-Svc: quic=":443"; ma=2592000; v="39,38,37,35"
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: Cache-Control: private
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: Content-Type: application/json; charset=UTF-8
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: Date: Fri, 22 Sep 2017 21:56:05 GMT
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: Server: ESF
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: Vary: Origin
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: Vary: X-Origin
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: Vary: Referer
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: X-Content-Type-Options: nosniff
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: X-Frame-Options: SAMEORIGIN
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: X-Xss-Protection: 1; mode=block
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: 
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: {
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google:   "projectNumber": "00000000000000000",
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google:   "projectId": "test-service3-id",
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google:   "lifecycleState": "DELETE_REQUESTED",
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google:   "name": "test-service3-name",
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google:   "createTime": "2017-09-16T22:19:51.329Z",
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google:   "parent": {
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google:     "type": "organization",
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google:     "id": "00000000000000000"
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google:   }
2017-09-22T17:56:04.810-0400 [DEBUG] plugin.terraform-provider-google: }
...
2017-09-22T17:56:04.811-0400 [DEBUG] plugin.terraform-provider-google: -----------------------------------------------------
2017-09-22T17:56:04.812-0400 [DEBUG] plugin.terraform-provider-google: 2017/09/22 17:56:04 [DEBUG] Google API Request Details:
2017-09-22T17:56:04.812-0400 [DEBUG] plugin.terraform-provider-google: ---[ REQUEST ]---------------------------------------
2017-09-22T17:56:04.812-0400 [DEBUG] plugin.terraform-provider-google: GET /v1/projects/test-service3-id/billingInfo?alt=json HTTP/1.1
2017-09-22T17:56:04.812-0400 [DEBUG] plugin.terraform-provider-google: Host: cloudbilling.googleapis.com
2017-09-22T17:56:04.812-0400 [DEBUG] plugin.terraform-provider-google: User-Agent: google-api-go-client/0.5 (linux amd64) Terraform/0.10.0-dev
2017-09-22T17:56:04.812-0400 [DEBUG] plugin.terraform-provider-google: Accept-Encoding: gzip
2017-09-22T17:56:04.813-0400 [DEBUG] plugin.terraform-provider-google: -----------------------------------------------------
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: 2017/09/22 17:56:05 [DEBUG] Google API Response Details:
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: ---[ RESPONSE ]--------------------------------------
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: HTTP/2.0 404 Not Found
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: Alt-Svc: quic=":443"; ma=2592000; v="39,38,37,35"
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: Cache-Control: private
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: Content-Type: application/json; charset=UTF-8
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: Date: Fri, 22 Sep 2017 21:56:05 GMT
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: Server: ESF
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: Vary: Origin
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: Vary: X-Origin
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: Vary: Referer
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: X-Content-Type-Options: nosniff
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: X-Frame-Options: SAMEORIGIN
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: X-Xss-Protection: 1; mode=block
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: 
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: {
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google:   "error": {
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google:     "code": 404,
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google:     "message": "Requested entity was not found.",
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google:     "errors": [
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google:       {
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google:         "message": "Requested entity was not found.",
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google:         "domain": "global",
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google:         "reason": "notFound"
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google:       }
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google:     ],
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google:     "status": "NOT_FOUND"
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google:   }
2017-09-22T17:56:05.023-0400 [DEBUG] plugin.terraform-provider-google: }

[ewbankkit]

Call handleNotFoundError here or check project.lifecycleState here?

[rosbo]

Hi Kit,

I am curious about what behavior you expect when the project is in DELETE_REQUESTED state?

A) Delete from tf.state (that's what calling handleNotFoundError like you suggested would do) and recreate a brand new project.

B) Call undelete on the project making it ACTIVE again.

[ewbankkit]

If the project is deleted via terraform destroy then the resource will be removed from the corresponding state file and we shouldn't get into this situation on the next terraform plan (we'd see a new project was going to be created). So we should only get into this situation if the project is deleted outside of Terraform.
In this case I would say go with B) - re-activate the project; If the user really wants to delete the project she will have to remove the corresponding resource from the code AND the state file (else if it's not removed from the state file the next terraform plan will get the same error).

However, a case can be made for A) also 😄.

BTW, it looks like folders can also be undeleted and that resource has lifecycle_state attribute. Consistency of behavior will be important I think.

I can think of no prior art here for other resources/cloud providers I know except for some similarity to re-creating the AWS default VPC for a region - hashicorp/terraform-provider-aws#1400.

[rosbo]

Agreed, folders and projects should behave the same way for this.

I am also leaning towards B).

@danawillow @paddycarver @selmanj Any thoughts before I update our project and folder resources to handle this case?

[danawillow]

Yup, same.

I think we should handle DELETE_REQUESTED as if it were actually deleted- so a plan would show it as being created, and apply would un-delete it. It's a bit tricky because if it has any properties (I guess just billing account right now) that change, they'd have to be updated as well alongside the undelete.

[selmanj]

I am hesitant about using an undelete; from the docs:

Project owners can restore a terminated project within the 30-day waiting period that starts when the project is shut down. Restoring a project returns it to the state it was in prior to being shut down.

As a user, it would be surprising to me if I deleted a project, recreated it, and then all the previous state was restored (more specifically, all the resources associated with the project get revived). That works different from every other resource in terraform.

[rosbo]

@selmanj you convinced me. undelete cloud be quite surprising. If they deleted it from the console by mistake, they can undelete it from there.

I will handle the not found error properly and this will trigger a new project creation.

[paddycarver]

(Chiming in late that support for undelete would be very cool in certain situations, but I too believe it would lead to confusing and unexpected edge cases and it would probably be better to delete from state. So what y'all said, just agreeing. :) )

[AbhinayGupta741]

Thanks to @ewbankkit .
Well, the restoring terminated project in GCP worked for me.
We are facing the same issue while creating the new project through Jenkins using terraform.

[danawillow]

Hi @AbhinayGupta741, this issue has been closed for over two years so I think most of us have lost context as this point. If you're experiencing problems still, could you please open a new one in the tracker and fill out the template completely? Thanks!