cannot edit a google_compute_regional_network_endpoint_group in use by a google_compute_backend_service

[n-oden]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

$ terraform -v
Terraform v0.12.28
+ provider.google v3.39.0
+ provider.google-beta v3.39.0

Affected Resource(s)

• google_compute_regional_network_endpoint_group

Terraform Configuration Files

variable "project" {
  default = "oden-qa"
}

variable "region" {
  default = "us-central1"
}

provider "google" {
  region = var.region
}

provider "google-beta" {
  region = var.region
}

data "google_storage_bucket_object" "testfunction_zip" {
  name   = "testfunction.zip"
  bucket = "terraform_debug_n-oden"
}

resource "google_cloudfunctions_function" "testfunction" {
  name                  = "testServerlessNeg"
  entry_point           = "takehomeValueAPI_v1"
  project               = var.project
  region                = var.region
  runtime               = "nodejs10"
  source_archive_bucket = "${var.project}_cloudfunctions"
  source_archive_object = data.google_storage_bucket_object.testfunction_zip.name
  timeout               = 60
  trigger_http          = true
}

resource "google_cloudfunctions_function_iam_member" "testfunction" {
  project        = google_cloudfunctions_function.testfunction.project
  region         = google_cloudfunctions_function.testfunction.region
  cloud_function = google_cloudfunctions_function.testfunction.name

  role   = "roles/cloudfunctions.invoker"
  member = "allUsers"
}

resource "google_compute_global_address" "testfunction" {
  name    = "testfunction-address"
  project = var.project
}

resource "google_compute_global_forwarding_rule" "testfunction" {
  name       = "testfunction-http"
  project    = var.project
  target     = google_compute_target_http_proxy.testfunction.id
  ip_address = google_compute_global_address.testfunction.address
  port_range = "80"
}

resource "google_compute_target_http_proxy" "testfunction" {
  name    = "testfunction-http-proxy"
  url_map = google_compute_url_map.testfunction.self_link
  project = var.project
}

resource "google_compute_url_map" "testfunction" {
  name        = "testfunction-urlmap"
  description = "url map for the testfunction site"
  project     = var.project

  default_service = google_compute_backend_service.testfunction.self_link
}

resource "google_compute_backend_service" "testfunction" {
  name       = "testfunction"
  project    = var.project
  enable_cdn = false

  backend {
    group = google_compute_region_network_endpoint_group.testfunction.id
  }
}

resource "google_compute_region_network_endpoint_group" "testfunction" {
  provider              = google-beta
  name                  = "testfunction"
  network_endpoint_type = "SERVERLESS"
  region                = var.region
  project               = var.project
  cloud_function {
    function = google_cloudfunctions_function.testfunction.name
  }
}

Debug Output

https://gist.github.com/n-oden/96e996f46203eec04fef0a3b7f9f49fa

Expected Behavior

There are two runs in play here:

First Run: creates a cloud function, and a serverless network endpoint group and load balancer (google_compute_backend_service, google_compute_url_map, google_compute_target_http_proxy, google_compute_global_forwarding_rule) linked to the cloud function.

Second run (where the failure occurs): same as the first, but with the region variable changed from us-east1 to us-central1

The expected behavior would be that the regional resources (google_cloudfunctions_function, google_cloudfunctions_function_iam_member, google_compute_region_network_endpoint_group) would be deleted in us-east1 and re-created in us-central1

Actual Behavior

Terraform fails when trying to update the google_compute_regional_network_endpoint_group:

Error: Error when reading or editing RegionNetworkEndpointGroup: googleapi: Error 400: The network_endpoint_group resource 'projects/oden-qa/regions/us-east1/networkEndpointGroups/testfunction' is already being used by 'projects/oden-qa/global/backendServices/testfunction', resourceInUseByAnotherResource

Steps to Reproduce

1. terraform apply
2. terraform apply -var region=us-central1

[venkykuberan]

Looks like global back end service is still using your resource. Please try using google_compute_region_backend_service instead of global back_end service.

[n-oden]

Ah, fair point, but this leads me into what appears to be another lacuna:

resource "google_compute_url_map" "testfunction" {
  name        = "testfunction-urlmap"
  description = "url map for the testfunction site"
  project     = var.project

  default_service = google_compute_region_backend_service.testfunction.self_link
}

resource "google_compute_health_check" "testfunction" {
  name               = "testfunction"
  check_interval_sec = 1
  timeout_sec        = 1
  project            = var.project

  tcp_health_check {
    port = "80"
  }
}

resource "google_compute_region_backend_service" "testfunction" {
  name          = "testfunction"
  project       = var.project
  region        = var.region
  health_checks = [google_compute_health_check.testfunction.id]

  backend {
    group = google_compute_region_network_endpoint_group.testfunction.id
  }
}

...fails to apply with this error:

google_compute_region_backend_service.testfunction: Creating...

Error: Error creating RegionBackendService: googleapi: Error 400: Invalid value for field 'resource.healthChecks': ''. 
A backend service cannot have a healthcheck with Serverless network endpoint group backends., invalid

  on main.tf line 79, in resource "google_compute_region_backend_service" "testfunction":
  79: resource "google_compute_region_backend_service" "testfunction" {

...but if I leave out the health_checks attribute from the google_compute_region_backend_service, the apply fails because it claims the attribute is required:

Error: Missing required argument

  on main.tf line 79, in resource "google_compute_region_backend_service" "testfunction":
  79: resource "google_compute_region_backend_service" "testfunction" {

The argument "health_checks" is required, but no definition was found.

[n-oden]

I guess the actual issue here is that google_compute_region_backend_service should be ommitting the health_checks attribute when the backend is a serverless NEG. Should I open a different ticket on that, and/or do you need debug output from the runs that generated this error?

This seems conceptually similar to #6876

[aximov]

An external HTTP(S) Load Balancing works with a global backend service, so I guess google_compute_backend_service is appropriate.

[aximov]

Maybe #1883 is related?

[c2thorn]

This issue seems to have sidetracked from its initial post. I've merged a PR to address the issue @n-oden was facing in #7311 (comment) to make health_checks no longer required. If there are still problems past this point, I ask that a new issue is filed so we can better track.