Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

google_bigquery_table error attempting to DROP COLUMN with column name containing reserved words #20750

Open
shoumoji opened this issue Dec 19, 2024 · 0 comments · May be fixed by GoogleCloudPlatform/magic-modules#12626
Open

google_bigquery_table error attempting to DROP COLUMN with column name containing reserved words #20750

shoumoji opened this issue Dec 19, 2024 · 0 comments · May be fixed by GoogleCloudPlatform/magic-modules#12626
Labels
bug forward/review In review; remove label to forward service/bigquery

Comments

@shoumoji
Copy link

shoumoji commented Dec 19, 2024 •
edited
Loading

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
  • Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment.
  • If an issue is assigned to a user, that user is claiming responsibility for the issue.
  • Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.10.2
on MacOS and GitHub Actions (ubuntu:22.04LTS)

  • provider registry.terraform.io/hashicorp/google v6.14.0

Affected Resource(s)

google_bigquery_table

Terraform Configuration

resource "google_bigquery_table" "table" {
  dataset_id          = var.bq_dataset_id
  table_id            = var.bq_table_id
  deletion_protection = false

  schema =  jsonencode(
    [
      { "name" : "dummy", "type" : "JSON", "mode" : "NULLABLE" },
      
      # Any column name containing BigQuery reserved words (e.g. x-forwarded-for, newbie-id),
      # 
      # After creating the column with `terraform apply`,
      # comment out the code and run `terraform apply` again to drop the column,
      # {"name" : "x-forwarded-for", "type" : "STRING", "mode" : "NULLABLE"},
    ]
  )
}

Debug Output

2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ---[ REQUEST ]---------------------------------------
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: GET /bigquery/v2/projects/<project_id>/datasets/<dataset_id>/tables/reserved-word-test?alt=json&prettyPrint=false HTTP/1.1
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Host: bigquery.googleapis.com
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: User-Agent: google-api-go-client/0.5 Terraform/1.10.2 (+https://www.terraform.io) Terraform-Plugin-SDK/2.33.0 terraform-provider-google/6.14.0
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Goog-Api-Client: gl-go/1.23.4 gdcl/0.206.0
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Accept-Encoding: gzip
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:39.274+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: -----------------------------------------------------
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Google API Response Details:
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ---[ RESPONSE ]--------------------------------------
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: HTTP/2.0 200 OK
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Cache-Control: private
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Content-Type: application/json; charset=UTF-8
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Date: Thu, 19 Dec 2024 05:36:39 GMT
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Etag: vE/w0CKk7jiIiZ2sF23+JA==
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Server: ESF
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Origin
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: X-Origin
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Referer
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Content-Type-Options: nosniff
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Frame-Options: SAMEORIGIN
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Xss-Protection: 0
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "kind": "bigquery#table",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "etag": "vE/w0CKk7jiIiZ2sF23+JA==",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "id": "<project_id>:<dataset_id>.reserved-word-test",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "selfLink": "https://bigquery.googleapis.com/bigquery/v2/projects/<project_id>/datasets/<dataset_id>/tables/reserved-word-test",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "tableReference": {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "projectId": "<project_id>",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "datasetId": "<dataset_id>",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "tableId": "reserved-word-test"
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: },
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "labels": {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "goog-terraform-provisioned": "true",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "managedby": "terraform",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "env": "dev"
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: },
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "schema": {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "fields": [
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "name": "data",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "type": "JSON",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "mode": "NULLABLE"
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: },
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "name": "x-forwarded-for",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "type": "STRING",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "mode": "NULLABLE"
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ]
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: },
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "timePartitioning": {
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "type": "DAY"
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: },
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "numBytes": "0",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "numLongTermBytes": "0",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "numRows": "0",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "creationTime": "1734574808432",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "lastModifiedTime": "1734574843702",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "type": "TABLE",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "location": "US",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "numTotalLogicalBytes": "0",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "numActiveLogicalBytes": "0",
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "numLongTermLogicalBytes": "0"
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: -----------------------------------------------------
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: Stopping retries, last request was successful
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: Returning after 1 attempts
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: Stopping retries, last request was successful
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: Returning after 1 attempts
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [INFO] Dropping columns in-place: ALTER TABLE <project_id>.<dataset_id>.reserved-word-test DROP COLUMN x-forwarded-for
2024-12-19T14:36:39.698+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [INFO] Instantiating Google Cloud BigQuery client for path https://bigquery.googleapis.com/bigquery/v2/
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: starting RoundTrip retry loop
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: request attempt 0
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: starting RoundTrip retry loop
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Retry Transport: request attempt 0
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:39 [DEBUG] Google API Request Details:
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ---[ REQUEST ]---------------------------------------
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: POST /bigquery/v2/projects/<project_id>/queries?alt=json&prettyPrint=false HTTP/1.1
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Host: bigquery.googleapis.com
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: User-Agent: google-api-go-client/0.5 Terraform/1.10.2 (+https://www.terraform.io) Terraform-Plugin-SDK/2.33.0 terraform-provider-google/6.14.0
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Content-Length: 113
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Content-Type: application/json
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Goog-Api-Client: gl-go/1.23.4 gdcl/0.206.0
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Accept-Encoding: gzip
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "query": "ALTER TABLE <project_id>.<dataset_id>.reserved-word-test DROP COLUMN x-forwarded-for",
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "useLegacySql": false
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:39.699+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: -----------------------------------------------------
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:40 [DEBUG] Google API Response Details:
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ---[ RESPONSE ]--------------------------------------
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: HTTP/2.0 400 Bad Request
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Cache-Control: private
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Content-Type: application/json; charset=UTF-8
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Date: Thu, 19 Dec 2024 05:36:39 GMT
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Server: ESF
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Origin
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: X-Origin
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Referer
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Content-Type-Options: nosniff
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Frame-Options: SAMEORIGIN
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Xss-Protection: 0
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "error": {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "code": 400,
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "message": "Syntax error: Expected end of input but got "-" at [1:66]",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "errors": [
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "message": "Syntax error: Expected end of input but got "-" at [1:66]",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "domain": "global",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "reason": "invalidQuery",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "location": "q",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "locationType": "parameter"
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ],
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "status": "INVALID_ARGUMENT"
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: -----------------------------------------------------
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:40 [DEBUG] Retry Transport: Stopping retries, last request failed with non-retryable error: googleapi: got HTTP response code 400 with body: HTTP/2.0 400 Bad Request
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Cache-Control: private
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Content-Type: application/json; charset=UTF-8
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Date: Thu, 19 Dec 2024 05:36:39 GMT
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Server: ESF
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Origin
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: X-Origin
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Referer
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Content-Type-Options: nosniff
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Frame-Options: SAMEORIGIN
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Xss-Protection: 0
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "error": {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "code": 400,
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "message": "Syntax error: Expected end of input but got "-" at [1:66]",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "errors": [
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "message": "Syntax error: Expected end of input but got "-" at [1:66]",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "domain": "global",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "reason": "invalidQuery",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "location": "q",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "locationType": "parameter"
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ],
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "status": "INVALID_ARGUMENT"
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:40 [DEBUG] Retry Transport: Returning after 1 attempts
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:40 [DEBUG] Retry Transport: Stopping retries, last request failed with non-retryable error: googleapi: got HTTP response code 400 with body: HTTP/2.0 400 Bad Request
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Cache-Control: private
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Content-Type: application/json; charset=UTF-8
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Date: Thu, 19 Dec 2024 05:36:39 GMT
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Server: ESF
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Origin
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: X-Origin
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: Vary: Referer
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Content-Type-Options: nosniff
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Frame-Options: SAMEORIGIN
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: X-Xss-Protection: 0
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "error": {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "code": 400,
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "message": "Syntax error: Expected end of input but got "-" at [1:66]",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "errors": [
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: {
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "message": "Syntax error: Expected end of input but got "-" at [1:66]",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "domain": "global",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "reason": "invalidQuery",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "location": "q",
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "locationType": "parameter"
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: ],
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: "status": "INVALID_ARGUMENT"
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: }
2024-12-19T14:36:40.086+0900 [DEBUG] provider.terraform-provider-google_v6.14.0_x5: 2024/12/19 14:36:40 [DEBUG] Retry Transport: Returning after 1 attempts
2024-12-19T14:36:40.087+0900 [ERROR] provider.terraform-provider-google_v6.14.0_x5: Response contains error diagnostic: tf_proto_version=5.7 tf_req_id=a4dd581e-e851-12cb-f67c-c62805218138 tf_resource_type=google_bigquery_table @caller=github.com/hashicorp/[email protected]/tfprotov5/internal/diag/diagnostics.go:58 @module=sdk.proto diagnostic_summary="googleapi: Error 400: Syntax error: Expected end of input but got "-" at [1:66], invalidQuery" tf_provider_addr=registry.terraform.io/hashicorp/google tf_rpc=ApplyResourceChange diagnostic_detail="" diagnostic_severity=ERROR timestamp="2024-12-19T14:36:40.087+0900"
2024-12-19T14:36:40.087+0900 [DEBUG] State storage *remote.State declined to persist a state snapshot
2024-12-19T14:36:40.087+0900 [ERROR] vertex "module.log_stream_reserved_word_test.google_bigquery_table.log_table" error: googleapi: Error 400: Syntax error: Expected end of input but got "-" at [1:66], invalidQuery

Expected Behavior

DROP column x-forwarded-for

Actual Behavior

Error: googleapi: Error 400: Syntax error: Expected end of input but got "-" at [1:66], invalidQuery

Steps to reproduce

  1. terraform apply to create any column with BigQuery reserved words (e.g. x-forwarded-proto, newbie-id)
  2. comment out the column definition
  3. terraform apply to drop the column
  4. Error

Important Factoids

  1. The cause of this bug is due to not backquoting the table name when constructing the query

terraform-provider-google/google/services/bigquery/resource_bigquery_table.go

Lines 1977 to 1987 in 81584b6

droppedColumns := []string{}
for _, field := range oldTable.Schema.Fields {
if !newTableFields[field.Name] {
droppedColumns = append(droppedColumns, field.Name)
}
}
if len(droppedColumns) > 0 {
droppedColumnsString := strings.Join(droppedColumns, ", DROP COLUMN ")
dropColumnsDDL := fmt.Sprintf("ALTER TABLE `%s.%s.%s` DROP COLUMN %s", tableReference.project, tableReference.datasetID, tableReference.tableID, droppedColumnsString)

  1. I created a PR to backquote the column names

GoogleCloudPlatform/magic-modules#12626

References

BigQuery reserved keywords
https://cloud.google.com/bigquery/docs/reference/standard-sql/lexical#reserved_keywords
@shoumoji shoumoji added the bug label Dec 19, 2024
@github-actions github-actions bot added forward/review In review; remove label to forward service/bigquery labels Dec 19, 2024
@shoumoji shoumoji mentioned this issue Dec 19, 2024
Closed
@shoumoji shoumoji changed the title google_bigquery_table error attempting to delete a column with a reserved word google_bigquery_table error attempting to DROP COLUMN with a reserved word Dec 19, 2024
@shoumoji shoumoji changed the title google_bigquery_table error attempting to DROP COLUMN with a reserved word google_bigquery_table error attempting to DROP COLUMN with column name containing reserved words Dec 19, 2024
@shoumoji shoumoji linked a pull request Dec 20, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug forward/review In review; remove label to forward service/bigquery
Projects
None yet
Milestone
No milestone
1 participant
@shoumoji