[ISSUE] Concurrency issue when updating databricks_job resource

[junwei-db]

Configuration

"databricks_job": {
  "routing_job_clean_room_events": {
    "continuous": {
      "pause_status": "UNPAUSED"
    },
    "control_run_state": true,
    "name": "Routing Job for clean_room_events (routing_job_clean_room_events)",
    "notification_settings": {
      "no_alert_for_canceled_runs": true,
      "no_alert_for_skipped_runs": true
    },
    "retry_on_timeout": false,
    "tags": {
      "clusterIdentifier": "routing_job",
      "clusterOwner": "eng-foresight-team"
    },
    "task": [
      {
        "existing_cluster_id": "${databricks_cluster.routing_jobs.id}",
        "max_retries": 0,
        "notebook_task": {
          "base_parameters": {},
          "notebook_path": "${databricks_notebook.routing_job.id}"
        },
        "task_key": "clean_room_events"
      }
    ]
  }
}

Expected Behavior

Modifying the job should complete successfully.

databricks_job.routing_job_clean_room_events: Modifying...

Actual Behavior

Deployed non-deterministically failed due to

│ Error: cannot update job: A concurrent request to run the continuous job is already in progress. Please wait for it to complete before issuing a new request.
│ 
│   with databricks_job.routing_job_clean_room_events,
│   on terraform.tf.json line 5918, in resource.databricks_job.routing_job_clean_room_events:
│ 5918:       },

It appears that we hit some concurrently issues when updating a job.

#3585 appears to have addressed this issue, but we are still seeing the error using the latest provider version.

Steps to Reproduce

1. Create a job
2. Update the job
3. Terraform apply

Terraform and provider versions

Terraform version = 1.7
Databricks provider version = 1.56.0

Is it a regression?

Debug Output

Important Factoids

Would you like to implement a fix?