feat: add credentials_secret field in azure blob storage block for google storage transfer job resource

[kumailkermalli-datatonic]

Modifies the handwritten config for google_storage_transfer_job resource to add the credentials_secret field in the azure_blob_storage_data_source block.

Note the credentials_secret field is in preview, so the change will only be present in the beta provider

Fixes: hashicorp/terraform-provider-google#16263

Note to reviewer: There are no existing tests which cover the fields in thetransfer_spec.azure_blob_storage_data_source block. Please advise on how to proceed!

Release Note Template for Downstream PRs (will be copied)

storagetransferservice: added field `transfer_spec.azure_blob_storage_data_source.credentials_secret` to `google_storage_transfer_job` (beta)

[modular-magician]

Hello! I am a robot. It looks like you are a: Community Contributor Googler Core Contributor. Tests will require approval to run.

@trodge, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

[kumailkermalli-datatonic]

Hey @trodge 👋

Was wondering if you had a chance to take a look at this?

At the moment, there are no tests which cover both the field I added and the existing fields in transfer_spec.azure_blob_storage_data_source block. Given that we would need the appropriate azure credentials in order to test them, please could you advise on how to proceed?

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 3 insertions(+), 1 deletion(-))
Terraform Beta: Diff ( 2 files changed, 30 insertions(+), 12 deletions(-))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_storage_transfer_job (21 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_storage_transfer_job" "primary" {
  transfer_spec {
    azure_blob_storage_data_source {
      credentials_secret = # value needed
    }
  }
}

[modular-magician]

Tests analytics

Total tests: 3170
Passed tests 2850
Skipped tests: 320
Affected tests: 0

$\textcolor{green}{\textsf{All tests passed in REPLAYING mode.}}$
View the build log

[kumailkermalli-datatonic]

Hi @trodge

Thanks for running the tests.

For the missing acceptance test corresponding to transfer_spec.azure_blob_storage_data_source.credentials_secret; even if we generate an empty secret and pass this through, the resource will fail to create. This is because the API will reject the request as it validates the Azure SAS Token in the secret before creating the resource.

AKA: we need a SAS token as part of the tests so that we can validate that the field works as intended

However, I can see that this resource isn't covered fully by tests, e.g transfer_spec.azure_blob_storage_data_source.azure_credentials.sas_token doesn't have a test associated with it either - I presume because we do not have a SAS token to hand.

If you could please advise on how to proceed that would be great!

[trodge]

This resource is maintained by a service team at Google and @thomasmaclean is our point of contact.

It looks like some additional infrastructure may need to be provisioned to test this new feature.

[kumailkermalli-datatonic]

Hey @trodge and @thomasmaclean! Hope you are both doing well

Was wondering if there was any update on this?

[danicomnea-wpp]

This resource is maintained by a service team at Google and @thomasmaclean is our point of contact.

It looks like some additional infrastructure may need to be provisioned to test this new feature.

👋 @trodge , i'm happy to join the effort to help with a testing the functionality. I'll reach out to my TAM to speak with you , hopefully that can maybe help with speeding up the validation/ merge.

[nija-computas]

@trodge @thomasmaclean also very interested in this coming out as using SAS tokens directly is a bit of a blocker for using this service - any status updates?

[melinath]

Sorry for the delay on this one, we've been working out how best to ensure that this has tests that verify it works, since it requires an Azure test environment. We've decided to move forward on this with manual test runs for now.

@kumailkermalli-datatonic - is there any chance you would be able to add new create and update tests that exercise the azure fields and share debug logs from successful beta & GA test runs? (The tests should be marked with t.Skip("requires azure test environment") for now; they could pull the credentials from the environment for now - be sure to scrub them from your debug logs.) This is not a blocker but would be nice to have.

@trodge we now have manual provisioning logs in b/306619208#comment38. Please consider this review active again at this point.

[trodge]

/gcbrun

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 3 insertions(+), 1 deletion(-))
Terraform Beta: Diff ( 2 files changed, 30 insertions(+), 12 deletions(-))

Missing test report

Your PR includes resource fields which are not covered by any test.

Resource: google_storage_transfer_job (25 total tests)
Please add an acceptance test which includes these fields. The test should include the following:

resource "google_storage_transfer_job" "primary" {
  transfer_spec {
    azure_blob_storage_data_source {
      credentials_secret = # value needed
    }
  }
}

[modular-magician]

Tests analytics

Total tests: 11
Passed tests: 11
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages

• storagetransfer

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log