auth: idtoken.NewCredentials doubly-impersonates an account when used with workload identity federation

[ericnorris]

Client

cloud.google.com/go/auth

Environment

Ubuntu 22.04 on GCE

Code and Dependencies

idtoken.NewCredentials(...)

Expected behavior

When used with GOOGLE_APPLICATION_CREDENTIALS=/path/to/workload-identity-federation-creds.json, idtoken.NewCredentials should:

• use the workload identity federation credentials to perform an STS exchange
• use the access token from the STS exchange to call getOpenIdToken on the service account in the service_account_impersonation_url field of workload-identity-federation-creds.json

Actual behavior

idtoken.NewCredentials instead:

• uses the workload identity federation credentials to perform an STS exchange
• uses the access token from the STS exchange to call getAccessToken on the service account in the service_account_impersonation_url field of workload-identity-federation-creds.json
• uses the access token from the previous step to call getOpenIdToken on the service account in the service_account_impersonation_url field of workload-identity-federation-creds.json

In other words, it impersonates the account in service_account_impersonation_url, and then as that account tries to impersonate itself again.

The final getOpenIdToken call fails with:

impersonate: status code 403: {
  "error": {
    "code": 403,
    "message": "Permission 'iam.serviceAccounts.getOpenIdToken' denied on resource (or it may not exist).",
    "status": "PERMISSION_DENIED",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.ErrorInfo",
        "reason": "IAM_PERMISSION_DENIED",
        "domain": "iam.googleapis.com",
        "metadata": {
          "permission": "iam.serviceAccounts.getOpenIdToken"
        }
      }
    ]
  }
}

This is because the service account does not have iam.serviceAccounts.getOpenIdToken on itself, which is expected, since it shouldn't need this permission. The workload identity principal has roles/iam.workloadIdentityUser on the target service account, and should be able to generate the ID token directly.

Screenshots

N/A

Additional context

This also occurs in the old version of the golang auth library, see googleapis/google-api-go-client#2301.

I believe this happens because idtoken.NewCredentials calls credentials.DetectDefault:

google-cloud-go/auth/credentials/idtoken/idtoken.go

Lines 125 to 130 in 79556c4

	creds, err := credentials.DetectDefault(&credentials.DetectOptions{
	Scopes: defaultScopes,
	CredentialsJSON: b,
	Client: opts.client(),
	UseSelfSignedJWT: true,
	})

...which eventually finds the external_account credentials:

google-cloud-go/auth/credentials/filetypes.go

Lines 60 to 69 in 79556c4

	case credsfile.ExternalAccountKey:
	f, err := credsfile.ParseExternalAccount(b)
	if err != nil {
	return nil, err
	}
	tp, err = handleExternalAccount(f, opts)
	if err != nil {
	return nil, err
	}
	universeDomain = resolveUniverseDomain(opts.UniverseDomain, f.UniverseDomain)

...and handleExternalAccount calls externalaccount.NewTokenProvider:

google-cloud-go/auth/credentials/filetypes.go

Line 185 in 79556c4

return externalaccount.NewTokenProvider(externalOpts)

...which wraps the token provider with an impersonate.NewTokenProvider:

google-cloud-go/auth/credentials/internal/externalaccount/externalaccount.go

Lines 243 to 261 in 79556c4

	if opts.ServiceAccountImpersonationURL == "" {
	return auth.NewCachedTokenProvider(tp, nil), nil
	}
	scopes := make([]string, len(opts.Scopes))
	copy(scopes, opts.Scopes)
	// needed for impersonation
	tp.opts.Scopes = []string{"https://www.googleapis.com/auth/cloud-platform"}
	imp, err := impersonate.NewTokenProvider(&impersonate.Options{
	Client: client,
	URL: opts.ServiceAccountImpersonationURL,
	Scopes: scopes,
	Tp: auth.NewCachedTokenProvider(tp, nil),
	TokenLifetimeSeconds: opts.ServiceAccountImpersonationLifetimeSeconds,
	})
	if err != nil {
	return nil, err
	}
	return auth.NewCachedTokenProvider(imp, nil), nil

So now idtoken.NewCredentials has a credential that will automatically impersonate the service_account_impersonation_url account, and it then calls credsFromDefault:

google-cloud-go/auth/credentials/idtoken/idtoken.go

Line 134 in 79556c4

return credsFromDefault(creds, opts)

...which creates a second impersonate.NewTokenProvider with these credentials:

google-cloud-go/auth/credentials/idtoken/file.go

Lines 78 to 106 in 79556c4

	case credsfile.ImpersonatedServiceAccountKey, credsfile.ExternalAccountKey:
	type url struct {
	ServiceAccountImpersonationURL string `json:"service_account_impersonation_url"`
	}
	var accountURL url
	if err := json.Unmarshal(b, &accountURL); err != nil {
	return nil, err
	}
	account := filepath.Base(accountURL.ServiceAccountImpersonationURL)
	account = strings.Split(account, ":")[0]
	config := impersonate.IDTokenOptions{
	Audience: opts.Audience,
	TargetPrincipal: account,
	IncludeEmail: true,
	Client: opts.client(),
	Credentials: creds,
	}
	idTokenCreds, err := impersonate.NewIDTokenCredentials(&config)
	if err != nil {
	return nil, err
	}
	return auth.NewCredentials(&auth.CredentialsOptions{
	TokenProvider: idTokenCreds,
	JSON: b,
	ProjectIDProvider: auth.CredentialsPropertyFunc(creds.ProjectID),
	UniverseDomainProvider: auth.CredentialsPropertyFunc(creds.UniverseDomain),
	QuotaProjectIDProvider: auth.CredentialsPropertyFunc(creds.QuotaProjectID),
	}), nil

Thus, it impersonates the service_account_impersonation_url in order to get an access token, and then tries to impersonate itself to get an ID token.

I believe that ideally, the library would instead use a non-impersonated credential provider to do the STS exchange and wrap that in a single impersonated credential provider to get the ID token.

[quartzmo]

@ericnorris Thank you for explaining this clearly. I will label this issue as a feature request, similar to googleapis/google-api-go-client#2301. I'm not sure when or if we will change this behavior as you suggest. The first step will be to try to understand why it was implemented in this way in this first place. Thank you for your patience.

[ericnorris]

Thanks for taking a look @quartzmo. I respectfully disagree, however, that this is a feature request and not a bug report.

https://cloud.google.com/iam/docs/workload-identity-federation#impersonation states:

As an alternative to providing direct resource access you can use service account impersonation.
You must grant your service account the role Workload Identity User (roles/iam.workloadIdentityUser).

It does not say that you must grant it roles/iam.workloadIdentityUser and roles/iam.serviceAccountTokenCreator or roles/iam.serviceAccountOpenIdTokenCreator, but due to the way this auth library is written, one of the latter roles is required.

In addition, I believe granting roles/iam.serviceAccountTokenCreator is a security risk. Once you have a valid token, you could then use the token to call iam.serviceAccounts.getOpenIdToken, which you can then use to call iam.serviceAccounts.getOpenIdToken again, ad infinitum.

Fixing this behavior wouldn't be a backwards compatibility break either, right? Users currently need to grant two IAM roles, and if it was fixed, it would still work; the second role would just be superfluous.

[quartzmo]

@ericnorris These are good points, thank you. I will see what I can learn about the reasons (if any) for current design.