[Authentication] Support for IAM Roles via inline profile parameters

[Fleid]

Hey all,

It would be great if we could support IAM Role authentication fully via profile parameters (without the need for a local AWS profile).

Currently, the adapter supports IAM User authentication, both via an AWS profile and inline parameters:

• IAM User via AWS profile:

~/.aws/config

[profile IAMuser1]
region = us-east-2
output = json

~/.aws/credentials

[IAMuser1]
aws_access_key_id = AAA
aws_secret_access_key = BBB

~/.dbt/profiles.yml

...
  method: IAM
  iam_profile: IAMuser1
...

• IAM User via inline fields:
  ~/.dbt/profiles.yml

...
  method: IAM
  region: us-east-2
  access_key_id: AAA
  secret_access_key: BBB
...

It would be amazing if we could support IAM Role based authentication similarly:

• IAM Role via AWS profile:

~/.aws/config

[profile IAMuser1]
region = us-east-2
output = json

[profile IAMrole1]
role_arn = arn:aws:iam::XXX:role/myIAMRole
source_profile = IAMuser1

~/.aws/credentials

[IAMuser1]
aws_access_key_id = AAA
aws_secret_access_key = BBB

~/.dbt/profiles.yml

...
  method: IAM
  iam_profile: IAMrole1
...

• IAM User via inline fields:
  ~/.dbt/profiles.yml

...
  method: IAM
  region: us-east-2
  access_key_id: AAA
  secret_access_key: BBB
  role_arn: arn:aws:iam::XXX:role/myIAMRole
...

Basically, if a role_arn is present in the profile, we check for the presence of an access_key_id and a secret_access_key.
If the three are there, we pass that to the Redshift Connector library to handle the connection via role for us.

This would look something like the following in

dbt-redshift/dbt/adapters/redshift/connections.py

Line 234 in cef201f

def _iam_role_kwargs(self) -> Dict[str, Optional[Any]]:

        # Either we get the 3 required fields together, or we load an AWS iam profile
        if self.credentials.access_key_id and self.credentials.secret_access_key and self.credentials.role_arn:
            kwargs.update(
                access_key_id=self.credentials.access_key_id,
                secret_access_key=self.credentials.secret_access_key,
                role_arn=self.credentials.role_arn
            )
        elif self.credentials.access_key_id or self.credentials.secret_access_key or self.credentials.role_arn:
            raise FailedToConnectError(
                "'access_key_id', 'secret_access_key' and 'role_arn' are all needed if providing explicit credentials"
            )
        else:
            kwargs.update(profile=self.credentials.iam_profile)
        if iam_profile := self.credentials.iam_profile:
            kwargs.update(profile=iam_profile)

NOTE

This is not supported by the Redshift Library as of now. If it does take all 3 parameters, it ignores role_arn and does an IAM User authentication on the keys. It looks like boto3 is the one actually doing that under the covers.

I'm thinking this is the case because I'm not seeing role_arn anywhere in aws_credentials_provider.

But it is supported via an AWS Profile: see the boto3 credentials option, we're in the assume role provider chapter, particularly: If MFA authentication is not enabled then you only need to specify a role_arn and a source_profile.