Support For AssumeRole

[armenr]

We run multiple accounts in AWS, one is a sandbox, others serve their own purposes. Is there some way to specify which account I want terraforming to use?

I have assume_role policies set up for the key/secret key I'm sourcing in my env, so the credentials work across any of the accounts.

I'm just trying to figure out what param to feed terraforming (if it supports it) so that it knows which account's resources to hit.

[coingraham]

Put the keys in your credentials file in ~/.aws/credentials under a profile name and use the --profile switch when calling terraforming commands.

[armenr]

First, please know how grateful I am that a tool like this exists out there. It has been very, very useful for me for many, many years. I think I first forked this project nearly two years ago. I've been a fan ever since.

Main account ID w/ AssumeRole user: 000000000000
Managed account ID where I can AssumeRole into AccountAdministrator: 111111111111

Confirmed working:

• Main account is where "myuser" is created in IAM
• Set up "Trust Relationship" in Managed account
• Give the AssumeRole in Managed account full Admin privileges
• Successfully AssumeRole into ManagedAccount through AWS Console and administer all resources (IAM, EC2, etc)

Steps to reproduce:

Using the AWS CLI instead of Terraforming, I have a ~/.aws/config file where I stored "myuser" AWS keys (from IAM credentials provided in the Main account. It looks like this:

[profile default]
aws_access_key_id = <REDACTED>         #myuser credential from IAM in Main account
aws_secret_access_key = <REDACTED>     #myuser credential from IAM in Main account

[profile dev-east]
role_arn = arn:aws:iam::111111111111:role/OrganizationAccountAccessRole
source_profile = default
aws_region = us-east-1

When I run the AWS CLI, it works:

╰─ aws iam list-users --profile dev-east
{
    "Users": [
        {
            "UserName": "something1",
            "Path": "/",
            "PasswordLastUsed": BLEEP,
            "CreateDate": BLEEP,
            "UserId": "BLEEP",
            "Arn": "arn:aws:iam::111111111111:user/something1"
        },
        {
            "UserName": "something2",
            "Path": "/",
            "PasswordLastUsed": BLEEP,
            "CreateDate": BLEEP,
            "UserId": "BLEEP",
            "Arn": "arn:aws:iam::111111111111:user/something2"
        }
    ]
}

I set up an identical profile in my ~/.aws/credentials file which looks like this:

[default]
aws_access_key_id = <REDACTED>         #myuser credential from IAM in Main account
aws_secret_access_key = <REDACTED>     #myuser credential from IAM in Main account

[dev-east]
aws_access_key_id = <REDACTED>         #myuser credential from IAM in Main account
aws_secret_access_key = <REDACTED>     #myuser credential from IAM in Main account
role_arn = arn:aws:iam::111111111111:role/OrganizationAccountAccessRole
source_profile = default
aws_region = us-east-1

When I run terraforming, here's what I get:

╰─ terraforming iamu --profile dev-east
/Users/armenr/.rvm/gems/ruby-2.4.0/gems/aws-sdk-core-
2.10.28/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call': User: 
arn:aws:iam::000000000000:user/myuser is not authorized to perform: iam:ListUsers on resource: 
arn:aws:iam::000000000000:user/ (Aws::IAM::Errors::AccessDenied)

The really troubling part is:

arn:aws:iam::000000000000:user/myuser is not authorized to perform: iam:ListUsers on resource: 
arn:aws:iam::000000000000:user/ (Aws::IAM::Errors::AccessDenied)

I get the same issue with any other type of resource - ec2, rds, etc...

It seems terraforming/AWS RubySDK is running listUsers on the 000000000000 account, instead of assuming the role in account 111111111111, and running IAM listUsers on that account...but I don't have that problem wit the AWS CLI version of the same command.

Moreover, it seems like it's a known issue elsewhere with the way in which the AWS Client is first constructed in the RubySDK. I'm also parsing through your code and can't seem to find anything in the repo regarding Aws::STS::Client.new (creating STS client to acquire temporary AssumeRole credentials) or Aws::SharedCredentials.net (building shared credentials profile objects).

For example, in your iam_user.rb, all I see is:

      def self.tf(client: Aws::IAM::Client.new)
        self.new(client).tf
      end

      def self.tfstate(client: Aws::IAM::Client.new)
        self.new(client).tfstate
      end

1. I guess we'd have to build a wrapper that would initialize all user profiles by parsing ~/.aws/credentials and construct profile objects first, using a tool like:
   https://github.com/a2ikm/aws_config

2. Then, we'd have to create an STS client

3. Then, we'd have to construct a credentials object

4. Then, we'd have to create an IAM client, passing in the credentials object.

Please forgive the code below, I'm doing most of this off of a tablet, while in transit, so it's probably totally messed up -

module Terraforming
  module Resource
    class IAMUser
      include Terraforming::Util

      def self.sts(sts_client: Aws::STS::Client.new(profile: 'dev-east')
        self.new(sts_client).tf
      end

      def self.credentials(Aws::AssumeRoleCredentials.new(
        client: client,
        role_arn: role_arn,
        role_session_name: "some_session",
        token_code: token_code
      ))
        self.new(credentials)
      end

      def

      def self.tf(client: Aws::IAM::Client.new(credentials: self.credentials))
        self.new(client).tf
      end

      def self.tfstate(client: Aws::IAM::Client.new(credentials: self.credentials))
        self.new(client).tfstate
      end

      def initialize(client)
        @client = client
      end

Basically, I'm wondering if terraforming is capable of supporting a way to AssumeRole to other account resources, in order to list them?

IF not, is it cool if I open a PR to try to work that out?

[cmedley]

@armenr we have multiple accounts and had same issue ... see if this works for you https://github.com/cmedley/terraforming/tree/support-assume-role (if it does, will open PR)

[armenr]

@cmedley --> Thanks for that! This is clearly more elegant than any Ruby I was going to be able to hack together.

I'm gonna take this for a test drive and see how it fares. Definitely glad someone around these parts forked and wrote it before I had the chance to mutilate the code.

Thank you!

[varunchandak]

@cmedley version is not working for me using --assume

[armenr]

I can confirm that I was able to get it working back in August/September when he shared it to me.

I can try to test again and let you know...maybe help to repro the issue.

[cmedley]

I will test as well and update over weekend ... need to remove the options[:client] from cli.rb and update the tests (but that wouldn't stop it from working)

…

On Fri, Oct 13, 2017 at 9:18 PM, Armen Rostamian ***@***.***> wrote: I can confirm that I was able to get it working back in August/September when he shared it to me. I can try to test again and let you know...maybe help to repro the issue. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#356 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMSY38EUk4py5o68DPUD4xQChcfjcAQks5ssAvJgaJpZM4O74eh> .

[cmedley]

@varunchandak @armenr created a new pull request for assume role support ... #379

[rmishra-github]

Amazing tool.Thank you so much for this ...is it possible to do terraforming for a specific resource?
something like terraforming ec2.{ec2-id} ?

[o6uoq]

+1

[dadreggors]

I still cannot make this work. I can use aws with --profile flag working perfectly but with terraforming, I use --profile <as seen in .aws/credentials> and --assume <my ARN as seen in .aws/config> and I get nothing returned....

Example Credentials profile used:

[MY-PROFILE]
aws_access_key_id =<REDACTED>
aws_secret_access_key = <REDACTED>
aws_session_token = <REDACTED>

Example Config file in use:

[profile MY-ACCOUNT/MY-ROLE]
region = us-west-2
output = json
role_arn = arn:aws:iam::<ACCOUNT ID>:role/<ROLE>
source_profile = MY-PROFILE

Example command and output...

$ terraforming --region=us-west-2 --profile=MY-PROFILE --assume=arn:aws:iam::<ACCOUNT ID>:role/<ROLE> ec2
Usage:
  terraforming ec2

Options:
  [--merge=MERGE]                                # tfstate file to merge
  [--overwrite], [--no-overwrite]                # Overwrite existing tfstate
  [--tfstate], [--no-tfstate]                    # Generate tfstate
  [--profile=PROFILE]                            # AWS credentials profile
  [--region=REGION]                              # AWS region
  [--assume=ASSUME]                              # Role ARN to assume
  [--use-bundled-cert], [--no-use-bundled-cert]  # Use the bundled CA certificate from AWS SDK

EC2

Example aws usage and output

$ aws --profile MY-ACCOUNT/MY-ROLE sts get-caller-identity
{
    "UserId": "<REDACTED>:botocore-session-<REDACTED>",
    "Account": "<REDACTED>",
    "Arn": "arn:aws:sts::<ACCOUNT ID>:assumed-role/<ROLE>/botocore-session-<REDACTED>"
}

I can also describe instances and see proper results (too large to list here)... yet terraforming does not return anything for ec2 when using this config setup.

[cmedley]

@ddreggors think ec2 needs to go before the options $ terraforming ec2 --region ...

[dadreggors]

@ddreggors think ec2 needs to go before the options $ terraforming ec2 --region ...

You are correct, that fixed it right up thanks!

[dimisjim]

@ddreggors @cmedley Using both --assume and --profile still doesn't work for me. I am getting:

/var/lib/gems/2.5.0/gems/aws-sigv4-1.1.0/lib/aws-sigv4/signer.rb:670:in `credentials_set?': undefined method `access_key_id' for nil:NilClass (NoMethodError)

My ~/.aws/credentials looks like this:

[default]
aws_access_key_id = ...
aws_secret_access_key = ...

[profile1]
role_arn = arn:aws:iam::<account-id-1>:role/<role-name-1>
source_profile = default
mfa_serial = arn:aws:iam::<central-account-id>:mfa/<my-username>

[profile2]
role_arn = arn:aws:iam::<account-id-2>:role/<role-name-2>
source_profile = default
mfa_serial = arn:aws:iam::<central-account-id>:mfa/<my-username>

[cmedley]

@dimisjim what does the command look like that you are using?

[dimisjim]

@cmedley terraforming s3 --region=eu-west-1 --assume=arn:aws:iam::<account-id-1>:role/<role-name-1> --profile=<profile-1>

Getting the same if I put --profile arg before --assume or if I have exported AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_REGION or not

[cmedley]

@dimisjim getting same error as you ... to make this work it looks like we need to pass serial number and token code in the params to support MFA devices. I can get a PR going soon.

Short term, you can use aws cli to create the credentials aws sts assume-role --serial arn:aws:iam::<central-account-id>:mfa/<my-username> --role-session-name=<a-name> --role-arn="arn:aws:iam::<account-id-1>:role/<role-name-1>" --token-code <your-mfa-token> then export AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN and then run terraforming s3

[armenr]

It's been 2 years, and you guys are still helping/supporting this feature and issue. So incredibly cool to see.

//end gushy pointless post.

[cmedley]

@dimisjim got a PR #463 that adds MFA support to assume role. Should work using terraforming s3 --profile=<your_profile> --assume=<role_arn> --mfa-serial=<mfa_arn> --token-code=<mfa_token>

[dimisjim]

@cmedley Awesome! How can I test this?

[cmedley]

@dimisjim can test this way

 $ git clone https://github.com/dtan4/terraforming.git
 $ cd terraforming
 $ git fetch origin pull/463/head:assume-role-with-mfa-support
 $ git checkout assume-role-with-mfa-support
 $ bundle exec rake install
 $ terraforming s3 --profile=<your_profile> --assume=<role_arn> --mfa-serial=<mfa_arn> --token-code=<mfa_token> --region=<region>

[dimisjim]

@cmedley

Ok, having some "require" difficulties with running the produced binary in terraforming/bin, I did the following:

Assuming your instructions up to the bundle exec command (+ having ran the scripts/setup prior to that) and by also editing after it:

• in /path/to/terraforming/lib/terraforming.rb:
  • all require lines to require_relative that reference paths like terraforming/util, terraforming/version, etc.
  • Added Terraforming::CLI.start(ARGV) at the end of the file.

Then, having my ~/.aws/credentials file containing my default mfa profile that links to all the other accounts that it can assume a role there, I ran the command you proposed by omitting the --profile flag:

ruby /path/to/terraforming/lib/terraforming.rb s3 --assume=arn:aws:sts::<AccountIdThatDefaultAccountCanAssumeItsRole>:role/<RoleName> --mfa-serial=arn:aws:iam::<AccountIdOfDefaultAccount>:mfa/<Username> --token-code=<code> --region=<region>

This worked beautifully!

Can we proceed with merging the relevant PR #463 ?