abstract user mapping and key management

[sigau]

Hello,

We are currently testing the S3 API for one of our projects, as we've received several requests from users who want to send data to iRODS through S3.

We're pleased with the progress so far—users can successfully send their data. However, I have a question regarding the configuration file setup.

At the moment, our configuration looks like this:

"static_authentication_resolver": {
    "name": "static_authentication_resolver",
    "users": {
        "user1": {
            "username": "user1",
            "secret_key": "secret1"
        },
        "user2": {
            "username": "user2",
            "secret_key": "secret2"
        },
        "user3": {
            "username": "user3",
            "secret_key": "secret3"
        },
        "user4": {
            "username": "user4",
            "secret_key": "user4"
        }
    }
}

Currently, we have only a few internal testers using the service, so we manually add their IDs and secret keys to the configuration. However, this approach becomes cumbersome and potentially insecure when we plan to open the service to external users.

My questions are:

1. Is there a way to automate the user ID and secret key management for the S3 API, similar to how it's handled with the HTTP API, where user mapping is done automatically without manual entry?

2. Alternatively, since our iRODS instance is accessible to external users, should we recommend that users who want to send data via S3 set up their own S3 API instances and connect to our iRODS? This way, each lab would manage their own S3 API, which would then interface with our iRODS. But with this, we don't want to give them the rods id and password for the proxy_admin_account.

I would appreciate any guidance or recommendations on how to best manage this.

Thank you!

[trel]

1. Not yet. But that was always the plan. We will provide a more abstract way to define users and keys. The current implementation (static_authentication_resolver) was the initial attempt and only reads the main configuration file. Once the users and keys can be loaded from elsewhere... that location can be managed by whatever process you find most useful and efficient (including rotating keys, updated from a separate source of truth, etc.). I will update this issue's title to reflect this goal. We would appreciate input / ideas on how you'd like to define this mapping. Is a separate JSON file sufficient?

2. Your assessment is correct - you do not want each lab to have rodsadmin access to the Zone. I would recommend against this plan.

[trel]

This will probably look similar to the efforts for the HTTP API - irods/irods_client_http_api#293.