[bug] AWS Bedrock model setup is not working anymore (extraneous key errors while testing)

[Faylixe]

We encountered an issue while deploying Onyx in a K8S cluster.

Context

The entier stack is deployed into an AWS EKS cluster, and the IAM role with proper permission to access Bedrock Runtime API is injected through a ServiceAccount. The underlying authentification method is through AWS_WEB_IDENTITY_TOKEN_FILE set into our API server container envvars.

Access to Bedrock model in this context is working, as we were able to successfully invoke conversation model eu.anthropic.claude-3-5-sonnet-20240620-v1:0 through following command into the target pod:

$ aws bedrock-runtime converse --model-id="eu.anthropic.claude-3-5-sonnet-20240620-v1:0" --messages '{"role": "user", "content": [{"text": "hello"}]}' --region "eu-west-1"

Configuration error

While configuring the model through the Admin Panel > LLM > AWS Bedrock setup screen (see screenshot below), with proper region, model and fast model filled, and AWS_ACCESS_KEY_ID plus AWS_SECRET_ACCESS_KEY left blank (as we use IAM role instead for authentication), the following error is returned:

WARNING: Failed to call LLM with the following error: litellm.BadRequestError: BedrockException - {"message":"The model returned the following errors: Malformed input request: #: extraneous key [AWS_ACCESS_KEY_ID] is not permitted#: extraneous key [AWS_SECRET_ACCESS_KEY] is not permitted#: extraneous key [AWS_REGION_NAME] is not permitted, please reformat your input and try again."}

Additional information

This bug has been introduced in last releases as we were able to reproduce it even locally through a docker compose setup while it wasn't the case 2 months ago while PoCing the solution. We weren't able to find the last version we had the configuration working but we still have the associated Docker image sha: c2f5827eb914.

[arindam2007b]

Facing the same issue. Any update on this?