Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

enclaver cannot reach S3 from within enclave #197

Open
pavankad opened this issue Sep 28, 2024 · 8 comments · Fixed by edgebitio/no-fly-list#38
Open

enclaver cannot reach S3 from within enclave #197

pavankad opened this issue Sep 28, 2024 · 8 comments · Fixed by edgebitio/no-fly-list#38

Comments

@pavankad
Copy link

I am trying to connect to S3 bucket to download file. KMS proxy works fine. But, it is not able to reach S3 bucket. As shown below in the logs, s3 is being proxied thru http://localhost:10000(I have not speficied this anywhere).

My use case is similar to the below github repo that you published. Any help is greatly appreciated.

https://github.com/edgebitio/no-fly-list/blob/main/server.py

I have following enclaver.yaml config

version: v1
name: "depa"
target: "depa:enclave-latest"
sources:
app: "depa:latest"
defaults:
memory_mb: 2048
kms_proxy:
listen_port: 9999
egress:
allow:
- kms.*.amazonaws.com
- s3.amazonaws.com
- 169.254.169.254
ingress:

  • listen_port: 8000

INFO enclaver::run > starting egress proxy on vsock port 17002
INFO enclaver::vsock > Listening on vsock port 17002
INFO enclaver::run > starting enclave

INFO enclaver::run > started enclave i-03731652486c31676-enc19239f9d47ef852
INFO enclaver::run > starting ingress proxy on port 8000
INFO enclaver::run > waiting for enclave to boot to stream logs
INFO enclaver::run > connected to enclave, starting log stream
INFO enclave > INFO enclaver::vsock > Listening on vsock port 17001
INFO enclave > INFO odyn::enclave > Bringing up loopback interface
INFO enclave > INFO odyn::enclave > Seeding /dev/random with entropy from nsm device
INFO enclave > INFO odyn > Enclave initialized
INFO enclave > INFO odyn::egress > Starting egress
INFO enclave > INFO odyn::ingress > Starting TCP ingress on port 8000
INFO enclave > INFO enclaver::vsock > Listening on vsock port 8000
INFO enclave > INFO odyn::kms_proxy > Starting KMS proxy
INFO enclave > INFO odyn::kms_proxy > Generating public/private keypair
INFO enclave > INFO odyn::kms_proxy > Fetching credentials from IMDSv2
INFO enclave > INFO odyn::kms_proxy > Credentials fetched
INFO enclave > INFO odyn > Starting ["python3", "-m", "flask", "run", "--host=0.0.0.0", "--port=8000"]
INFO enclave > * Serving Flask app '/app/server.py'
INFO enclave > * Debug mode: off
INFO enclave > WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
INFO enclave > * Running on all addresses (0.0.0.0)
INFO enclave > * Running on http://127.0.0.1:8000
INFO enclave > * Running on http://127.0.0.1:8000
INFO enclave > Press CTRL+C to quit
INFO enclave > Starting server...
INFO enclave > Here are params received>>>>>>>>
INFO enclave > Instance Profil name KMS_instance
INFO enclave > us-east-1
INFO enclave > Could not read file from S3
INFO enclave > Failed to connect to proxy URL: "http://127.0.0.1:10000/"
INFO enclave > [2024-09-28 18:53:22,721] ERROR in app: Exception on /config [POST]
INFO enclave > Traceback (most recent call last):
INFO enclave > File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 712, in urlopen
INFO enclave > self._prepare_proxy(conn)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1014, in _prepare_proxy
INFO enclave > conn.connect()
INFO enclave > File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 374, in connect
INFO enclave > self._tunnel()
INFO enclave > File "/usr/lib64/python3.9/http/client.py", line 930, in _tunnel
INFO enclave > raise OSError(f"Tunnel connection failed: {code} {message.strip()}")
INFO enclave > OSError: Tunnel connection failed: 401 Unauthorized
INFO enclave >
INFO enclave > During handling of the above exception, another exception occurred:
INFO enclave >
INFO enclave > Traceback (most recent call last):
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/httpsession.py", line 464, in send
INFO enclave > urllib_response = conn.urlopen(
INFO enclave > File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 801, in urlopen
INFO enclave > retries = retries.increment(
INFO enclave > File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 527, in increment
INFO enclave > raise six.reraise(type(error), error, _stacktrace)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/urllib3/packages/six.py", line 769, in reraise
INFO enclave > raise value.with_traceback(tb)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 712, in urlopen
INFO enclave > self._prepare_proxy(conn)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1014, in _prepare_proxy
INFO enclave > conn.connect()
INFO enclave > File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 374, in connect
INFO enclave > self._tunnel()
INFO enclave > File "/usr/lib64/python3.9/http/client.py", line 930, in _tunnel
INFO enclave > raise OSError(f"Tunnel connection failed: {code} {message.strip()}")
INFO enclave > urllib3.exceptions.ProxyError: ('Cannot connect to proxy.', OSError('Tunnel connection failed: 401 Unauthorized'))
INFO enclave >
INFO enclave > During handling of the above exception, another exception occurred:
INFO enclave >
INFO enclave > Traceback (most recent call last):
INFO enclave > File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2529, in wsgi_app
INFO enclave > response = self.full_dispatch_request()
INFO enclave > File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1825, in full_dispatch_request
INFO enclave > rv = self.handle_user_exception(e)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
INFO enclave > rv = self.dispatch_request()
INFO enclave > File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
INFO enclave > return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
INFO enclave > File "/app/server.py", line 186, in get_config
INFO enclave > encrypted_data_from_s3 = read_file_from_s3(s3_client, s3_bucket, filename)
INFO enclave > File "/app/server.py", line 142, in read_file_from_s3
INFO enclave > response = s3_client.get_object(Bucket=bucket_name, Key=file_key)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 553, in _api_call
INFO enclave > return self._make_api_call(operation_name, kwargs)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 989, in _make_api_call
INFO enclave > http, parsed_response = self._make_request(
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 1015, in _make_request
INFO enclave > return self._endpoint.make_request(operation_model, request_dict)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 119, in make_request
INFO enclave > return self._send_request(request_dict, operation_model)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 202, in _send_request
INFO enclave > while self._needs_retry(
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 354, in _needs_retry
INFO enclave > responses = self._event_emitter.emit(
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/hooks.py", line 412, in emit
INFO enclave > return self._emitter.emit(aliased_event_name, **kwargs)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/hooks.py", line 256, in emit
INFO enclave > return self._emit(event_name, kwargs)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/hooks.py", line 239, in _emit
INFO enclave > response = handler(**kwargs)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/retryhandler.py", line 207, in call
INFO enclave > if self._checker(**checker_kwargs):
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/retryhandler.py", line 284, in call
INFO enclave > should_retry = self._should_retry(
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/retryhandler.py", line 320, in _should_retry
INFO enclave > return self._checker(attempt_number, response, caught_exception)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/retryhandler.py", line 363, in call
INFO enclave > checker_response = checker(
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/retryhandler.py", line 247, in call
INFO enclave > return self._check_caught_exception(
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/retryhandler.py", line 416, in _check_caught_exception
INFO enclave > raise caught_exception
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 281, in _do_get_response
INFO enclave > http_response = self._send(request)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 377, in _send
INFO enclave > return self.http_session.send(request)
INFO enclave > File "/usr/local/lib/python3.9/site-packages/botocore/httpsession.py", line 495, in send
INFO enclave > raise ProxyConnectionError(
INFO enclave > botocore.exceptions.ProxyConnectionError: Failed to connect to proxy URL: "http://127.0.0.1:10000/"
INFO enclave > 127.0.0.1 - - [28/Sep/2024 18:53:22] "POST /config HTTP/1.1" 500 -
@eyakubovich
Copy link
Contributor

Your allow list for S3 is missing a region, I think:

- s3.amazonaws.com

I think the S3 addresses are of the form s3.REGION.amazonaws.com. Just change it to s3.*.amazonaws.com

@robszumski robszumski mentioned this issue Sep 30, 2024
Merged
@robszumski
Copy link
Contributor

Whoops didn't intend to close this issue, but let us know if that fixes your issue.

@pavankad
Copy link
Author

pavankad commented Oct 1, 2024

thanks for responding. Changing S3 address to "s3.*.amazonaws.com" did not help. I get the same error as before. Is it trying to connect to S3 proxy listening on 10000 on host ec2 instance?

[ec2-user@ip-172-31-16-159 aws_depa_training]$ sudo enclaver run --publish 8000:8001 depa:enclave-latest
INFO enclaver::run > starting egress proxy on vsock port 17002
INFO enclaver::vsock > Listening on vsock port 17002
INFO enclaver::run > starting enclave
INFO enclaver::run > started enclave i-03731652486c31676-enc192478cd0b21d3f
INFO enclaver::run > starting ingress proxy on port 8001
INFO enclaver::run > waiting for enclave to boot to stream logs
INFO enclaver::run > connected to enclave, starting log stream
INFO enclave > INFO enclaver::vsock > Listening on vsock port 17001
INFO enclave > INFO odyn::enclave > Bringing up loopback interface
INFO enclave > INFO odyn::enclave > Seeding /dev/random with entropy from nsm device
INFO enclave > INFO odyn > Enclave initialized
INFO enclave > INFO odyn::egress > Starting egress
INFO enclave > INFO odyn::ingress > Starting TCP ingress on port 8001
INFO enclave > INFO enclaver::vsock > Listening on vsock port 8001
INFO enclave > INFO odyn::kms_proxy > Starting KMS proxy
INFO enclave > INFO odyn::kms_proxy > Generating public/private keypair
INFO enclave > INFO odyn::kms_proxy > Fetching credentials from IMDSv2
INFO enclave > INFO odyn::kms_proxy > Credentials fetched
INFO enclave > INFO odyn > Starting ["python3", "-m", "flask", "run", "--host=0.0.0.0", "--port=8001"]
INFO enclave > * Serving Flask app '/app/server.py'
INFO enclave > * Debug mode: off
INFO enclave > WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
INFO enclave > * Running on all addresses (0.0.0.0)
INFO enclave > * Running on http://127.0.0.1:8001
INFO enclave > * Running on http://127.0.0.1:8001
INFO enclave > Press CTRL+C to quit

Here are the open ports, I don't find 10000, or even vsock ports 17002

[ec2-user@ip-172-31-16-159 aws_depa_training]$ sudo lsof -nP -iTCP -sTCP:LISTEN
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
rpcbind 2160 rpc 8u IPv4 16237 0t0 TCP *:111 (LISTEN)
rpcbind 2160 rpc 11u IPv6 16240 0t0 TCP *:111 (LISTEN)
master 2621 root 13u IPv4 15135 0t0 TCP 127.0.0.1:25 (LISTEN)
sshd 2808 root 3u IPv4 19769 0t0 TCP *:22 (LISTEN)
sshd 2808 root 4u IPv6 19771 0t0 TCP *:22 (LISTEN)
container 9479 root 11u IPv4 2191916 0t0 TCP 127.0.0.1:40533 (LISTEN)

@eyakubovich
Copy link
Contributor

port 10000 is a TCP port inside the enclave. It gets advertised as http_proxy env variable.
Your first log excerpt contained "401 Unauthorized" but this one does not. Are you still seeing it? I'm not seeing any errors in the last log you posted.

There is a vsock listening on the host but it's inside of a container (enclaver run runs a docker container) so it's hard see with regular tooling.

@eyakubovich eyakubovich reopened this Oct 1, 2024
@pavankad
Copy link
Author

pavankad commented Oct 1, 2024

thanks for those answers. Sorry I missed the logs, Following is the log. It is the same error

[ec2-user@ip-172-31-16-159 aws_depa_training]$ sudo enclaver run --publish 8000:8001 depa:enclave-latest
INFO enclaver::run > starting egress proxy on vsock port 17002
INFO enclaver::vsock > Listening on vsock port 17002
INFO enclaver::run > starting enclave
INFO enclaver::run > started enclave i-03731652486c31676-enc192478cd0b21d3f
INFO enclaver::run > starting ingress proxy on port 8001
INFO enclaver::run > waiting for enclave to boot to stream logs
INFO enclaver::run > connected to enclave, starting log stream
INFO enclave > INFO enclaver::vsock > Listening on vsock port 17001
INFO enclave > INFO odyn::enclave > Bringing up loopback interface
INFO enclave > INFO odyn::enclave > Seeding /dev/random with entropy from nsm device
INFO enclave > INFO odyn > Enclave initialized
INFO enclave > INFO odyn::egress > Starting egress
INFO enclave > INFO odyn::ingress > Starting TCP ingress on port 8001
INFO enclave > INFO enclaver::vsock > Listening on vsock port 8001
INFO enclave > INFO odyn::kms_proxy > Starting KMS proxy
INFO enclave > INFO odyn::kms_proxy > Generating public/private keypair
INFO enclave > INFO odyn::kms_proxy > Fetching credentials from IMDSv2
INFO enclave > INFO odyn::kms_proxy > Credentials fetched
INFO enclave > INFO odyn > Starting ["python3", "-m", "flask", "run", "--host=0.0.0.0", "--port=8001"]
INFO enclave > * Serving Flask app '/app/server.py'
INFO enclave > * Debug mode: off
INFO enclave > WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
INFO enclave > * Running on all addresses (0.0.0.0)
INFO enclave > * Running on http://127.0.0.1:8001
INFO enclave > * Running on http://127.0.0.1:8001
INFO enclave > Press CTRL+C to quit

INFO enclave > Starting server...
INFO enclave > Here are params received>>>>>>>>
INFO enclave > Instance Profil name KMS_instance
INFO enclave > us-east-1

INFO enclave > Could not read file from S3
INFO enclave > Failed to connect to proxy URL: "http://127.0.0.1:10000/"
INFO enclave > [2024-10-01 10:08:10,050] ERROR in app: Exception on /config [POST]
INFO enclave > Traceback (most recent call last):
INFO enclave > File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 712, in urlopen
INFO enclave > self._prepare_proxy(conn)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 1014, in _prepare_proxy
INFO enclave > conn.connect()
INFO enclave > File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 374, in connect
INFO enclave > self._tunnel()
INFO enclave > File "/usr/local/lib/python3.8/http/client.py", line 905, in _tunnel
INFO enclave > raise OSError("Tunnel connection failed: %d %s" % (code,
INFO enclave > OSError: Tunnel connection failed: 401 Unauthorized
INFO enclave >
INFO enclave > During handling of the above exception, another exception occurred:
INFO enclave >
INFO enclave > Traceback (most recent call last):
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/httpsession.py", line 464, in send
INFO enclave > urllib_response = conn.urlopen(
INFO enclave > File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 801, in urlopen
INFO enclave > retries = retries.increment(
INFO enclave > File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 527, in increment
INFO enclave > raise six.reraise(type(error), error, _stacktrace)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/urllib3/packages/six.py", line 769, in reraise
INFO enclave > raise value.with_traceback(tb)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 712, in urlopen
INFO enclave > self._prepare_proxy(conn)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 1014, in _prepare_proxy
INFO enclave > conn.connect()
INFO enclave > File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 374, in connect
INFO enclave > self._tunnel()
INFO enclave > File "/usr/local/lib/python3.8/http/client.py", line 905, in _tunnel
INFO enclave > raise OSError("Tunnel connection failed: %d %s" % (code,
INFO enclave > urllib3.exceptions.ProxyError: ('Cannot connect to proxy.', OSError('Tunnel connection failed: 401 Unauthorized'))
INFO enclave >
INFO enclave > During handling of the above exception, another exception occurred:
INFO enclave >
INFO enclave > Traceback (most recent call last):
INFO enclave > File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2529, in wsgi_app
INFO enclave > response = self.full_dispatch_request()
INFO enclave > File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1825, in full_dispatch_request
INFO enclave > rv = self.handle_user_exception(e)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1823, in full_dispatch_request
INFO enclave > rv = self.dispatch_request()
INFO enclave > File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1799, in dispatch_request
INFO enclave > return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
INFO enclave > File "/app/server.py", line 186, in get_config
INFO enclave > encrypted_data_from_s3 = read_file_from_s3(s3_client, s3_bucket, filename)
INFO enclave > File "/app/server.py", line 142, in read_file_from_s3
INFO enclave > response = s3_client.get_object(Bucket=bucket_name, Key=file_key)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/client.py", line 553, in _api_call
INFO enclave > return self._make_api_call(operation_name, kwargs)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/client.py", line 989, in _make_api_call
INFO enclave > http, parsed_response = self._make_request(
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/client.py", line 1015, in _make_request
INFO enclave > return self._endpoint.make_request(operation_model, request_dict)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/endpoint.py", line 119, in make_request
INFO enclave > return self._send_request(request_dict, operation_model)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/endpoint.py", line 202, in _send_request
INFO enclave > while self._needs_retry(
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/endpoint.py", line 354, in _needs_retry
INFO enclave > responses = self._event_emitter.emit(
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/hooks.py", line 412, in emit
INFO enclave > return self._emitter.emit(aliased_event_name, **kwargs)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/hooks.py", line 256, in emit
INFO enclave > return self._emit(event_name, kwargs)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/hooks.py", line 239, in _emit
INFO enclave > response = handler(**kwargs)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/retryhandler.py", line 207, in call
INFO enclave > if self._checker(**checker_kwargs):
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/retryhandler.py", line 284, in call
INFO enclave > should_retry = self._should_retry(
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/retryhandler.py", line 320, in _should_retry
INFO enclave > return self._checker(attempt_number, response, caught_exception)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/retryhandler.py", line 363, in call
INFO enclave > checker_response = checker(
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/retryhandler.py", line 247, in call
INFO enclave > return self._check_caught_exception(
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/retryhandler.py", line 416, in _check_caught_exception
INFO enclave > raise caught_exception
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/endpoint.py", line 281, in _do_get_response
INFO enclave > http_response = self._send(request)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/endpoint.py", line 377, in _send
INFO enclave > return self.http_session.send(request)
INFO enclave > File "/usr/local/lib/python3.8/site-packages/botocore/httpsession.py", line 495, in send
INFO enclave > raise ProxyConnectionError(
INFO enclave > botocore.exceptions.ProxyConnectionError: Failed to connect to proxy URL: "http://127.0.0.1:10000/"
INFO enclave > 127.0.0.1 - - [01/Oct/2024 10:08:10] "POST /config HTTP/1.1" 500 -

@pavankad
Copy link
Author

pavankad commented Oct 2, 2024 •
edited
Loading

Following is the code to access S3 bucket. S3 bucket has public access, Also code works running in Docker. It is simple code, 1. get data key from KMS, 2.read pre-encrypted file in S3 bucket, 3. decrypt the file using data key fetched from KMS and do processing on the file.

I have tested the code accessing google website, that worked too, which means egress proxy works. But, not sure why it is not working for S3.

    try:
        # Produce attestion and get data key from KMS 
        kms_client = get_kms_client(region, credentials['access_key_id'], credentials['secret_access_key'], credentials['token'])
        response = kms_client.decrypt(KeyId=KMS_KEY1_ID, CiphertextBlob=base64.b64decode(data_key))
        plain_text = base64.b64encode(response['Plaintext'])
        response = {'key':plain_text}
        print(response)
    except Exception as err:
        print("Could not decrypt key")
        raise


    # Step 2: Read the encrypted file from S3
    try:
        print(region)
        #s3_client = get_s3_client(region, credentials['access_key_id'], credentials['secret_access_key'], credentials['token'])
        #encrypted_data_from_s3 = read_file_from_s3(s3_client, s3_bucket, filename)
        s3 = boto3.resource('s3')
        obj = s3.Object(s3_bucket, filename)
        encrypted_data_from_s3 = obj.get()['Body'].read()
        #url = 'http://www.google.com'
        #resp = requests.get(url)
        #print(resp.text) # Printing response



    except Exception as err:
        print("Could not read file from S3")
        print(str(err))
        raise

    # Step 3: Decrypt the file data
    try:
        decrypted_data = decrypt_data(plain_text, encrypted_data_from_s3)
        print(decrypted_data)
    except Exception as err:
        print("could not decrypt S3 data")
        raise

Here is the enclaver.yaml

version: v1
name: "depa"
target: "depa:enclave-latest"
sources:
app: "depa:latest"
defaults:
memory_mb: 2048
kms_proxy:
listen_port: 9999
egress:
allow:
- kms..amazonaws.com
- s3..amazonaws.com
- 169.254.169.254
- www.google.com
ingress:

  • listen_port: 8001

BTW, under egress:allow: S3 has "s3.*.amazonaws.com", not showing above.

Another question is
INFO enclave > raise OSError("Tunnel connection failed: %d %s" % (code,
INFO enclave > OSError: Tunnel connection failed: 401 Unauthorized

what does this mean, is it tunnel connection b/w enclave and host?

@robszumski
Copy link
Contributor

robszumski commented Oct 3, 2024 •
edited
Loading

Still debugging on our side, but try with '**.amazonaws.com' which is a little relaxed, but should get you a demo that works.

egress:
  allow:
    - kms.*.amazonaws.com
    - '**.amazonaws.com'
    - 169.254.169.254

@pavankad
Copy link
Author

pavankad commented Oct 4, 2024

it works with the above change, thanks so much @robszumski . For now, I can get the data coming in and going out. I will wait for your update to enable this specific to S3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update enclaver.yaml for any S3 region edgebitio/no-fly-list
3 participants
@robszumski @eyakubovich @pavankad