'Forbidden' Permission Error when attempting to access AWS radar data #1353
-
|
Per the request of @mgrover1. This occurs when running AWS data access example in the PyArt documentation. Error message: File ~\miniconda3\lib\site-packages\aiobotocore\client.py:358, in AioBaseClient._make_api_call(self, operation_name, api_params) ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden The above exception was the direct cause of the following exception: PermissionError Traceback (most recent call last) File ~\miniconda3\lib\site-packages\pyart\io\nexrad_archive.py:106, in read_nexrad_archive(filename, field_names, additional_metadata, file_field_names, exclude_fields, include_fields, delay_field_loading, station, scans, linear_interp, storage_options, **kwargs) File ~\miniconda3\lib\site-packages\pyart\io\common.py:42, in prepare_for_read(filename, storage_options) File ~\miniconda3\lib\site-packages\fsspec\core.py:135, in OpenFile.open(self) File ~\miniconda3\lib\site-packages\fsspec\core.py:103, in OpenFile.enter(self) File ~\miniconda3\lib\site-packages\fsspec\spec.py:1106, in AbstractFileSystem.open(self, path, mode, block_size, cache_options, compression, **kwargs) File ~\miniconda3\lib\site-packages\s3fs\core.py:640, in S3FileSystem._open(self, path, mode, block_size, acl, version_id, fill_cache, cache_type, autocommit, requester_pays, cache_options, **kwargs) File ~\miniconda3\lib\site-packages\s3fs\core.py:1989, in S3File.init(self, s3, path, mode, block_size, acl, version_id, fill_cache, s3_additional_kwargs, autocommit, cache_type, requester_pays, cache_options) File ~\miniconda3\lib\site-packages\fsspec\spec.py:1462, in AbstractBufferedFile.init(self, fs, path, mode, block_size, autocommit, cache_type, cache_options, size, **kwargs) File ~\miniconda3\lib\site-packages\fsspec\spec.py:1475, in AbstractBufferedFile.details(self) File ~\miniconda3\lib\site-packages\fsspec\asyn.py:113, in sync_wrapper..wrapper(*args, **kwargs) File ~\miniconda3\lib\site-packages\fsspec\asyn.py:98, in sync(loop, func, timeout, *args, **kwargs) File ~\miniconda3\lib\site-packages\fsspec\asyn.py:53, in _runner(event, coro, result, timeout) File ~\miniconda3\lib\site-packages\s3fs\core.py:1210, in S3FileSystem._info(self, path, bucket, key, refresh, version_id) File ~\miniconda3\lib\site-packages\s3fs\core.py:339, in S3FileSystem._call_s3(self, method, *akwarglist, **kwargs) File ~\miniconda3\lib\site-packages\s3fs\core.py:139, in _error_wrapper(func, args, kwargs, retries) PermissionError: Forbidden` #** CONDA LIST:** |
Beta Was this translation helpful? Give feedback.
Replies: 26 comments
-
|
@kylejgillett can you print out the file path you are sending to the reader? Ex "s3://..." |
Beta Was this translation helpful? Give feedback.
-
|
's3://noaa-nexrad-level2/2022/03/22/KHGX/KHGX20220322_120125_V06' |
Beta Was this translation helpful? Give feedback.
-
|
So to confirm here, the following throws the error? import pyart
radar = pyart.io.read_nexrad_archive('s3://noaa-nexrad-level2/2022/03/22/KHGX/KHGX20220322_120125_V06') |
Beta Was this translation helpful? Give feedback.
-
|
Yep, I get the error running that. |
Beta Was this translation helpful? Give feedback.
-
|
So it looks like there is an issue with the newer release of aiobotocore... can you try to see if that resolves the issue? that should reinstall working versions of the file packages. |
Beta Was this translation helpful? Give feedback.
-
|
Otherwise, try installing aiobotocore3 version 2.4.0 |
Beta Was this translation helpful? Give feedback.
-
|
So I uninstalled/reinstalled arm-pyart and got this message afterwards...
|
Beta Was this translation helpful? Give feedback.
-
|
can you install boto3 == 1.26.27 ? |
Beta Was this translation helpful? Give feedback.
-
|
So this is as far as I got trying to trouble shoot last night. It leaves me here: |
Beta Was this translation helpful? Give feedback.
-
|
Okay - yeah! So you will need |
Beta Was this translation helpful? Give feedback.
-
|
Its the newest release of aiobotocore3 that is causing some issues I think... so rolling back one version there should help? |
Beta Was this translation helpful? Give feedback.
-
|
is that the correct version of aiobotocore3? I am getting this: |
Beta Was this translation helpful? Give feedback.
-
|
ope sorry - aiobotocore is the name of the package, no 3 |
Beta Was this translation helpful? Give feedback.
-
|
Now this lol: |
Beta Was this translation helpful? Give feedback.
-
|
Can you try running that code snippet from above to see if this update fixed the issue? |
Beta Was this translation helpful? Give feedback.
-
This one! |
Beta Was this translation helpful? Give feedback.
-
|
I am still Forbidden LOL |
Beta Was this translation helpful? Give feedback.
-
|
Can you run conda list one more time? I can take a another look likely on Monday and dig into this more... |
Beta Was this translation helpful? Give feedback.
-
|
This is what I am running on my machine that is working: In [1]: import aiobotocore
In [2]: aiobotocore.__version__
Out[2]: '2.4.0'
In [3]: import boto3
In [4]: boto3.__version__
Out[4]: '1.26.27'
In [5]: import botocore
In [6]: botocore.__version__
Out[6]: '1.27.59' |
Beta Was this translation helpful? Give feedback.
-
|
Yep! |
Beta Was this translation helpful? Give feedback.
-
|
Mine is: 2.4.1 |
Beta Was this translation helpful? Give feedback.
-
|
So there is a mismatch in the environment you listed there, and the environment you are running your notebook or kernel? That first one should be 2.4.0 according to your environment listing |
Beta Was this translation helpful? Give feedback.
-
|
You would need to restart the kernel in order for the environment to update where you are executing the code |
Beta Was this translation helpful? Give feedback.
-
|
Ha, smh, duh. It is running now! Thanks a bunch, Max! |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for letting us know about this issue! |
Beta Was this translation helpful? Give feedback.
Okay - yeah! So you will need