Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix #1469 Use a different checksum calculation method to run in FIPS env #1473

Merged
merged 1 commit into from
Nov 28, 2024
Merged

Fix #1469 Use a different checksum calculation method to run in FIPS env #1473

merged 1 commit into from
Nov 28, 2024

Conversation

markesha
Copy link
Contributor

Python 3.10 and later versions rely on OpenSSL 1.1.1 or newer, which includes FIPS-compliance checks.

MD5 is not an approved algorithm in FIPS mode, so attempting to instantiate self.blob.download_to_file(self._file) will fail when the system is running in FIPS mode.

The change configures the download_to_file function to use an alternative algorithm provided by gcloud storage SDK - 'crc32c' - for checksum calculation.

@jschneier
Copy link
Owner

Just making sure, this change is backwards compatible, right?

@markesha markesha force-pushed the gcloud_md5_fips_fix branch 4 times, most recently from 8dd91da to 2578499 Compare November 18, 2024 10:51
@markesha
Copy link
Contributor Author

@jschneier the change depends on a specific version of the google storage lib, that I'm locking with the change. All the test python versions were already using the latest one 2.18.2, e.g. see 3.7 logs:

py3.7-django3.2: apache-libcloud==3.8.0,asgiref==3.7.2,azure-core==1.30.1,azure-storage-blob==12.19.1,bcrypt==4.2.0,boto3==1.33.13,botocore==1.33.13,cachetools==5.5.0,certifi==2024.8.30,cffi==1.15.1,charset-normalizer==3.4.0,coverage==7.2.7,cryptography==43.0.3,Django==3.2.25,django-storages @ file:///home/runner/work/django-storages/django-storages/.tox/.tmp/package/1/django-storages-1.14.4.tar.gz#sha256=fd2920db03cfb63bf4d6f600ee0e6bdb668931505a261c3f0107cda7e5947bc8,dropbox==12.0.2,exceptiongroup==1.2.2,google-api-core==2.23.0,google-auth==2.36.0,google-cloud-core==2.4.1,google-cloud-storage==**2.18.2**,

I guess it should be compatible.

@markesha markesha force-pushed the gcloud_md5_fips_fix branch from 2578499 to 3c7a065 Compare November 19, 2024 10:00
@markesha
Copy link
Contributor Author

@jschneier the failing test tests/test_gcloud.py::GoogleCloudGzipClientTests::test_storage_save_gzipped appears unrelated to the change and also fails on earlier revisions.

 AssertionError: expected call not found.
 Expected: upload_from_file(<ANY>, rewind=True, retry=<google.api_core.retry.retry_unary.Retry object at 0x10dad5300>, size=11, predefined_acl=None, content_type='application/javascript')
 Actual: upload_from_file(<ContentFile: Raw content>, rewind=True, retry=<google.api_core.retry.retry_unary.Retry object at 0x10dad5300>, size=11, predefined_acl=None, content_type='text/javascript')

@jschneier
Copy link
Owner

@markesha any idea what's going on here? There are no failures now but the tests were never "fixed".

jschneier
jschneier requested changes Nov 22, 2024
View reviewed changes
Copy link
Owner

@jschneier jschneier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good although I don't understand the test reality.

pyproject.toml Outdated Show resolved Hide resolved
@markesha markesha force-pushed the gcloud_md5_fips_fix branch from 3c7a065 to 5736ede Compare November 22, 2024 09:15
@markesha
Copy link
Contributor Author

@jschneier I'm not sure what's happening - it still fails locally for me

@markesha markesha requested a review from jschneier November 22, 2024 09:20
@jschneier
Copy link
Owner

@jschneier I'm not sure what's happening - it still fails locally for me

Well, in #1476 I just change it to use CSS which should fix whatever weird mimetype problem we are having.

@markesha
Fix jschneier#1469 Use a different checksum calculation method to run…
ab961ba 
… in FIPS env

Python 3.10 and later versions rely on OpenSSL 1.1.1 or newer, which includes FIPS-compliance checks.

MD5 is not an approved algorithm in FIPS mode, so attempting to instantiate self.blob.download_to_file(self._file) will fail when the system is running in FIPS mode.

The change configures the `download_to_file` function to use an alternative algorithm provided by gcloud storage SDK - 'crc32c' - for checksum calculation.
Configurable checksumming is available in the google-storage lib since v1.31.0, but pinning to >=1.32 for the retry import.
@markesha markesha force-pushed the gcloud_md5_fips_fix branch from 5736ede to ab961ba Compare November 25, 2024 09:04
@jschneier jschneier linked an issue Nov 28, 2024 that may be closed by this pull request
Use a different checksum calculation method to run in FIPS enabled environment with python3.10+ #1469
Closed
@jschneier jschneier merged commit b3513ec into jschneier:master Nov 28, 2024
22 checks passed
@markesha markesha deleted the gcloud_md5_fips_fix branch November 28, 2024 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jschneier jschneier jschneier approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Use a different checksum calculation method to run in FIPS enabled environment with python3.10+
2 participants
@markesha @jschneier