Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(SE) Temporary folder handling #432

Closed
Dooruk opened this issue Sep 20, 2024 · 4 comments
Closed

(SE) Temporary folder handling #432

Dooruk opened this issue Sep 20, 2024 · 4 comments
Assignees
Labels
core development design related issues and improvements

Comments

@Dooruk
Copy link
Collaborator

Dooruk commented Sep 20, 2024

There are certain codetests that create templated YAMLs in /tmp, see below:

destination_yaml_temp = os.path.join('/tmp', f'task_questions_{random_string}.yaml')

This location is not reliably erased and NCCS doesn't want us to fill this up so we should have a better temporary file handling.

For codestests;
if everything works fine, they can be deleted.
in case of an error, they should be kept until the user no longer needs them, which could be when the terminal is closed?

@Dooruk Dooruk added the core development design related issues and improvements label Sep 20, 2024
@Dooruk
Copy link
Collaborator Author

Dooruk commented Sep 20, 2024

Also somewhat relevant discussion in the GEOSgcm context:

GEOS-ESM/GEOSgcm#814

@mathomp4
Copy link
Member

If nothing else, that code fragment should probably use the tempfile module which is the more Pythonic way of handling temporary files.

Of course, even that uses TMPDIR so we'd want to make sure TMPDIR was not /tmp.

ashiklom added a commit that referenced this issue Sep 30, 2024
Dooruk pushed a commit that referenced this issue Oct 1, 2024
* Remove hardcoded /tmp from code_tests

See #432

* Fix GEOS_mksi test

Since GEOS-ESM/GEOS_mksi#2 was merged, GEOS_mksi
main and develop are now in sync. This was causing the GEOS_mksi test
to fail.

This changes the test to point to a known (very old!) version of
GEOS_mksi that will cause an error, so the test actually passes.
@mranst
Copy link
Contributor

mranst commented Nov 14, 2024

This was addressed by #436 right? TMPDIR should be cleared at job close/exit. I believe that's the only instance of this

@Dooruk
Copy link
Collaborator Author

Dooruk commented Nov 17, 2024

This was addressed by #436 right? TMPDIR should be cleared at job close/exit. I believe that's the only instance of this

Yup closed by #436

@Dooruk Dooruk closed this as completed Nov 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core development design related issues and improvements
Projects
None yet
Development

No branches or pull requests

4 participants