Update jcsda emc spack stack from authoritative 20220404 #50
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add compiler hint to the root spec for Windows Reporters on Windows (spack#26038) Reporters use Jinja2 as the templating engine, and Jinja2 indexes templates by Unix separators, even on Windows, so search using Unix paths on all systems. Support patching on win via git (spack#25871) Handle GRP on windows
Fixup common tests
* Remove requirement for Python 2.6
* Skip new failing test
Windows: Update url util to handle Windows paths (spack#27959)
* update url util to handle windows paths
* Update tests to handle fixed url handling
* canonicalize path only when the path type matches the host platform
* Skip some url tests on Windows
Co-authored-by: Omar Padron <[email protected]>
Use threading.TIMEOUT_MAX when available (spack#24246)
This value was introduced in Python 3.2. Specifying a timeout greater than
this value will raise an OverflowError.
Co-authored-by: Lou Lawrence <[email protected]>
Co-authored-by: John Parent <[email protected]>
Co-authored-by: Betsy McPhail <[email protected]>
* Incorporate new search location * Add external user option * proper doc string * Explicit commands in getting started * raise during chgrp on Win recover installer changes Notate admin privleges Windows phase install hooks Find external python and install ninja (spack#23496) Allow external find python to find windows python and spack install ninja Co-authored-by: Adam J. Stewart <[email protected]> Co-authored-by: Betsy McPhail <[email protected]>
Consolidate Spack's internal filepath logic to a select few places and refactor to consistent internal useage of os.path utilities. Creates a prefix, and a series of utilities in the path utility module that facilitate handling paths in a platform agnostic manner. Convert Windows paths to posix paths internally Prefer posixpath.join instead of os.path.join Updated util/ directory to account for Windows integration Co-authored-by: Stephen Crowell <[email protected]> Co-authored-by: John Parent <[email protected]> Module template format for windows (spack#23041)
Consistency is restored on next transaction
Broaden support for execution of the test suite on Windows. General bug and review fixups
Include support for powershell Prepend cmd and pwsh with [spack] to denote spack enabled shell
* [py-jaconv] created template * [py-jaconv] - added homepage - added description - depends on setuptools - removed fixmes
* Make boost composable Currently Boost enables a few components through variants by default, which means that if you want to use only what you need and no more, you have to explicitly disable these variants, leading to concretization errors whenever a second package explicitly needs those components. For instance if package A only needs `+component_a` it might depend on `boost +component_a ~component_b`. And if packge B only needs `+component_b` it might depend on `boost ~component_a +component_b`. If package C now depends on both A and B, this leads to unsatisfiable variants and hence a concretization error. However, if we default to disabling all components, package A can simply depend on `boost +component_a` and package B on `boost +component_b` and package C will concretize to depending on `boost +component_a +component_b`, and whatever you install, you get the bare minimum. * Fix style * Added composable boost dependencies for folly * fixing akantu merge issue * hpctoolkit boost dependencies already defined * Fix Styles * Fixup style once more * Adding isort fix * isort one more time * Fix for package audit issue Co-authored-by: Harmen Stoppels <[email protected]> Co-authored-by: Ryan O'Malley <[email protected]>
This PR removes a few outdated sections from the "Basics" part of the documentation. It also makes a few topic under the environment section more prominent by removing an unneeded spack.yaml subsection and promoting everything under it.
* Use same cxx value as root * Remove pointer syntax from non-pointer type in source * Run patch function before build * Use raw string in filter_file and merge edit function with patch * Escape parentheses * Use gDirectory from ROOT instead of CurrentDirectory function
Adds `spack external read-cray-manifest`, which reads a json file that describes a set of package DAGs. The parsed results are stored directly in the database. A user can see these installed specs with `spack find` (like any installed spec). The easiest way to use them right now as dependencies is to run `spack spec ... ^/hash-of-external-package`. Changes include: * `spack external read-cray-manifest --file <path/to/file>` will add all specs described in the file to Spack's installation DB and will also install described compilers to the compilers configuration (the expected format of the file is described in this PR as well including examples of the file) * Database records now may include an "origin" (the command added in this PR registers the origin as "external-db"). In the future, it is assumed users may want to be able to treat installs registered with this command differently (e.g. they may want to uninstall all specs added with this command) * Hash properties are now always preserved when copying specs if the source spec is concrete * I don't think the hashes of installed-and-concrete specs should change and this was the easiest way to handle that * also specs that are concrete preserve their `.normal` property when copied (external specs may mention compilers that are not registered, and without this change they would fail in `normalize` when calling `validate_or_raise`) * it might be this should only be the case if the spec was installed - [x] Improve testing - [x] Specifically mark DB records added with this command (so that users can do something like "uninstall all packages added with `spack read-external-db`) * This is now possible with `spack uninstall --all --origin=external-db` (this will remove all specs added from manifest files) - [x] Strip variants that are listed in json entries but don't actually exist for the package Co-authored-by: Harmen Stoppels <[email protected]>
* lower priority of package-provided urls This change favors urls found in a scraped page over those provided by the package from `url_for_version`. In most cases this doesn't matter, but R specifically returns known bad URLs in some cases, and the fallback path for a failed fetch uses `fetch_remote_versions` to find a substitute. This fixes that problem. fixes spack#29204 * consider what links actually exist in all cases Checksum was only actually scraping when called with no versions. It now always scrapes and then selects URLs from the set of URLs known to exist whenever possible. fixes spack#25831 * bow to the wrath of flake8 * test-fetch urls from package, prefer if successful * Update lib/spack/spack/package.py Co-authored-by: Seth R. Johnson <[email protected]> * reword as suggested * re-enable mypy specific ignore and ignore pyflakes * remove flake8 ignore from .flake8 * address review comments * address comments * add sneaky missing substitute I missed this one because we call substitute on a URL that doesn't contain a version component. I'm not sure how that's supposed to work, but apparently it's required by at least one mock package, so back in it goes. Co-authored-by: Seth R. Johnson <[email protected]>
- these versions are a synchronization with the OpenCL v3.0.10 specification release.
A new release hasn't been tagged in over a year.
Also add options to build documentation and Python support.
* tests for rewiring pure specs to spliced specs * relocate text, binaries, and links * using llnl.util.symlink for windows compat. Note: This does not include CLI hooks for relocation. Co-authored-by: Nathan Hanford <[email protected]>
* Add space after multi-value variant and before compiler spec * Check that spack_compiler.cc exists before passing to os.path.dirname to avoid external find error because of "[expected str, bytes or os.PathLike object, not NoneType]" MPICH and OpenMPI share the same logic for these and these fixes have already been applied to MPICH. See: spack#29284
climbfuji
force-pushed
the
feature/update_jcsda_emc_spack_stack_from_authoritative_20220404
branch
from
df43156
to
4d5f0ef
Compare
…pack into feature/update_jcsda_emc_spack_stack_from_authoritative_20220404
…pack into feature/update_jcsda_emc_spack_stack_from_authoritative_20220404
climbfuji
force-pushed
the
feature/update_jcsda_emc_spack_stack_from_authoritative_20220404
branch
from
f12d0fe
to
493faae
Compare
|
@kgerheiser This PR is ready for review and merging. Apart from the changes in the two commits mentioned in the description above, the updates here are all from the authoritative spack repo. |
Sigh. Just when I wrote this the macos-gcc CI build failed ... https://github.com/NOAA-EMC/spack-stack/runs/5973199079?check_suite_focus=true - guess more changes need to go in. |
…_20220404' of https://github.com/climbfuji/spack into feature/update_jcsda_emc_spack_stack_from_authoritative_20220404
2 tasks
|
@kgerheiser Finally - ready for review. No more changes. |
kgerheiser
approved these changes
Apr 12, 2022
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR pulls in the updates from the authoritative spack repository as of April 4, 2022.
I resolved the conflicts and had to make the following changes:
py-scipy,netcdf-fortran,fftw(see commit f563930).url.py- see commit 493faae and the open issue in the authoritative repository spack buildcache not working with custom S3_ENDPOINT_URL spack/spack#29910.lib/spack/llnl/util/tty/color.py(commit 9a9b36b) because this didn't work as expected (had\[and\]all over the place when spack colorized output likespack info).[email protected]because of build problems (see Installation issue: py-scipy 1.8.0 on macOS with gcc+gfortran spack/spack#30022) - created an issue to remind us putting it back in: Re-enable [email protected] in the py-scipypackage.py#51This PR has been tested together with JCSDA/spack-stack#112, see this PR for more information.
Fixes #27