forked from easybuilders/easybuild-framework
-
Notifications
You must be signed in to change notification settings - Fork 0
/
RELEASE_NOTES
2625 lines (2343 loc) · 168 KB
/
RELEASE_NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
This file contains a description of the major changes to the easybuild-framework EasyBuild package.
For more detailed information, please see the git log.
These release notes can also be consulted at https://easybuild.readthedocs.io/en/latest/Release_notes.html.
v4.6.0 (July 8th 2022)
----------------------
feature release
- various enhancements, including:
- allow searching for sources/patches in alternative location by specifying 'alt_location' in source/patch spec (#3994)
- show URLs used for download attempts in trace output (#4026)
- add support for setting environment variables via 'pushenv' with modextravars (#4030)
- add support for OneAPI compilers using toolchain option 'oneapi' (#4031, #4032, #4039)
- make check_linked_shared_libs more robust by taking into account that 'ldd' may fail (#4033)
- fall back to sequential extension install if parallel install is not implemented (#4034)
- add support for using template values in name/version of extensions (#4036)
- various bug fixes, including:
- make sure that ARCH constant has 'aarch64' (rather than 'arm64') as value on macOS ARM (#4018)
- tweak 'eb' wrapper script to correctly handle errors when python command being considered fails to run (#4019)
- tweak is_patch_for function to make it more robust (#4028)
- other changes:
- update Lmod used to run tests to version 8.7.6 (#4027, #4030)
- tweak apply_patch to not create .orig files (by default) when applying patch files (#4038)
v4.5.5 (June 8th 2022)
----------------------
update/bugfix release
- various enhancements, including:
- add toolchain definitions for nvompi (NVHPC + OpenMPI) (#3969), nvpsmpi (NVHPC + ParaStationMPI) (#3970), gfbf (GCC, FlexiBLAS, FFTW) (#4006)
- add support for FFTW.MPI toolchain component ($FFT*DIR variables) (#4012)
- add support for customizing EasyBuild command used in jobs via --job-eb-cmd (#4016)
- various bug fixes, including:
- fix copying of easyconfig file with --copy-ec without --rebuild if module is already installed (#3993)
- ignore deprecation warnings regarding cryptography in Python 3.6 + Blowfish with Python 3.10 in test suite output (#3999)
- fix typo in debug log message in easyblock.py (#4000)
- fix printing of list of attempted download URLs when url-encoded characters are used in URL (#4005)
- set $FFT(W)_LIB_DIR to imkl-FFTW's lib path in build environment if usempi toolchain option is enabled (#4011)
- correctly identify Apple Silicon M1 as Arm 64-bit by also considering arm64 next to aarch64 (#4014)
- fix 'eb --show-system-info' on Apple M1 system (#4015)
- other changes:
- change 'eb' command to import easybuild.framework to check if EasyBuild framework is available (#3995, #3998)
v4.5.4 (March 31st 2022)
------------------------
update/bugfix release
- various enhancements, including:
- warn about potentially missing patches in --new-pr (#3759, #3966)
- add support for 'clone_into' field in git_config source spec to specify different name for top-level directory (#3949)
- add bash completion for easyconfigs from local dir but not robot search path (#3953)
- add a 'sync pr' message when the PR has a mergeable state but is showing a failed status for the test suite on the last commit (#3967)
- add gmpit toolchain definition (GCC + MPItrampoline) (#3971)
- use 'zypper search -i' to check whether specified OS dependency is installed on openSUSE + make sure that rpm is considered for checking OS dependencies on RHEL8 (#3973)
- add support for post-install patches (#3974)
- add support for 'download_instructions' easyconfig parameter key to specify some download or installation steps for user in case of complicated way of obtaining needed files (#3976, #3981)
- also try collecting AMD GPU info (via rocm-smi) for --show-system-info (#3978, #3982)
- various bug fixes, including:
- ensure --review-pr can find dependencies included in PR (#3979)
- run 'apt-get update' in GitHub Actions workflow for container tests to update container package list before installing anything (#3985)
- other changes:
- enable code linting check for all supported Python versions (#3725)
- update copyright lines for 2022 (#3986)
v4.5.3 (February 11th 2022)
---------------------------
update/bugfix release
- various enhancements, including:
- also check for git in --check-eb-deps (#3954)
- add end2end test for 'eb --containerize' (#3958)
- various bug fixes, including:
- take into account that patch files can also be zipped when checking filename extension for patches (#3936)
- initialize BACKUP_PKG_URL with empty string in install_eb_dep.sh script (#3939)
- fix get_os_name and get_os_version to avoid reporting UNKNOWN in output of eb --show-system-info (#3942)
- consistently use actions/setup-python@v2 in CI workflows (#3944)
- switch to using pip3 for installing EasyBuild in container recipes generated by EasyBuild (#3945)
- specify easybuild.io as EasyBuild homepage in setup.py (#3947)
- avoid crash in get_os_version on modern SLES-based OSs (#3955)
- other changes:
- indicate compatibility with Python 3.10 in setup.py (#3940)
v4.5.2 (January 24th 2022)
--------------------------
update/bugfix release
- various enhancements, including:
- automatically prepend value for env-for-shebang configuration setting with sysroot (#3919)
- enhance update_build_option to return original value + add support for update_build_options (#3923)
- fix compatibility with Python 3.10 (#3926, #3933)
- also take /etc/os-release into account in get_os_name and get_os_version (#3930)
- enhance get_cpu_architecture and get_cpu_family to be aware of RISC-V (#3931)
- various bug fixes, including:
- relax pattern checks in test_toy_exts_parallel (#3921)
- use sources.easybuild.io as backup URL for downloading Environment Modules sources to run test suite (#3928)
- make intelfftw toolchain component aware of GCCcore (#3929)
- other changes:
- deprecate use of (actual) patch files that don't have a filename ending with .patch (#3920)
- remove version restriction for keyring in requirements.txt to test with latest version (#3925)
- simplify requirements.txt by removing Python 2.6 support (#3927)
- only run GitHub tests when testing with Lua module syntax, to avoid hitting GitHub rate limit when running tests (#3938)
v4.5.1 (December 13th 2021)
---------------------------
update/bugfix release
- various enhancements, including:
- also dump environment to reprod directory (#3374)
- determine which extensions can be skipped in parallel (if --parallel-extensions-install is enabled) (#3890)
- fall back to sequential installation for extensions with unknown dependencies when using --parallel-extensions-install (#3906)
- allow oversubscription in sanity check for OpenMPI-based toolchains (#3909)
- various bug fixes, including:
- don't try to ensure absolute path for path part of repositorypath (#3893, #3899)
- fix typo in EULA agreement error message (#3894)
- only remove lock if it was created in the same EasyBuild session (not if it existed already) (#3889)
- introduce EasyBlock.post_init method to correctly define builddir variable when build-in-installdir mode is enabled in easyconfig or easyblock (#3900)
- also show download progress bar when using --inject-checksums (#3905)
- pick up custom extract_cmd specified for extension (#3907)
- make test_run_cmd_async more robust against fluke failures (#3908)
v4.5.0 (October 29th 2021)
--------------------------
feature release
- various enhancements, including:
- add --review-pr-max and --review-pr-filter options to limit easyconfigs used by --review-pr + retain order of easyconfigs being compared with (#3754)
- use Rich (if available) to show progress bars when installing easyconfigs (#3823, #3826, #3833, #3835, #3844, #3864, #3867, #3882)
- see also https://docs.easybuild.io/en/latest/Progress_bars.html
- add support for checking required/optional EasyBuild dependencies via 'eb --check-eb-deps' (#3829)
- add support for collecting GPU info (via nvidia-smi), and include it in --show-system-info and test report (#3851)
- added support for --insecure-download configuration option (#3859)
- make intelfftw toolchain component aware of imkl-FFTW module (#3861)
- make sure the contrib/hooks tree is included in the source tarball for easybuild-framework (#3862)
- add check_async_cmd function to facilitate checking on asynchronously running commands (#3865, #3881)
- add initial/experimental support for installing extensions in parallel (#3667, #3878)
- see also https://docs.easybuild.io/en/latest/Installing_extensions_in_parallel.html
- filter out duplicate paths added to module files, and print warning when they occur (#3770, #3874)
- various bug fixes, including:
- ensure that path configuration options have absolute path values (#3832)
- fix broken test by taking into account changed error raised by Python 3.9.7 when copying directory via shutil.copyfile (#3840)
- ensure newer location of CUDA stubs is taken into account by RPATH filter (#3850)
- replace 'which' by 'command -v' in 'eb' wrapper script to avoid dependency on 'which' command (#3852)
- refactor EasyBlock to decouple collecting of information on extension source/patch files from downloading them (#3860)
- in this context, the EasyBlock.fetch_extension_sources method was deprecated, and replaced by EasyBlock.collect_exts_file_info
- fix library paths to add to $LDFLAGS for intel-compilers toolchain component (#3866)
- remove '--depth 1' from git clone when 'commit' specified (#3871, #3872)
- make sure correct include directory is used for FlexiBLAS toolchain component (#3875)
- explictly disable rebase when pulling develop branch to create a merge commit, since not specifying how to reconcile divergent branches is an error with Git 2.33.1 and newer (#3879)
- tweak test_http_header_fields_urlpat to download from sources.easybuild.io rather than https://ftp.gnu.org (#3883)
- other changes:
- change copy_file function to raise an error when trying to copy non-existing file (#3836, #3855, #3877)
- only print the hook messages if EasyBuild is running in debug mode (#3843)
- deprecate old toolchain versions (pre-2019a common toolchains) (#3876, #3884)
- see also https://docs.easybuild.io/en/latest/Deprecated-easyconfigs.html#deprecated-toolchains
v4.4.2 (September 7th 2021)
---------------------------
update/bugfix release
- various enhancements, including:
- add per-extension timing in output produced by eb command (#3734)
- add definition for new toolchain nvpsmpic (NVHPC + ParaStationMPI + CUDA) (#3736)
- include list of missing libraries in warning about missing FFTW libraries in imkl toolchain component (#3776)
- check for recursive symlinks by default before copying a folder (#3784)
- add --filter-ecs configuration option to filter out easyconfigs from set of easyconfigs to install (#3796)
- check type of source_tmpl value for extensions, ensure it's a string value (not a list) (#3799)
- also define $BLAS_SHARED_LIBS & co in build environment (analogous to $BLAS_STATIC_LIBS) (#3800)
- report use of --ignore-test-failure in success message in output (#3806)
- add get_cuda_cc_template_value method to EasyConfig class (#3807)
- add support for fix_bash_shebang_for (#3808)
- pick up $MODULES_CMD to facilitate using Environment Modules 4.x as modules tool (#3816)
- use more sensible branch name for creating easyblocks PR with --new-pr (#3817)
- various bug fixes, including:
- remove Python 2.6 from list of supported Python versions in setup.py (#3767)
- don't add directory that doesn't include any files to $PATH or $LD_LIBRARY_PATH (#3769)
- make logdir writable also when --stop/--fetch is used and --read-only-installdir is enabled (#3771)
- fix forgotten renaming of 'l' to 'char' __init__.py that is created for included Python modules (#3773)
- fix verify_imports by deleting all imported modules before re-importing them one by one (#3780)
- fix ignore_test_failure not set for Extension instances (#3782)
- update iompi toolchain to intel-compiler subtoolchain for oneAPI versions (>= iompi 2020.12) (#3785)
- don't parse patch files as easyconfigs when searching for where patch file is used (#3786)
- make sure git clone with a tag argument actually downloads a tag (#3795)
- fix CI by excluding GC3Pie 2.6.7 (which is broken with Python 2) and improve error reporting for option parsing (#3798)
- correctly resolve templates for patches in extensions when uploading to GitHub (#3805)
- add --easystack to ignored options when submitting job (#3813)
- other changes:
- speed up tests by caching checked paths in set_tmpdir + less test cases for test_compiler_dependent_optarch (#3802)
- speed up set_parallel method in EasyBlock class (#3812)
v4.4.1 (July 6th 2021)
----------------------
update/bugfix release
- various enhancements, including:
- enhance detection of patch files supplied to 'eb' command with better error messages (#3709)
- add per-step timing information (#3716)
- add module-write hook (#3728)
- add ignore-test-failure configuration option to ignore failing test step (#3732)
- add toolchain definition for nvompic (NVHPC + OpenMPI) (#3735)
- warn about generic milestone in --review-pr and --merge-pr (#3751)
- various bug fixes, including:
- don't override COMPILER_MODULE_NAME, inherited from Ffmpi, in Fujitsu toolchain definition (#3721)
- avoid overwritting pr_nr in post_pr_test_report for reports with --include-easyblocks-from-pr (#3724, #3726)
- fix crash in get_config_dict when copying modules that were imported in easyconfig file (like 'import os') (#3729)
- parse C standard flags in CFLAGS for Fujitsu compiler (#3731)
- fix error message for --use-ccache (#3733)
- error out when passing a list to run_cmd* to avoid running wrong command unintended, unless shell=True is used (#3737)
- minor fixes to output of test reports when using multiple PRs (#3741)
- fix location for modules installed with intel-compilers toolchain in HierarchicalMNS by always checking toolchain compiler name against template map (#3745)
- avoid checking msg attribute of GitCommandError (#3756)
- consider sources provided as dict in EasyBlock.check_checksums_for (#3758)
- don't make changes to software installation directory when using --sanity-check-only (#3761)
- honor specified easyblock for extensions (#3762)
- other changes:
- make sure that tests requiring a github token have 'github' in the test name so that they can be easily filtered (#3730)
- deprecate EasyBuild bootstrap script (#3742)
- use temporary download folder in test_http_header_fields_urlpat (#3755)
v4.4.0 (June 2nd 2021)
----------------------
feature release
- various enhancements, including:
- enhance apply_regex_substitutions to allow specifying action to take in case there are no matches (#3440)
- performance improvements for easyconfig parsing and eb startup (#3555)
- add support for downloading easyconfigs from multiple PRs with --from-pr (#3605, #3707, #3708)
- add support for prepending custom library paths in RPATH section via --rpath-override-dirs (#3650)
- allow amending easyconfig parameters which are not the default (#3651)
- update HierarchicalMNS for Intel OneAPI compilers (#3653)
- add support for --sanity-check-only (#3655)
- add support for running commands asynchronously via run_cmd + complete_cmd functions (#3656)
- add support for using oneAPI versions of 'intel' toolchain components (#3665)
- add toolchain definition for gofbf (foss with FlexiBLAS rather than OpenBLAS) (#3666)
- add support for using ARCH constant in easyconfig files (#3670)
- honor keyboard interrupt in 'eb' command (#3674)
- add toolchain definition for Fujitsu toolchain (#3677, #3704, #3712, #3713, #3714, #3717)
- extend sanity check step to check whether specific libraries are not linked into installed binaries/libraries (#3678)
- via banned-linked-shared-libs and required-linked-shared-libs EasyBuild configuration options
- via banned_linked_shared_libs and required_linked_shared_libs methods in toolchain support
- via banned_linked_shared_libs and required_linked_shared_libs methods in easyblock
- via banned_linked_shared_libs and required_linked_shared_libs easyconfig parameters
- add locate_solib function to locate Linux shared libraries (#3682)
- add system agnostic function to locate shared libraries (#3683)
- add update_build_option function to update specific build options after initializing the EasyBuild configuration (#3684)
- allow opting out of recursively unloaded of modules via recursive_module_unload easyconfig parameter (#3689)
- check for correct version values when parsing easystack file (#3693)
- run post-install commands specified for a specific extension (#3696)
- add support for --skip-extensions (#3702)
- include PR title in output produced by --merge-pr (#3706)
- various bug fixes, including:
- avoid metadata greedy behaviour when probing for external module metadata (mostly relevant for integration with Cray Programming Environment) (#3559)
- catch problems early on if --github-user is not specified for --new-pr & co (#3644)
- re-enable write permissions when installing with read-only-installdir (#3649)
- also run sanity check for extensions when using --module-only (#3655)
- improve logging when failing to load class from exts_classmap (#3657)
- fix use of --module-only on existing installations without write permissions (#3659)
- correct help text for subdir-user-modules (#3660)
- avoid picking up easyblocks outside of sandbox in framework tests (#3680)
- use unload/load in ModuleGeneratorLua.swap_module, since 'swap' is not supported by Lmod (#3685)
- update HierarchicalMNS to also return 'Toolchain/<name>/<version>' as $MODULEPATH extension for cpe* Cray toolchains (#3686)
- make EasyConfigParser.get_config_dict return a copy rather than a reference (#3692)
- make sure that $TAPE is unset when using piped tar (#3698)
- fix extending message for changed files in new_pr_from_branch (#3699)
- enhance sched_getaffinity function to avoid early crash when counting available cores on systems with more than 1024 cores (#3701)
- correctly strip extension from filename in extract_cmd and back_up_file functions (#3705)
- other changes:
- deprecate adding a non-existing path to $MODULEPATH (#3637)
- bump cryptography requirement from 3.2.1 to 3.3.2 (#3643, #3648)
- test bootstrap script in separate workflow, and limit test configurations a bit (#3646)
- update setup.py to indicate compatibility with Python 3.8 and 3.9 (#3647)
- replace log_error parameter of which() by on_error (#3661, #3664)
- don't skip sanity check for --module-only --rebuild (#3645)
- move disable_templating function into the EasyConfig class (#3668)
- pin GitPython version for Python<3.6, don't test bootstrap script against Python 3.8/3.9 (#3675)
- tweak foss toolchain definition to switch from OpenBLAS to FlexiBLAS in foss/2021a (#3679)
- suggest missing SSH key when not able to read from remote repository in --check-github (#3681)
- drop support for Python 2.6 (#3715)
v4.3.4 (April 9th 2021)
-----------------------
update/bugfix release
- various enhancements, including:
- add support for filtering dependencies by using False as version (#3506)
- add create_unused_dir function to create a directory which does not yet exist (#3551)
- avoid running expensive 'module use' and 'module unuse' commands when using Lmod as modules tool, update $MODULEPATH directly instead (#3557, #3633)
- create CUDA cache (for JIT compiled PTX code) in build dir instead of $HOME (#3569)
- add "Citing" section to module files (#3596)
- add support for using fallback 'arch=*' key in dependency version specified as arch->version mapping (#3600)
- also check for pending change requests and mergeable_state in check_pr_eligible_to_merge (#3604)
- ignore undismissed 'changes requested' review if there is an 'approved' review by the same user (#3607, #3608)
- sort output of 'eb --search' in natural order (respecting numbers) (#3609)
- enhance 'eb' command to ensure that easybuild.main can be imported before settling on python* command to use (#3610)
- add --env-for-shebang configuration option to define the env command to use for shebangs (#3613)
- add templates for architecture independent Python wheels (#3618)
- mention easyblocks PR in gist when uploading test report for it + fix clean_gists.py script (#3622)
- also accept regular expression value for --accept-eula-for (#3630)
- update validate_github_token function to accept GitHub token in new format (#3632)
- various bug fixes, including:
- fix $BLAS_LIB_MT for OpenBLAS, ensure -lpthread is included (#3584)
- use '--opt=val' for passing settings from config file to option parser to avoid error for values starting with '-' or '--' (#3594)
- avoid raised exception when getting output from interactive command in run_cmd_qa (#3599)
- add option to write file from file-like object and use in download_file (#3614)
- make sure that path to eb is always found by tests (#3617)
- other changes:
- add pick_default_branch function to clean up duplicate code in tools/github.py (#3592)
- refactor the CI configuration to use inclusion instead of exclusion (#3616)
- use develop branch when testing push access in --check-github (#3629)
- deprecate --accept-eula, rename to --accept-eula-for (#3630)
v4.3.3 (February 23rd 2021)
---------------------------
update/bugfix release
- various enhancements, including:
- advise PR labels in --review-pr and add support for --add-pr-labels (#3177)
- add support for using customized HTTP headers in download_file (#3472, #3583)
- also take toolchain dependencies into account when defining template values (#3541, #3560)
- add support for --accept-eula configuration option + 'accept_eula' easyconfig parameter (#3535, #3536, #3546)
- detect 'SYSTEM' toolchain as special case in easystack files (#3543)
- enhance extract_cmd function to use 'cp -a' for shell scripts (.sh) (#3545)
- allow use of alternate envvar(s) to $HOME for user modules (#3558)
- use https://sources.easybuild.io as fallback source URL (#3572, #3576)
- add toolchain definition for iibff toolchain (#3574)
- add %(cuda_cc_space_sep)s and %(cuda_cc_semicolon_sep)s templates (#3578)
- add support for intel-compiler toolchain (>= 2021.x versions, oneAPI) (#3581, #3582)
- various bug fixes, including:
- add --init and --recursive options to 'git submodule update' command that is used when creating source tarball for specific commit (#3537)
- filter out duplicate paths in RPATH wrapper script (#3538)
- don't clean up imported modules after verifying imports of included Python modules (#3544)
- avoid no-op changes to $LD_* environment variables in ModulesTool (#3553)
- fix UTF-8 encoding errors when running EasyBuild with Python 3.0.x-3.6.x (#3565)
- create lib64 symlink as a relative symlink (#3566)
- don't reuse variable name in the loop to fix adding extra compiler flags via toolchainopts (#3571)
- symlink 'lib' to 'lib64' if it doesn't exist (#3580)
- include %(mpi_cmd_prefix)s and %(cuda_*)s templates in output of --avail-easyconfig-templates (#3586)
- other changes:
- rename EasyBlock._skip_step to EasyBlock.skip_step, to make it part of the public API (#3561)
- make symlinking of posix_c.so to posix.so in test suite configuration conditional (#3570)
- use 'main' rather than 'master' branch in GitHub integration functionality (#3589)
v4.3.2 (December 10th 2020)
---------------------------
update/bugfix release
- add (experimental) support for specifying easyconfig files via an "easystack" file (#3479, #3511, #3515, #3517, #3520, #3521)
- see also https://easybuild.readthedocs.io/en/latest/Easystack-files.html
- add definition for new 'gobff' toolchain using BLIS and LibFLAME (#3505)
- various enhancements, including:
- add support for toolchain options like 'extra_cxxflags' to specify extra compiler options (#2193)
- fix combination of --copy-ec and --from-pr (#3482)
- enhance copy_files function: support single file target, error on empty input list, support verbose mode (#3483)
- cache result of fetch_files_from_pr function (mainly to speed up framework test suite) (#3484)
- add locate_files function to filetools module (#3485)
- add support for %(module_name)s template value (#3497)
- clarify input format for --cuda-compute-capabilities in 'eb --help' output (#3509)
- add support for skiping unit tests (test step) via --skip-test-step (#3524)
- various bug fixes, including:
- also ignore vsc.* imports coming from from pkg_resources/__init__.py (setuptools) in fake vsc namespace (#3491)
- don't pass username in github_api_get_request when no GitHub token is available (#3494)
- also inject -rpath options for all entries in $LIBRARY_PATH in RPATH wrappers (#3495)
- avoid TypeError being raised by list_toolchains (#3499)
- check if PR is already merged in --merge-pr (#3502)
- graciously handle wrong PR id in fetch_pr_data (#3503)
- fix regression in apply_regex_substitutions: also accept list of paths to patch (#3507)
- update installation procedure for EasyBuild in generated Singularity container recipes (#3510)
- fix GitHub Actions workflow for test suite: run outside of repo checkout + also test bootstrap script with Python 3.9 (#3518)
- bump cryptography from 2.9.2 to 3.2 for Python 2 in requirements.txt (#3519)
- fix 'eb --help=rst' when running with Python 3 (#3525)
- other changes:
- exclude test configurations with Lmod 7 and Python 3, except for Python 3.6 (#3496)
- significantly speed up parsing of easyconfig files by only extracting comments from an easyconfig file when they're actually needed (#3498)
- don't include file/ldd/readelf commands run during RPATH sanity check in --trace output (#3508)
v4.3.1 (October 29th 2020)
--------------------------
update/bugfix release
- various enhancements, including:
- further GCC toolchain fixes for aarch64 (#3433)
- take into account --include-easyblocks-from-pr when uploading test reports (#3446)
- add path to pkg-config files in sysroot to $PKG_CONFIG_PATH when --sysroot is specified (#3451)
- add support for NVHPC compiler + toolchain (based on PGI) (#3454)
- check for _VERSION and _PREFIX Cray environment variables with both software and module name (#3460)
- allow including easyblocks from multiple PRs (#3480, #3481)
- various bug fixes, including:
- avoid UnicodeDecodeError in apply_regex_substitutions when patching files that include non-UTF-8 characters (#3450)
- avoid appending lib stubs pattern to RPATH filter over and over again (#3452)
- fix missing string template on error for incorrect extension 'sources' value (#3461)
- fix compatibility with Python 3.9 by renaming fancy root logger (#3465)
- also remove empty checksums list specified in easyconfig file when using --inject-checksums (#3466)
- avoid confusing error log message when determining impi version while trying to define value for %(mpi_cmd_prefix)s template (#3474)
- unset $LD_LIBRARY_PATH when checking for OS dependencies with 'rpm' & co (#3477)
- don't change directory in download_repo function in tools.github (#3486)
- take source_urls, checksums, patches into account when extension source is specified via 'sources' (#3487)
- other changes:
- consider $EB_INSTALLPYTHON in 'eb' command to specify 'python' command to use for running EasyBuild (#3428)
- use only the sub folder name for createSubmoduleDeps script (#3464)
v4.3.0 (September 13th 2020)
----------------------------
feature release
- various enhancements, including:
- add script to create 'sources' entries for git submodules (#3369, #3436)
- add templates for CUDA compute capabilities (#3382)
- '%(cuda_compute_capabilities)s', '%(cuda_sm_comma_sep)s', '%(cuda_sm_space_sep)s'
- add EasyBuild configuration option --generate-devel-module (#3388)
- to allow disabling generating of "devel" modules via --disable-generate-devel-module
- set up a minimal build environment when using system compiler (#3399)
- by default, $CC is set to 'gcc' and $CXX is set to 'g++' when using 'system' toolchain
- minimal build environment can be customized via --minimal-build-env configuration option
- add --sysroot configuration option to specify alternative location of system root (#3419)
- this is useful when installing software in a Gentoo Prefix environment (for example)
- various bug fixes, including:
- allow including easyblocks from multiple locations, by combining --include-easyblocks and --include-easyblocks-from-pr (#3311)
- also escape backslashes in quote_py_str() (#3386)
- use one argument 'module swap' statements in Tcl modulefiles (required by Modules 4.2.3+) (#3397)
- fix copying of (non-existing) file with apply_patch (#3400)
- create symlink from lib64 to lib subdir in installation directories to avoid that GCC prefers /lib64 system directories (#3401)
- fix default value for 'lib64_fallback_sanity_check' build option (#3402)
- correctly determine commit status in --merge-pr (#3406)
- stop installing ccache wrapper for Fortran compiler (#3409)
- fix issues with applying PR patch in --from-pr (#3414)
- make RPATH wrapper script more robust by using 'python -E -s -S' to run rpath_args.py (#3422)
- don't inject -Wl,-rpath options when '-x c++-header' compiler option is used (#3424)
- fix lack of -mno-recip on aarch64 by tweaking GCC options used when 'precise' toolchain option is enabled (#3425)
- make sure self.start_dir is set in ExtensionEasyBlock (#3426, #3435)
- exclude local variables from typo check in easyconfig files (#3427)
- other changes:
- stick to cryptography 2.9.2 when using Python 2.7 to avoid broken test suite in CI (#3392)
- automatically enable --ignore-locks with --fetch (#3404)
- switch to status badge based on tests run in GitHub Actions CI in README (#3415)
- make flake8 code style checks pass (#3416, #3417)
- limit Travis to only test with Python 2.6 + Lomod 7 (#3418)
- ignore deprecation warning raised for cryptography when using Python 3.5 produced by output of test suite (#3423)
v4.2.2 (July 8th 2020)
----------------------
update/bugfix release
- various enhancements, including:
- add support for using 'sources' and 'git_config' for extensions in 'exts_list' (#3294)
- add support for software minver template (like %(pyminver)s, etc.) (#3344, #3345)
- add support for updating dictionary or tuple easyconfig parameters with self.cfg.update (#3356)
- various bug fixes, including:
- fix crash in --avail-easyconfig-constants when using --output-format=rst + ensure sorted output (#3341)
- always take into account builddependencies when generating template values, also when we're not iterating over builddependencies (#3346)
- fix running command as 'easybuild' user in generated Singularity definition file (#3347)
- allow ignoring versionsuffix in --try-update-deps (#3350, #3353)
- retain order of paths when generating prepend_path statements for module file (don't sort them alphabetically) (#3367)
- also put easyblocks used by extensions in 'reprod' directory (#3375)
- also copy template values in EasyConfig.copy method to ensure they are defined when installing extensions (#3377)
- skip lines that start with 'module-version' when determining whether a module exists in ModulesTool.exist (#3379)
v4.2.1 (May 20th 2020)
----------------------
update/bugfix release
- various enhancements, including:
- also mention CPU architecture (x86_64, POWER) in comment for test reports (#3281)
- add support for enhancing existing sanity check in easyconfigs, as opposed to overwriting paths/commands (#3288)
- clean up locks when EasyBuild session is cancelled with a signal like SIGTERM (#3291, #3321)
- add 'find_glob_pattern' function to filetools module (#3297)
- add constants for common OS dependencies (OS_PKG_IBVERBS_DEV, OS_PKG_OPENSSL_DEV, ...) (#3309, #3334)
- flesh out get_mpi_cmd_template function from Mpi.mpi_cmd_for method (#3312)
- add variable 'moddependpaths' to specify extra $MODULEPATH entry to consider for loading dependency modules (#3324)
- allow copying of tweaked easyconfigs when using --try-* with --copy-ec (#3332)
- various bug fixes, including:
- make ModulesTool.exist more robust w.r.t. module wrappers, aliases, defaults, etc. (#3216, #3337)
- clean up rst output of --list-toolchains (#3246)
- cast CPU arch name provided by archspec to a regular string (#3286)
- get pr_title and pr_descr built_options in new_pr_from_branch instead of new_pr (and commit_msg in both) (#3298)
- make pypi_source_urls more robust by using HTMLParser rather than xml.etree.ElementTree (#3303, #3329)
- fix broken test for --include-easyblocks-from-pr (#3304)
- don't use distutils.dir_util in copy_dir (#3310)
- print trace message for sanity check command before running it (#3316)
- fix problems with processing of easyconfigs using a Cray* toolchain when there are no actual external modules (#3319)
- make test_find_eb_script more robust in case $EB_SCRIPT_PATH is already set (#3320)
- fix several small problems wit --try-update-deps (experimental feature) (#3325, #3326, #3330)
- add --disable-job in 'eb' command used in jobs, to prevent infinite job cycle (#3328)
- avoid empty entries in $LD_LIBRARY_PATH and other path-like environment variables (#3333)
- other changes:
- fix code style issues in easybuild.tools + add flake8 linting test (#3282)
- introduce contextmanager for disabling templating and reduce resolving errors (#3287)
- add 'change_into_dir' named argument to 'extract_file' + print deprecation warning if it's not specified (#3292)
- improve install_eb_dep.sh script to install EasyBuild dependencies in CI environment (#3314)
v4.2.0 (April 14th 2020)
------------------------
feature release
- add support for --try-update-deps (experimental feature), to upgrade dependencies based on available easyconfigs (#2599)
- adding locking to prevent two parallel builds of the same installation directory (#3009)
- for more information, see https://easybuild.readthedocs.io/en/latest/Locks.html
- significantly speed up -D/--dry-run by avoiding useless 'module show' (#3203)
- add support for creating an index & using it when searching for easyconfigs (#3210)
- for more information, see https://easybuild.readthedocs.io/en/latest/Easyconfigs_index.html
- additional GitHub integration features:
- add support for targeting easyblocks and framework repositories in --new-pr (#1876, #3189)
- add support for --include-easyblocks-from-pr (#3206)
- for more information, https://easybuild.readthedocs.io/en/latest/Integration_with_GitHub.html
- various other enhancements, including:
- add a contrib/hooks dir with some examples of hooks used (#2777)
- also mention working directory + input passed via stdin (if any) in trace output of run_cmd (#3168)
- probe external modules for missing metadata that is not provided via extermal module metadata file (#3174)
- also update $CMAKE_PREFIX_PATH and $CMAKE_LIBRARY_PATH in generated module file (#3176)
- optionally call log.warning in print_warning (#3195)
- add an option to git_config to retain the .git directory (#3197)
- allow use of SYSTEM as --try-toolchain option (#3213)
- mention CPU arch name in comment for uploaded test report, if it's known by archspec (#3227)
- make --merge-pr take into account --pr-target-branch (#3236)
- make --check-contrib print a warning when None is used for checksums (#3244)
- update install-EasyBuild-develop.sh script and create script for 2020a merge sprint (#3245)
- add template for mpi_cmd_prefix (#3264)
- update copy_dir to include option to merge directories (#3270)
- support template name for CUDA version (#3274)
- various bug fixes, including:
- use correct module for errors_found_in_log (#3119)
- fix EasyConfig.update code to handle both strings and lists as input (#3170)
- fix removing temporary branch on --check-github (#3182)
- fix shebang even if first line doesn't start with '#!' (#3183)
- make boostrap script work with Python 3 (#3186)
- read patch files as bytestring to avoid UnicodeDecodeError for patches that include funky characters (#3191)
- fix online check in --check-github: try repeatedly and with different URLs to cater for HTTP issues (#3194)
- don't crash when GitPython is not installed when using Python 3 (#3198)
- fix os_name_map for RHEL8 (#3201)
- don't add shebang to binary files (#3208)
- use checkout@v2 in GitHub Actions to fix broken re-triggered tests (#3219)
- don't filter out None values in to_checksums, leave them in place (#3225)
- fix defining of $MPI_INC_* and $MPI_LIB_* environment variables for impi version 2019 and later (#3229)
- use correct target account/repo when creating test report & posting comment in PR (#3234)
- reorganize EasyBlock.extensions_step to ensure correct filtering of extensions (#3235)
- also support %(installdir)s and %(builddir)s templates for extensions (#3237)
- unset $GITHUB_TOKEN in Travis after installing token, to avoid failing test_from_pr_token_log (#3252)
- fix reporting when skipping extensions (#3254)
- avoid API rate limit errors on online check in --check-github (#3257)
- show easyconfig filenames for parallel build (#3258)
- make EasyConfig.dump aware of toolchain hierarchy, to avoid hardcoded subtoolchains in dependencies easyconfig parameters (#3261)
- fix use of --copy-ec with a single argument, assume copy to current working directory (#3262)
- fix posting of comment in PR with --upload-test-report (#3272)
- take into account dependencies marked as external modules when composing template values like %(pyver)s (#3273)
- other changes:
- increase timeout & use api.github.com for connectivity check in check_github (#3192)
- restore flake8 default ignores (#3193)
- enable tracking of test suite coverage using coveralls (#3204)
- make tests use easybuilders/testrepository rather than hpcugent/testrepository after it was moved (#3238)
- improve raised error in remove_dir and deprecate rmtree2 (#3228)
v4.1.2 (March 16th 2020)
------------------------
bugfix release
- fix gitdb dependency on Python 2.6 in test configuration (#3212)
- fix broken test for --review-pr by using different PR to test with (#3226)
- censor authorization part of headers before logging ReST API request (#3248)
- see also https://github.com/easybuilders/easybuild-framework/security/advisories/GHSA-2wx6-wc87-rmjm
v4.1.1 (January 16th 2020)
--------------------------
update/bugfix release
- various enhancements, including:
- add check_log_for_errors function (in easybuild.tools.run) to detect and handle multiple errors (#3118)
- implement support for 'eb --show-ec' to show contents of specified easyconfig file (#3132)
- also update $XDG_DATA_DIR (share/) and $GI_TYPELIB_PATH environment variables (lib*/girepository-*) in generated module files (#3133)
- add support for --copy-ec to copy easyconfig file to specified location (#3142)
- mention --disable-* option in --help output for boolean options enabled by default (#3151)
- add --cuda-compute-capabilities configuration option (#3161)
- various bug fixes, including:
- ignore imports from vsc namespace made from pkgutil.py (#3120)
- only actually change permissions using os.chmod in adjust_permissions if the current permissions are not correct already (#3125)
- use shutil.copyfile to just copy file contents if target path exists and is owned by someone else (#3127)
- fix or avoid warnings that commonly arise in build log (#3129)
- disable buffering in asyncprocess.Popen using bufsize=0, to fix run_cmd_qa missing output (#3130)
- update pip & install wheel package in generated Singularity container recipes (#3136)
- avoid crash in modify_env & unset unset_env_vars when using (older versions) of Python 3.5 & 3.6 by using list(...) (#3140)
- take into account that lib64 could be a symlink to lib (or vice versa) in get_software_libdir function (#3141)
- only parse docstring if it exists in gen_easyblock_doc_section_rst function (#3144)
- only add useful entries for $CPATH, $(LD_)LIBRARY_PATH and $PATH (non-empty directories) (#3145, #3152)
- fix --list-software=detailed when using Python 3 by leveraging sort_looseversions function from py2vs3 module (#3146)
- ensure subdirectories in software install directory have correct search (exec) permission (#3147)
- take into account that a checksum value may be a tuple of valid checksum in EasyBlock.check_checksums (#3153)
- other changes:
- bump to Lmod 8.2.9 in GitHub CI config (#3115)
- update copyright statements for 2020 (#3149)
- make Hound CI code style checker ignore "Black would make changes" produced by flake8-black (#3162)
v4.1.0 (December 4th 2019)
--------------------------
update/bugfix release
- various enhancements, including:
- performance improvements:
- skip validation when copying EasyConfig object for extension (#3071)
- correctly specify that ActiveMNS & co are singleton classes when using Python 3 (#3073)
- don't call out to prohibitively expensive getRootLoggerName in getLogger, just use 'root' instead (#3074)
- fix inconsistent module path usage that leads to repeated reloading in HMNS (#3099)
- add support for specifying different dependency version based on processor architecture (#3047)
- support use of glob patterns for paths to files with external modules metadata (#3075)
- take into account that external modules may not be visible directly (due to module hierarchy) (#3083)
- add support for including 'extensions' statement in Lua modules with Lmod 8.2.8+ (#3085, #3107, #3110)
- add support for --sync-pr-with-develop (#3087)
- add support for --new-branch-github, --new-pr-from-branch, --sync-branch-with-develop, --update-branch-github (#3103)
- fix typo in docstring for new_branch_github (#3106)
- various bug fixes, including:
- correctly handle iccifortcuda toolchain with standalone iccifort in det_toolchain_compilers (#3055)
- init git repo with git.repo.clone() instead of copy_dir() (#3062)
- fix regular expression so depends-on statements are recognized correctly in Tcl module files (#3065)
- update GitPython to latest version that supports Python 2.6 in requirements.txt to fix broken test_new_update_pr (#3066)
- imply --disable-pre-create-installdir with --inject-checksums (#3069)
- handle patches in extensions more like normal patches (#3067)
- take into account that platform.linux_distribution and platform.dist was removed in Python 3.8 (#3078)
- always include mandatory easyconfig parameters in dumped easyconfig (#3081)
- hide backup module file when using Lmod 6.x (fixes #9302) (#3089)
- add better error message when mandatory key is missing from a dictionary easyconfig parameter (#3092)
- also create symlinks for default modules in class module folders (#3093)
- fix semantics of --set-default-module: only set default for specified easyconfigs, not for the ones that are installed as dependencies via --robot (#3094)
- fix various issues in extracting comments from original easyconfig file and including them again in dumped easyconfig (#3095)
- don't use %(version)s template in exts_default_options in dumped easyconfig (#3096)
- fix generated module statements in case only a single version is listed in multi_deps (#3097)
- fix broken test_show_system_info on macOS due to 'Python' binary (#3105)
- take into account that dependency version could be a dict rather than a string value in template_constant_dict (#3111)
- other changes:
- deprecate running EasyBuild with Python 2.6 via new check_python_version() function (#3076)
- deprecate support for using Lmod 6.x (#3077)
- trim set of test configurations in Travis CI (#3086)
- flesh out env_vars_external_module from Toolchain._simulated_load_dependency_module (#3088)
v4.0.1 (October 15th 2019)
--------------------------
update/bugfix release
- various enhancements, including:
- add 'parallel' to list of config templates (#3036)
- add GitHub Actions workflow to run easybuild-framework test suite (#3039)
- add 'retest' as a reason to --close-pr, to close/re-open PRs to trigger re-test in Travis (#3040)
- define $EB_SCRIPT_PATH in 'eb' wrapper script, and consider it before location of 'eb' determined via $PATH in get_paths_for function (#3046)
- add support for --remove-ghost-install-dirs configuration option, and warn about (potential) ghost install dirs by default when --force/--rebuild is used (#3050)
- various bug fixes, including:
- update bootstrap script to support installing EasyBuild v4.0 (#3017)
- fix broken test_download_repo due to archiving of easyconfigs (#3019, #3023)
- avoid that --inject-checksums introduces list of patches for extensions as a single long line (#3025, #3034)
- enhance regex in fix_shebang method to fix more Python/Perl shebangs + avoid patching binary files (#3029)
- delete test gist that is created by --check-github (#3031)
- disable templates when defining easyconfig parameters in EasyConfig.set_keys() (#3037)
- avoid setting GC3Pie's max_in_flight to None if --job-max-jobs is not specified (#3038)
- fix use of obtain_file method for extensions (#3042)
- error out if some GC3Pie job failed (#3044)
v4.0.0 (September 20th 2019)
----------------------------
feature release (incl. backwards-incompatible changes)
A concise overview of major changes in EasyBuild v4.0 is available at https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-of-changes.html .
- fixed compatibility with Python 3.5+ (#2708, #2713, #2714, #2719, #2721, #2723, #2729, #2743, #2744, #2751, #2756, #2759, #2761, #2762, #2765, #2766, #2767, #2768, #2774, #2775, #2778, #2780, #2785, #2787, #2789, #2791, #2792, #2794, #2800, #2801, #2805, #2806, #2895, #2932, #2982, #2992, #3007, #3011)
- supported Python versions: 2.6, 2.7, 3.5, 3.6, 3.7
- some functionality from the Python standard library should be imported from the new easybuild.tools.py2vs3 package
- see https://easybuild.readthedocs.io/en/latest/Python-2-3-compatibility.html & https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-relocated-functions-constants.html for more information
- ingested relevant code from vsc-base & vsc-install (#2708, #2713, #2714, #2763, #2790, #2993)
- vsc-base & vsc-install are no longer required as dependencies
- the functionality that was provided by these packages is now (mostly) available from the easybuild.base package
- any import statements from the 'vsc' namespace will result in an error
- see also https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-of-changes.html#ingested-functionality-from-vsc-base-and-vsc-install and https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-relocated-functions-constants.html
- setuptools is longer required (neither for installing or using EasyBuild) (#2836, #2837, #2984, #2986, #2988)
- see also https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-of-changes.html#no-more-required-python-packages
- this required moving some classes and constants, see https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-relocated-functions-constants.html
- the 'dummy' toolchain is deprecated and replaced by the 'system' toolchain (#2877, #3001)
- if 'dummy' is still used as a toolchain in easyconfig files, a warning will be printed
- use "toolchain = SYSTEM" instead
- for more information, see https://easybuild.readthedocs.io/en/latest/System_toolchain.html
- a warning is printed when local variables in easyconfig file don't follow the recommended naming scheme (#2938, #2968)
- see also https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html
- names of software installation directories are independent of module naming scheme by enabling --fixed-installdir-naming-scheme by default (#2999)
- see https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-of-changes.html#fixed-installdir-naming-scheme-enabled-by-default for more information
- various other small enhancements, including:
- use requests instead of urllib2 for 403 errors (#2695)
- use pip requirements file in Travis (#2874)
- add support for --fix-deprecated-easyconfigs (#2881, 2898, #2938)
- add -L$EBROOTIMKL/lib/intel64 to $LDFLAGS for MKL (#2930)
- handle dict-type checksums in check_checksums_for (#2974)
- allow that icc & ifort are not actual dependencies in iccifort toolchain module (#2995)
- define mapping for iccifort->intel for iccifort compiler-only toolchain for HMNS (#2996)
- also consider concatenation of compiler module names to determine details of toolchain compiler component (#2997)
- update metadata for Cray-provided external modules (#3013)
- various bug fixes, including:
- update the PyPI trove classifiers in setup.py (#2875)
- reverse lists for $LDFLAGS and $CPPFLAGS (#2931)
- enhance/fix to_template_str function to do a better job at using template values in dumped easyconfigs (#2948)
- also take into account --filter-deps when finalizing parsed dependencies to fix problem with dependency filters using version ranges (#2983)
- fix broken --from-pr tests due to archiving of easyconfigs (#2985)
- fixes required to avoid breaking Cray support (#3008)
- fixes for --read-only-installdir: avoid crash with ModuleRC easyblock + also make log file in installdir read-only (#3012)
- other changes:
- add check to ensure that --robot argument specifies an existing directory (#2086)
- remove old scripts that are no longer useful (+ minor fixes to others) (#2897)
- use 'command -v' to avoid requiring 'which' in 'eb' command (#2979)
- add modluafooter & modtclfooter at the end of the generated module file (#3003)
- print warning when 'eb' command is not found in $PATH and for empty robot search path (#3006)
v3.9.4 (August 23rd 2019)
-------------------------
update/bugfix release
- various enhancements, including:
- add support for specifying checksum via dict providing a filename-to-checksum mapping (#2946)
- enhance 'completed' message with how much time was needed for the installation (#2956)
- add support for specifying tuple of alternative checksums (#2958)
- support using 'system' as alias for 'dummy' toolchain + SYSTEM constant (#2960)
- ensure 'docurls' easyconfig parameter value is a list of string values, not a single string value (#2963)
- automatically enable --ignore-osdeps when using --check-contrib or --check-style (#2965)
- various bug fixes, including:
- escape '+' in search queries + handle invalid search queries better (#2967)
- also consider extension patches when determining for which easyconfig a given patch file is intended in --new-pr/--update-pr (#2969)
- unset $CDPATH early on if it is defined (#2970)
- create specified temporary log directory if it doesn't exist yet in init_logging (#2972)
- don't indicate first multi_deps version as default in module help text when multi_deps_load_default=False (#2973)
- other changes:
- deprecate toolchains older than gompi/2016a and foss/2016a (#2951)
v3.9.3 (July 8th 2019)
----------------------
update/bugfix release
- various enhancements, including:
- add a URL whatis line to generated module files (#2933)
- various bug fixes, including:
- stick to Ubuntu Trusty when testing with Python 2.6 in Travis (#2928)
- honor --tmp-logdir when determining location of temporary log file (#2937)
v3.9.2 (June 9th 2019)
----------------------
update/bugfix release
- various enhancements, including:
- enhance (experimental) support for generating Singularity container recipes/images (#2884, #2900, #2902, #2903, #2907, #2909, #2910, #2913, #2915)
- add support for specifying container configuration via --container-config (--container-base option has been removed)
- add support for providing custom template for container recipe via --container-template-recipe
- add support for 'sif' Singularity 3.x container image format
- add support for specifying non-image based Singularity bootstrap agents
- install default set of OS packages in container recipes starting from scratch
- configure Lmod and update Lmod spider cache in generated Singularity container recipes
- configure EasyBuild via environment variables (to allow overriding configuration settings via options to 'eb' command)
- also consider /tmp/easybuild/sources in source path (to allow seeding in sources in container build environment)
- updated documentation is available at https://easybuild.readthedocs.io/en/latest/Containers.html
- various bug fixes, including:
- make sure that easyconfig file for EasyBuild version being bootstrapped is found in robot search path (#2899)
- remove interpreter options when fixing script shebang (#2905, #2906)
- fix crash on iterated installation when using Cray toolchains (#2914)
- disable checking of read/exec permissions when checking for availability of 'sudo' command (#2918)
- other changes:
- stick to vsc-base<2.9.0 in bootstrap script to avoid requiring 'future' dependency (#2892)
v3.9.1 (May 20th 2019)
----------------------
update/bugfix release
- various enhancements, including:
- add function to determine whether specified easyblock is generic or not (#2838)
- add support to apply_regex_substitutions function to skip creation of backup (#2844)
- add support for %(pymajver)s (& co) (#2850)
- add support for 'fix_python_shebang_for' and 'fix_perl_shebang_for' easyconfig parameters (#2878)
- add multi_deps information to generated module, help and whatis sections (#2882)
- add support for 'eb --missing' ('eb -M') (#2883)
- see also https://easybuild.readthedocs.io/en/latest/Using_the_EasyBuild_command_line.html#getting-an-overview-of-missing-installations-missing-m
- various bug fixes, including:
- only call 'scontrol release' when there's at least one job submitted (#2834)
- fix small bug in find_eb_script + various minor issues with tests (related to environment in which tests are run) (#2835)
- list build dependencies *before* runtime dependencies, so dependencies listed via multi_deps are loaded first in toolchain environment (#2839)
- allow setting parallel to 0 or False to disable adding the -j argument (#2842)
- be more careful when using single quotes for Tcl syntax (#2847)
- append to existing .modulerc file rather than overwriting it (if --force is used) (#2848)
- correctly resolve template values used for extensions (#2852)
- update template values after updating iterative easyconfig parameters (#2854)
- add top-level directories of hierarchical module naming scheme to $MODULEPATH before loading dependencies (#2857)
- inject dependencies specified via multi_deps before normal build dependencies (#2861)
- include Python version(s) in PR titles auto-generated by --new-pr (#2863, #2880)
- update HierarchicalMNS for GCCcore toolchain (#2870)
- bump iteration index counter at the correct time (#2886)
- don't specify --ntasks-per-node when submitting Slurm jobs (#2887)
- fix order of easyconfig parameters in output generated by 'eb --avail-easyconfig-params --output-format rst' (#2888)
- other changes:
- lower required version in Slurm job backend to 16.05 (#2833)
- add Lmod 8 to test suite (#2849)
- deprecate useless 'skip_lower' named argument in template_constant_dict, always define *lower templates (#2856)
- fix Python classifiers in setup.py, should be (only) Python 2.6 & 2.7 (#2876)
v3.9.0 (April 12th 2019)
------------------------
feature release
- various enhancements, including:
- add (pre)testopts easyconfig parameters (#2793)
- add support for multi_deps easyconfig parameter (#2741, #2810, #2811, #2812, #2813, #2825, #2826, #2827, #2828)
- various bug fixes, including:
- fix argument name in close_pr (#2752)
- fix undefined variable 'prefix' in Compiler._set_compiler_vars (#2757)
- fix test account for GitHub-related tests (#2760)
- make sure read permissions are always set in permissions_step (#2770)
- make test for dep_graph function robust against changing order of lines in resulting dot file (#2772)
- fix problem with import_available_modules when running from easybuild-framework directory (#2786)
- prepend location of test easyblocks to $PYTHONPATH to test_generate_software_list, rather than append (#2788)
- use pid in backup name candidate to reduce risk of two processes colliding (#2796)
- automatically enable --ignore-osdeps when using --preview-pr (#2799)
- use temporary checkout of repository as robot path for --new-pr and --update-pr to determine locations for patch files (#2803)
- avoid relying on order in which keys are processed in to_dependency (#2804)
- fix yeb format to work with PyYAML 5 (#2809)
- fix typo in error message for use of unknown easyconfig parameter (#2817)
- enhance log_file_format to fix problem when %(name)s template value is used for log directory (#2818)
- iterate over subdirectories in order in find_egg_dir_for of bootstrap script, to ensure oldest vsc-install is picked (#2819)
- replace -Wl,--enable-new-dtags compiler option with -Wl,--disable-new-dtags in RPATH wrapper script (#2821)
- updated COMPILER_F77 for PGI >= 19.1 (#2823)
- added FFTW_STATIC_LIBS_MT to FFTW_MAP_CLASS, otherwise FFTW_STATIC_LIBS_MT are incorrectly defined (#2822)
- sort available version list when selecting/generating easyconfig (#2732)
- other changes:
- use assertEqual rather than assertEquals (which is deprecated) (#2746)
- test with Tmod 4.1.4 (packaged in RHEL8) in Travis CI (#2747)
- replace deps by hidden deps instead of removing them from the lists (#2748)
- ensure non-zero exit code for all test subsuites (+ fix trivial style issues) (#2773)
- silence printed deprecation warnings for easyconfigs/toolchains while running (easyconfigs) test suite (#2781)
- avoid printing of messages/warnings in framework test suite (#2782)
- enable silent mode for --new-pr and --update-pr (#2802)
- update Travis config to reflect that PyYAML 5.x is no longer compatible with Python 2.6 (#2807)
v3.8.1 (January 29th 2019)
--------------------------
bugfix release
- various minor enhancements, including:
- speed up checking of OS dependencies (#2703)
- add support for 'eb --show-system-info' (#2722)
- add support for '%(arch)s' template value in easyconfig files (x86-64, aarch64, ppc64le, ...) (#2728)
- various bug fixes, including:
- add timestamp to reprod dir while in tmp space (#2705)
- avoid '+' in directory name for g++ rpath wrapper (#2710)
- pre-install vsc-install < 0.11.4 in bootstrap script to avoid requiring 'mock' Python package (#2717)
- fix check in --merge-pr whether PR is eligible for merging in to only consider the status of the last test report (#2720)
- don't recreate build directory when 'buildininstalldir' is enabled for iterative installations (#2724)
- filter (CUDA) lib*/stubs paths in RPATH wrapper script (#2725)
- other changes:
- replace 'except IOError, err:' pattern with 'except IOError as err:' for compatibility with Python 3.x (#2711)
- make all print statements compatible with Python 3 (#2715)
- avoid use of sys.maxint in dependencies_for (#2716)
v3.8.0 (December 18th 2018)
---------------------------
feature release
- various enhancements, including:
- support use of version ranges in --filter-deps (#2357)
- add support for --list-prs (#2400, #2668)
- add support for --close-pr (#2401)
- allow setting optarch compiler flags in the easyconfig via toolchainopts easyconfig parameter (#2595)
- add option to skip pre-creation of install directory: --disable-pre-create-installdir (#2629, #2637)
- flesh out setting up of configuration into dedicated 'set_up_configuration' function (#2638)
- clean error when 'eb' is cancelled by user (#2641)
- add support for using Slurm as backend for --job, via --job-backend=Slurm (#2642, #2666)
- save easyblocks along with easyconfig in 'reprod' subdirectory of install directory (#2653)
- add support for deprecating easyconfig files & toolchains (#2656)
- changed easyconfig.update to be able to specify not to allow duplicate values (#2657)
- move adding of dependencies to prepare step instead of check readiness step (#2674)
- add support for disabling mapping of (sub)toolchains when --try-toolchain is used, via --disable-map-toolchains (#2682)
- add 'astro' and 'quantum' module classes (#2693)
- various bug fixes, including:
- make --from-pr always try to apply PR patch on top of PR target branch (#2631)
- fix bug that could cause to silently overwrite an existing easyconfig when using --try-* (#2635)
- don't pass down specific environment variables into submitted jobs (#2643)
- update template constants for source URLs to use https (#2648)
- fix lib64 fallback for 'lib'/'lib64' dirs entry in sanity_check_paths (#2649)
- change PGI F90 Fortran compiler to pgf90 (#2650)
- use --set-upstream-to in install-EasyBuild-develop.sh script to deal with deprecated/no longer supported --set-upstream (#2651)
- fix order of keys in 'toolchain' value for dumped easyconfig file (name, version) + run style check on dumped easyconfigs in dump tests (#2660)
- ensure checksums beside sources in dumped easyconfigs (#2661)
- fix problems with easyconfig file saved in 'reprod' directory by copying it before running any installation steps (#2664)
- make print functions more robust w.r.t. arguments being passed to format the message to be printed (#2670)
- use 'git am' to apply patch for PRs in fetch_easyconfigs_from_pr (#2680)
- correctly define $LIBFFT_MT for Intel MKL (#2688)
- don't always require easyconfig files to resolve dependencies (#2690, #2692, #2697, #2698, #2699)
- fixes for Travis CI config:
- stop testing with Lmod 6.6.3, testing with Lmod 6.5.1 is sufficient (#2627)
- Python 2.6 requires python-daemon 1.x as dep for GC3Pie (#2673)
- stick to idna<2.8 with Python 2.6 in Travis + distable broken test for HgRepository (#2678)
- other changes:
- always skip symlinks in adjust_permissions (#2644)
- flesh out 'avail_easyblocks' function from support for --list-easyblocks (#2663)
- deprecate the ictce toolchain (#2667)
- deprecate goolf and goolfc toolchains (#2676)
- deprecate intel toolchains older than intel/2016a (+ iccifort & iimpi subtoolchains) (#2677)
- deprecate ancient gompi toolchain versions (#2684)
- use 'tar xzf' (gzip) rather than 'tar xZf' (ancient compress) to unpack *.tar.Z source files (#2686)
- flesh out 'set_parallel' method to it can be called separately (#2687)
v3.7.1 (October 18th 2018)
--------------------------
bugfix release
- various enhancements, including:
- generate .modulerc.lua when Lua syntax and Lmod >= 7.8 is used (#2597)
- allow --force to use regex if --try-toolchain can not map intelligently (#2605)
- add support for disabling modules tool version check (#2610)
- add support to ModuleGenerator.modulerc method to also write .modulerc file (#2611)
- check whether module file being wrapped exists in same directory as module wrapper when using Lmod 6.x (#2611)
- various bug fixes, including:
- stop relying on 'easy_install' in bootstrap script, use 'python -m easy_install' instead (#2590)
- fix templating of values in list_software function (#2591)
- fix composing of lib64 fallback paths in sanity check (#2602)
- determine file_info for all easyconfigs before any actual copying in copy_easyconfigs function (#2604)
- also check for module wrappers in 'ModulesTool.exist' method (#2606)
- add trailing newline to module load message if it's not there yet (#2613)
- retain all dependencies when determining dependency tree of a toolchain (#2617)
- protect exts_lists from templating in dump method (#2619)
- making CUDA capability detection more robust (#2621)
- other changes:
- lower required Lmod version to 6.5.1 (#2593)
v3.7.0 (September 25th 2018)
----------------------------
feature release
- minimal Lmod version requirement bumped to 6.6.3 (#2575)
- various enhancements, including:
- add support to bootstrap script to force install specific EasyBuild version (#2382, #2580)
- consider potential of multiple subtoolchains when resolving dependencies (#2464, #2465, #2466, #2585)
- fall back to downloading using the requests Python package (if installed) when urllib2 fails due to SSL error (#2538)
- make --try-toolchain more aware of subtoolchains (#2539)
- subtoolchain of original toolchains are now mapped to subtoolchains of target toolchain
- add support for BLIS and goblf toolchain that uses BLIS for BLAS (#2540)
- allow skipping of sanity check step via 'skipsteps' easyconfig parameter (#2549)
- add support for --check-contrib (#2551)
- equivalent with --check-style, but also verifies presence of SHA256 checksums (+ more checks in the future)
- added support to 'download' sources from git (#2555)
- see https://easybuild.readthedocs.io/en/latest/Writing_easyconfig_files.html#downloading-from-a-git-repository
- add 'parse' hook to add support for applying site-specific customisations to the 'raw' easyconfig (#2562, #2566)