ncl installation on Mac with M1 chip and arm64 machine #187
Comments
|
Hi,
I've never tried to build for the M1 architecture. However, all of those
missing headers are from libraries NCL depends on. I don't understand the
context of these two lines:
LDFLAGS="-L/usr/local/opt/freetype/lib -L/usr/local/opt/libpng/lib"
CPPFLAGS="-I/usr/local/opt/freetype/include -I/usr/local/opt/libpng/include
-I/usr/local/opt/freetype/include/freetype2" pip install matplotlib
LDFLAGS="-L/opt/X11/lib -L/usr/local/opt/libpng/lib"
CPPFLAGS="-I/opt/X11/include/freetype2"
Namely, 1st line -- what does a pip install of matplotlib have to do with
building NCL sources? And 2nd line -- there's an include directive for
freetype -- what about all the other library headers like netcdf, grib2,
udunits, etc.?
These are the build instructions, including all dependencies:
http://ncl.ucar.edu/Download/build_from_src.shtml
Good luck!
Rick
…On Fri, Jun 3, 2022 at 11:36 AM Liyuan He ***@***.***> wrote:
Hi,
I'm using a new Mac with M1 chip and arm64 machine, while the precompiled
ncl was based on old compilers are not eligible for new Mac that can't
install old version compilers. I tried to compile it using new compilers on
my Mac, a lot of issues came up because of updates of compilers. I wonder
whether anyone has successfully installed ncl on Mac with M1 chip and arm64
machine and willing to share the precompiled code or the experience, so
more people with new Macs can have access to it.
Some of my issues are about missing header files, even I have reinstalled
the command line tools for several times.
/usr/X11R6/include/cairo/cairo-ft.h:46:10: fatal error: ft2build.h: No
such file or directory
46 | #include <ft2build.h>
hdf.c:54:10: fatal error: hdf.h: No such file or directory
54 | #include <hdf.h>
/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/usr/include/signal.h:69:42:
error: 'NSIG' undeclared here (not in a function)
69 | extern __const char *__const sys_signame[NSIG];
| ^~~~
udunitsW.c:5:10: fatal error: udunits2.h: No such file or directory
5 | #include <udunits2.h>
NclGRIB2.h:1:10: fatal error: grib2.h: No such file or directory
1 | #include <grib2.h>
NclApi.c:39:10: fatal error: netcdf.h: No such file or directory
39 | #include <netcdf.h>
udunitsW.c:5:10: fatal error: udunits2.h: No such file or directory
5 | #include <udunits2.h>
LDFLAGS="-L/usr/local/opt/freetype/lib -L/usr/local/opt/libpng/lib"
CPPFLAGS="-I/usr/local/opt/freetype/include -I/usr/local/opt/libpng/include
-I/usr/local/opt/freetype/include/freetype2" pip install matplotlib
LDFLAGS="-L/opt/X11/lib -L/usr/local/opt/libpng/lib"
CPPFLAGS="-I/opt/X11/include/freetype2"
I would appreciate any help, thanks in advance.
Best,
Lynn
—
Reply to this email directly, view it on GitHub
<#187>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADLWOXVHOIIKOJUISZX65UTVNI7BZANCNFSM5XZVSDYQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
@email-lhe Did you have any success with this? I managed to at least compile the ncar graphics libraries, which I wanted for an old fortran plotting program. NCL works fine in emulation, but I'll see if I can get that to compile, as well (without hdf4 and some other options for now). The freetype error is fixed by adding it to the include paths (e.g., the Darwin_Arm config file in my fork) My fork is https://github.com/MicroTed/ncl.git branch apple-arm64 I installed most of the dependencies with homebrew, and am using gcc-12 (also from homebrew). gcc apparently does not compile using xcode 14, so I'm using 13.4.1. I have netcdf4 compiled separately (netcdf-4.9.0). It built an ncl executable, but that hits a seg fault somewhere quickly and fails. |
|
Macports is reportedly building M1/arm64 versions from source. You might give that a try. If it works, please let us know, including your OS and Xcode versions. |
|
Thanks for the news on that. I tried installing it on my M1 macbook pro. It compiles and creates arm64 binaries: /opt/local/bin/ncl: Mach-O 64-bit executable arm64 but ncl gives a seg fault: % ncl gsun01n.ncl The libraries might work, e.g., for linking to fortran, but I haven't tried that yet (and that part was working before, too). It's really just the calling fortran from C that needs help at this point. That is the source of the seg fault because the fortran side (CPNUMB) thinks a string is a lot longer than it really is and tries to initialize it. I haven't had time to learn how to fix that. -- Ted |
FYI, I appear to have successfully built an arm64 version of NCL using MacPorts. No extensive testing, but can generate graphics successfully using some of the sample scripts. I have similar issues as @MicroTed with homebrew. Did not try building from source. with in my .profile. Process for me was simple, install MacPorts following instructions here and then: Took about ~15-20 minutes. MacOS 13.2.1 |
|
I am the maintainer of ncarg in MacPorts. Thanks to @MicroTed and my student I am aware of segfault. I ran all the examples and found that text, colour bars, and maps works, but not xy and contour plots. ncargex seems to work so it should be a problem with NCL or depending libraries. I still get segfault even with the workaround suggested by @MicroTed and am investigating the issue. |
|
I got segfault with C/Fortran as well when hlu is used. The failure occurs when NhlCreate() is called at line 151 in xy01c.c with NhlxyPlotClass. ni/src/lib/hlu/XyPlot.c does not seem to call cpnumb at least directly. https://www.ncl.ucar.edu/Applications/BasicExamples/XyPlot/xy01.shtml A minimum example in NCL would be: |
|
I built ncl on FreeBSD arm64/UTM and had same error. The stack trace in the dumped core indicates an issue with len_trim. |
|
Yes, I can reproduce I get the same behavior @tenomoto notes. NCL compiles and runs some scripts/code fine but then segfaults elsewhere. Using xy_2.ncl example... |
|
@tenomoto, please show more of that stack trace. Rebuild ncarg with -g, if necessary to add ncarg symbols to the stack trace, so that we can better understand the context. Len_trim is called only from fortran code, so this should be traceable back into ncarg source code. |
|
I haven't tried on macOS, but on FreeBSD len_trim is everything I get even with -g option. |
|
Okay. I do not find a direct call to len_trim, anywhere in NCL 6.6.2 source code, or in current Macports patches. Either it is called by a dependency, or else the stack or execution sequence is broken. |
|
Is it the same as calling 'len', e.g., len(cbuf)? Although I found that it returned a faulty value, the seg fault occured when cpnumb tried to set cbuf to blanks: cbuf=' ' or cbuf(1:lbuf)=' ' (with faulty lbuf from len) |
Sorry for the late response. Not yet. I used python instead. |
No, len() returns the declared length of a fortran character string. Len_trim() returns the string length, minus any trailing blanks. It is possible that len_trim() would call len() internally, but not the reverse. If len(cbuf) returned a faulty value that exceeded the string length, then cbuf(1:lbuf)=' ' would possibly cause seg fault. However, bad value from len(cbuf) is concerning by itself. |
|
I believe it only happens when C code passes the string to a Fortran subroutine, and the fortran gets an incorrect length. If len_trim tries to test the string for trailing blanks and also has a bad length, then it would go out of bounds? |
|
Ted, the short answer is potentially yes, that could cause an out of bounds string reference. I have trouble following this, because as I already said, I do not find any len_trim references anywhere in NCL source. If you could point me to a more specific scenario, I could try interpreting the NCL source code. I do not have access to an arm64 system to run my own testing. NCL internally uses both cfortran and iftran, both obsolescent systems for extending fortran 77. Neither has been touched since the advent of Mac M1. This is fertile ground for systemic problems porting to M1/arm. |
|
If I run the example given by @tenomoto with the macports compiled ncl (i.e., on M1), I get a traceback through cpnumb. So maybe gfortran uses len_trim as a generic for len? Or for some other string operation, perhaps. But when I compiled my own from source, it didn't fault on the len function itself. I haven't used lldb very much, and I'm not sure I can get the point in cpnumb where it crashes. |
|
I have 6.6.2 working via rosetta2 on my M1 using a precompiled binary and some x86 libraries I brought over from another machine so this is less pressing for me. However, I'm happy to help debug, although my arm64 NCL is built using MacPorts (i.e., someone would need to tell me how to re-build with a However, a simple xy script (below) gives me a similar trace to @MicroTed. Script for reference. |
|
I went back to one of my 6.6.2 testing versions and got more info in the backtrace: The write statement was obviously added for debugging, and, fwiw, the output was 'cpnum: lcx1 = 8233' It has been a while since I looked at this. It could be that my little hack in cpnumb (noted in the PR) only worked if compiled with -g -O0 (overrides the default -O). But I don't recall for sure. |
|
I came across something that suggested the C code might need to declare the string length variables as 'long int', so in Format.c, if I change the two declarations of
to
|
|
@MicroTed 's fix works fine! I only changed Format.c. The aarch64 architecture is LP64, i.e. int is 32 and long is 64 bits. I need to check the fix on Intel mac (x86_64) before I commit. I don't see why the same error does not arise on x86_64. On 32-bit architectures (LP32), int = long = 32 and the fix shouldn't cause any problems. --- ni/src/lib/hlu/Format.c.orig 2019-02-28 08:44:39.000000000 +0900
+++ ni/src/lib/hlu/Format.c 2023-03-09 15:33:04.000000000 +0900
@@ -940,7 +940,7 @@
NGCALLF(cpinrc,CPINRC)();
{
- int len1,len2,len3,len4;
+ long len1,len2,len3,len4;
NGstring cex1_f;
NGstring cex2_f;
NGstring cex3_f;
@@ -1283,7 +1283,7 @@
NGCALLF(cpinrc,CPINRC)();
{
- int len1,len2,len3,len4;
+ long len1,len2,len3,len4;
NGstring cex1_f;
NGstring cex2_f;
NGstring cex3_f; |
|
@tenomoto I don't know about gfortran, but I found this for Intel ifort:
Was there any intel64 other than the itanium? Anyway, it sounds like it would still use 4-byte on regular intel processors, even if compiled -m64, so I assume gfortran does the same on x86? |
|
Committed the fix macports/macports-ports@108d5f3 |
|
The size of hidden argument is % ./a.out
int 4
long 8
size_t 8This makes me wonder why ncl works without the fix. #include <stdio.h>
int main(void)
{
printf("int %lu\n", sizeof(int));
printf("long %lu\n", sizeof(long));
printf("size_t %lu\n", sizeof(size_t));
} |
|
FYI, I pulled @tenomoto's MacPorts update down, rebuilt, and was able to successfully run some of the tests we were having trouble with (e.g., xy plotting, etc.). However, there remains a segfault when setting a file. Originally noticed in when invoking Script: LLDB output: Grepping through the source pops up a lot of C code which is (way) out of my wheelhouse. |
|
@zarzycki, thank you for testing. Please run this test and show output. Run with |
|
Sure. Same behavior. In case it matters, my "hacked" rosetta binary (i.e., a precompiled binary pulled with gcc@7 libs from a working intel Mac) returns normally with the above test. |
Yes. |
|
All right. This appears to prove some basic M1/arm64 functionality such as read and write netcdf-3, basic command execution, and some graphics as you showed earlier. That is a good start. I do not have any more insights on the setfileoption problem at this time. |
|
@Dave-Allured, FYI, I ran your short script with no error: % ncl -Q setfile.ncl (M1, Monterey) |
|
I have been trying to install NCL using Conda on Mac with M2 chip but no luck yet. But have not tried using Macports (https://ports.macports.org/port/ncarg/). Just wondering if there is a get around to fix the issue. I have codes written in NCL earlier that are not converted to Python yet. So, it will be a great help if someone found a fix yet. |
Hi,
I'm using a new Mac with M1 chip and arm64 machine, while the precompiled ncl was based on old compilers are not eligible for new Mac that can't install old version compilers. I tried to compile it using new compilers on my Mac, a lot of issues came up because of updates of compilers. I wonder whether anyone has successfully installed ncl on Mac with M1 chip and arm64 machine and willing to share the precompiled code or the experience, so more people with new Macs can have access to it.
Some of my issues are about missing header files, even I have reinstalled the command line tools for several times.
/usr/X11R6/include/cairo/cairo-ft.h:46:10: fatal error: ft2build.h: No such file or directory
46 | #include <ft2build.h>
hdf.c:54:10: fatal error: hdf.h: No such file or directory
54 | #include <hdf.h>
/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/usr/include/signal.h:69:42: error: 'NSIG' undeclared here (not in a function)
69 | extern __const char *__const sys_signame[NSIG];
| ^~~~
udunitsW.c:5:10: fatal error: udunits2.h: No such file or directory
5 | #include <udunits2.h>
NclGRIB2.h:1:10: fatal error: grib2.h: No such file or directory
1 | #include <grib2.h>
NclApi.c:39:10: fatal error: netcdf.h: No such file or directory
39 | #include <netcdf.h>
udunitsW.c:5:10: fatal error: udunits2.h: No such file or directory
5 | #include <udunits2.h>
LDFLAGS="-L/usr/local/opt/freetype/lib -L/usr/local/opt/libpng/lib" CPPFLAGS="-I/usr/local/opt/freetype/include -I/usr/local/opt/libpng/include -I/usr/local/opt/freetype/include/freetype2" pip install matplotlib
LDFLAGS="-L/opt/X11/lib -L/usr/local/opt/libpng/lib" CPPFLAGS="-I/opt/X11/include/freetype2"
I would appreciate any help, thanks in advance.
Best,
Lynn
The text was updated successfully, but these errors were encountered: