Cannot install tractor

[HironaoMiyatake]

I could not install the latest master branch (5908427). When I rolled it back to 3b4cc61, I was able to install. I attached the error message below.

(base) [miyatake@galaxy-gw tractor_clone]$ make
make -C tractor mix
make[1]: Entering directory `/work/miyatake/tractor_clone/tractor'
python setup-mix.py build_ext --inplace
running build_ext
building '_mix' extension
swigging mix.i to mix_wrap.c
swig -python -o mix_wrap.c mix.i
Warning: Can't read registry to find the necessary compiler setting
Make sure that Python modules winreg, win32api or win32con are installed.
C compiler: gcc -pthread -B /home/miyatake/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

creating build
creating build/temp.linux-x86_64-3.7
compile options: '-I/home/miyatake/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/home/miyatake/anaconda3/include/python3.7m -c'
gcc: mix_wrap.c
gcc -pthread -shared -B /home/miyatake/anaconda3/compiler_compat -L/home/miyatake/anaconda3/lib -Wl,-rpath=/home/miyatake/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/mix_wrap.o -o /work/miyatake/tractor_clone/tractor/_mix.cpython-37m-x86_64-linux-gnu.so
make[1]: Leaving directory `/work/miyatake/tractor_clone/tractor'
make -C tractor emfit
make[1]: Entering directory `/work/miyatake/tractor_clone/tractor'
python setup-emfit.py build_ext --inplace
running build_ext
building '_emfit' extension
swigging emfit.i to emfit_wrap.c
swig -python -o emfit_wrap.c emfit.i
Warning: Can't read registry to find the necessary compiler setting
Make sure that Python modules winreg, win32api or win32con are installed.
C compiler: gcc -pthread -B /home/miyatake/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

compile options: '-UNDEBUG -I/home/miyatake/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/home/miyatake/anaconda3/include/python3.7m -c'
gcc: emfit_wrap.c
gcc -pthread -shared -B /home/miyatake/anaconda3/compiler_compat -L/home/miyatake/anaconda3/lib -Wl,-rpath=/home/miyatake/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/emfit_wrap.o -o /work/miyatake/tractor_clone/tractor/_emfit.cpython-37m-x86_64-linux-gnu.so
make[1]: Leaving directory `/work/miyatake/tractor_clone/tractor'
make -C tractor mpf
make[1]: Entering directory `/work/miyatake/tractor_clone/tractor'
rm mp_fourier_wrap.c mp_fourier.py _mp_fourier*.cpython-37m-x86_64-linux-gnu.so
rm: cannot remove `mp_fourier_wrap.c': No such file or directory
rm: cannot remove `mp_fourier.py': No such file or directory
rm: cannot remove `_mp_fourier*.cpython-37m-x86_64-linux-gnu.so': No such file or directory
make[1]: [mpf] Error 1 (ignored)
make mp_fourier
make[2]: Entering directory `/work/miyatake/tractor_clone/tractor'
python setup-mpf.py build_ext --inplace
running build_ext
building '_mp_fourier' extension
swigging mp_fourier.i to mp_fourier_wrap.c
swig -python -o mp_fourier_wrap.c mp_fourier.i
Warning: Can't read registry to find the necessary compiler setting
Make sure that Python modules winreg, win32api or win32con are installed.
C compiler: gcc -pthread -B /home/miyatake/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC

compile options: '-I/home/miyatake/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/home/miyatake/anaconda3/include/python3.7m -c'
gcc: mp_fourier_wrap.c
mp_fourier_wrap.c:3039: error: expected ';', ',' or ')' before 'img'
mp_fourier_wrap.c:3129: error: expected ';', ',' or ')' before 'img'
mp_fourier_wrap.c:3237: error: expected ';', ',' or ')' before 'inimg'
mp_fourier_wrap.c:3375: error: expected ';', ',' or ')' before 'inimg'
mp_fourier_wrap.c:3402: error: expected ';', ',' or ')' before 'amps'
mp_fourier_wrap.c: In function '_wrap_correlate7':
mp_fourier_wrap.c:4029: warning: implicit declaration of function 'correlate7'
mp_fourier_wrap.c: In function '_wrap_correlate7f':
mp_fourier_wrap.c:4125: warning: implicit declaration of function 'correlate7f'
mp_fourier_wrap.c: In function '_wrap_correlate7f_inout':
mp_fourier_wrap.c:4239: warning: implicit declaration of function 'correlate7f_inout'
mp_fourier_wrap.c: In function '_wrap_lanczos_shift_3f':
mp_fourier_wrap.c:4379: warning: implicit declaration of function 'lanczos_shift_3f'
mp_fourier_wrap.c: In function '_wrap_gaussian_fourier_transform_zero_mean':
mp_fourier_wrap.c:4487: warning: implicit declaration of function 'gaussian_fourier_transform_zero_mean'
error: Command "gcc -pthread -B /home/miyatake/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/miyatake/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/home/miyatake/anaconda3/include/python3.7m -c mp_fourier_wrap.c -o build/temp.linux-x86_64-3.7/mp_fourier_wrap.o" failed with exit status 1
make[2]: *** [_mp_fourier.cpython-37m-x86_64-linux-gnu.so] Error 1
make[2]: Leaving directory `/work/miyatake/tractor_clone/tractor'
make[1]: *** [mpf] Error 2
make[1]: Leaving directory `/work/miyatake/tractor_clone/tractor'
make: *** [mpf] Error 2

[jhennawi]

We are having the same problem. Has this issue been resolved?

[dstndstn]

This is probably coming from an old Swig version. What version do you have (swig -version)? 4.0.1 is known to work.

[dstndstn]

Can you please attach the generated mp_fourier_wrap.c file also.

[jhennawi]

Hi @dstndstn,

We updated to swig version 4.0.2 and are sure this is the version that Make and the python setup scripts are using. However, we get the same failure.

'''
make -C tractor mpf
make[1]: Entering directory /mnt/quasar/joe/python/tractor/tractor' rm mp_fourier_wrap.c mp_fourier.py _mp_fourier*.cpython-39-x86_64-linux-gnu.so rm: cannot remove ‘_mp_fourier*.cpython-39-x86_64-linux-gnu.so’: No such file or directory make[1]: [mpf] Error 1 (ignored) make mp_fourier make[2]: Entering directory /mnt/quasar/joe/python/tractor/tractor'
python setup-mpf.py build_ext --inplace
running build_ext
building '_mp_fourier' extension
swigging mp_fourier.i to mp_fourier_wrap.c
swig -python -o mp_fourier_wrap.c mp_fourier.i
Warning: Can't read registry to find the necessary compiler setting
Make sure that Python modules winreg, win32api or win32con are installed.
C compiler: gcc -pthread -B /mnt/quasar/joe/miniconda3/envs/qso_fitting/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /mnt/quasar/joe/miniconda3/envs/qso_fitting/include -fPIC -O2 -isystem /mnt/quasar/joe/miniconda3/envs/qso_fitting/include -fPIC

compile options: '-I/mnt/quasar/joe/miniconda3/envs/qso_fitting/lib/python3.9/site-packages/numpy/core/include -I/mnt/quasar/joe/miniconda3/envs/qso_fitting/include/python3.9 -c'
gcc: mp_fourier_wrap.c
mp_fourier_wrap.c:2724:41: error: expected ‘;’, ‘,’ or ‘)’ before ‘img’
static void correlate7(double* restrict img, int img_dim1, int img_dim2,
^
mp_fourier_wrap.c:2814:42: error: expected ‘;’, ‘,’ or ‘)’ before ‘img’
static void correlate7f(float* restrict img, int img_dim1, int img_dim2,
^
mp_fourier_wrap.c:2922:47: error: expected ‘;’, ‘,’ or ‘)’ before ‘inimg’
static void correlate7f_inout(float* restrict inimg, int inimg_dim1, int inimg_dim2,
^
mp_fourier_wrap.c:3060:46: error: expected ‘;’, ‘,’ or ‘)’ before ‘inimg’
static void lanczos_shift_3f(float* restrict inimg, int inimg_dim1, int inimg_dim2,
^
mp_fourier_wrap.c:3087:23: error: expected ‘;’, ‘,’ or ‘)’ before ‘amps’
double * restrict amps, int amps_len,
^
mp_fourier_wrap.c: In function ‘_wrap_correlate7’:
mp_fourier_wrap.c:3717:3: warning: implicit declaration of function ‘correlate7’ [-Wimplicit-function-declaration]
correlate7(arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10);
^
mp_fourier_wrap.c: In function ‘_wrap_correlate7f’:
mp_fourier_wrap.c:3810:3: warning: implicit declaration of function ‘correlate7f’ [-Wimplicit-function-declaration]
correlate7f(arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10);
^
mp_fourier_wrap.c: In function ‘_wrap_correlate7f_inout’:
mp_fourier_wrap.c:3920:3: warning: implicit declaration of function ‘correlate7f_inout’ [-Wimplicit-function-declaration]
correlate7f_inout(arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10,arg11,arg12,arg13);
^
mp_fourier_wrap.c: In function ‘_wrap_lanczos_shift_3f’:
mp_fourier_wrap.c:4055:3: warning: implicit declaration of function ‘lanczos_shift_3f’ [-Wimplicit-function-declaration]
lanczos_shift_3f(arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10,arg11);
^
mp_fourier_wrap.c: In function ‘_wrap_gaussian_fourier_transform_zero_mean’:
mp_fourier_wrap.c:4159:3: warning: implicit declaration of function ‘gaussian_fourier_transform_zero_mean’ [-Wimplicit-function-declaration]
gaussian_fourier_transform_zero_mean(arg1,arg2,arg3,arg4,arg5,arg6,arg7,arg8,arg9,arg10,arg11,arg12,arg13);
^
error: Command "gcc -pthread -B /mnt/quasar/joe/miniconda3/envs/qso_fitting/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /mnt/quasar/joe/miniconda3/envs/qso_fitting/include -fPIC -O2 -isystem /mnt/quasar/joe/miniconda3/envs/qso_fitting/include -fPIC -I/mnt/quasar/joe/miniconda3/envs/qso_fitting/lib/python3.9/site-packages/numpy/core/include -I/mnt/quasar/joe/miniconda3/envs/qso_fitting/include/python3.9 -c mp_fourier_wrap.c -o build/temp.linux-x86_64-3.9/mp_fourier_wrap.o" failed with exit status 1
make[2]: *** [_mp_fourier.cpython-39-x86_64-linux-gnu.so] Error 1
make[2]: Leaving directory /mnt/quasar/joe/python/tractor/tractor' make[1]: *** [mpf] Error 2 make[1]: Leaving directory /mnt/quasar/joe/python/tractor/tractor'
make: *** [mpf] Error 2
'''
mp_fourier_wrap.c.txt

[jhennawi]

The mp_fourier_wrap.c is also attached. Note we are using Linux CentOS 7.0.

[dstndstn]

Thanks -- if you are using a git checkout, could you also send 'git describe' and 'git branch'?

Note that we renamed the 'master' branch to 'main', so if you had an older checkout (where 'git branch' shows 'master'), you would have to do:

git checkout master
git branch -m master main
git fetch
git branch --unset-upstream
git branch -u origin/main

[dstndstn]

Oh, this is ringing a bell. What compiler are you using? gcc --version?

I get exactly the same mp_fourier_wrap.c file, and if you look at the line it's complaining about, the compiler is not recognizing the 'restrict' keyword. That's a C99 feature, so maybe you need

make CPPFLAGS="-std=c99"

OR, try building using setup.py rather than make:

python setup.py build_ext --inplace
# for in-place operation, or to install:
python setup.py install --home=/where/to/install/it

[dstndstn]

yeah, when I try centos7 (with docker) and yum install gcc, I get gcc 4.8.5... from 2015, and way back then, the default language supported by gcc was gnu90/c90.

So try that CPPFLAGS setting, and if that works for you I'll update the Makefile.

[Rik-ad-astra]

The CPPFLAGS setting still raises an error that is ignored:

'''(base) [rnanni@igm tractor]$ make CPPFLAGS="-std=c99"
make -C tractor mix
make[1]: Entering directory `/mnt/quasar2/rnanni/softwares/tractor/tractor'
python setup-mix.py build_ext --inplace
running build_ext
building '_mix' extension
swigging mix.i to mix_wrap.c
swig -python -o mix_wrap.c mix.i
Warning: Can't read registry to find the necessary compiler setting
Make sure that Python modules winreg, win32api or win32con are installed.
C compiler: gcc -pthread -B /home/rnanni/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c99 -fPIC

creating build
creating build/temp.linux-x86_64-3.7
compile options: '-I/home/rnanni/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/home/rnanni/anaconda3/include/python3.7m -c'
gcc: mix_wrap.c
In file included from mix_wrap.c:4024:0:
mix_wrap.c: In function ‘_wrap_c_gauss_2d_approx3’:
approx3.c:458:11: warning: ‘temp20’ may be used uninitialized in this function [-Wmaybe-uninitialized]
(*sy1)++;
^
mix_wrap.c:4529:7: note: ‘temp20’ was declared here
int temp20 ;
^
mix_wrap.c:723:42: warning: ‘temp19’ may be used uninitialized in this function [-Wmaybe-uninitialized]
#define PyInt_FromLong(x) PyLong_FromLong(x)
^
mix_wrap.c:4527:7: note: ‘temp19’ was declared here
int temp19 ;
^
In file included from mix_wrap.c:4024:0:
approx3.c:457:11: warning: ‘temp18’ may be used uninitialized in this function [-Wmaybe-uninitialized]
(*sx1)++;
^
mix_wrap.c:4525:7: note: ‘temp18’ was declared here
int temp18 ;
^
mix_wrap.c:723:42: warning: ‘temp17’ may be used uninitialized in this function [-Wmaybe-uninitialized]
#define PyInt_FromLong(x) PyLong_FromLong(x)
^
mix_wrap.c:4523:7: note: ‘temp17’ was declared here
int temp17 ;
^
gcc -pthread -shared -B /home/rnanni/anaconda3/compiler_compat -L/home/rnanni/anaconda3/lib -Wl,-rpath=/home/rnanni/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ -std=c99 build/temp.linux-x86_64-3.7/mix_wrap.o -o /mnt/quasar2/rnanni/softwares/tractor/tractor/_mix.cpython-37m-x86_64-linux-gnu.so
make[1]: Leaving directory /mnt/quasar2/rnanni/softwares/tractor/tractor' make -C tractor emfit make[1]: Entering directory /mnt/quasar2/rnanni/softwares/tractor/tractor'
python setup-emfit.py build_ext --inplace
running build_ext
building '_emfit' extension
swigging emfit.i to emfit_wrap.c
swig -python -o emfit_wrap.c emfit.i
Warning: Can't read registry to find the necessary compiler setting
Make sure that Python modules winreg, win32api or win32con are installed.
C compiler: gcc -pthread -B /home/rnanni/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c99 -fPIC

compile options: '-UNDEBUG -I/home/rnanni/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/home/rnanni/anaconda3/include/python3.7m -c'
gcc: emfit_wrap.c
emfit_wrap.c: In function ‘_wrap_em_fit_2d_reg2’:
emfit_wrap.c:3833:41: warning: ‘temp10’ may be used uninitialized in this function [-Wmaybe-uninitialized]
resultobj = SWIG_Python_AppendOutput(resultobj, SWIG_From_double((arg10)));
^
gcc -pthread -shared -B /home/rnanni/anaconda3/compiler_compat -L/home/rnanni/anaconda3/lib -Wl,-rpath=/home/rnanni/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ -std=c99 build/temp.linux-x86_64-3.7/emfit_wrap.o -o /mnt/quasar2/rnanni/softwares/tractor/tractor/_emfit.cpython-37m-x86_64-linux-gnu.so
make[1]: Leaving directory /mnt/quasar2/rnanni/softwares/tractor/tractor' make -C tractor mpf make[1]: Entering directory /mnt/quasar2/rnanni/softwares/tractor/tractor'
rm mp_fourier_wrap.c mp_fourier.py _mp_fourier.cpython-37m-x86_64-linux-gnu.so
rm: cannot remove ‘mp_fourier_wrap.c’: No such file or directory
rm: cannot remove ‘mp_fourier.py’: No such file or directory
rm: cannot remove ‘_mp_fourier*.cpython-37m-x86_64-linux-gnu.so’: No such file or directory
make[1]: [mpf] Error 1 (ignored)
make mp_fourier
make[2]: Entering directory `/mnt/quasar2/rnanni/softwares/tractor/tractor'
python setup-mpf.py build_ext --inplace
running build_ext
building '_mp_fourier' extension
swigging mp_fourier.i to mp_fourier_wrap.c
swig -python -o mp_fourier_wrap.c mp_fourier.i
Warning: Can't read registry to find the necessary compiler setting
Make sure that Python modules winreg, win32api or win32con are installed.
C compiler: gcc -pthread -B /home/rnanni/anaconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c99 -fPIC

compile options: '-I/home/rnanni/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/home/rnanni/anaconda3/include/python3.7m -c'
gcc: mp_fourier_wrap.c
gcc -pthread -shared -B /home/rnanni/anaconda3/compiler_compat -L/home/rnanni/anaconda3/lib -Wl,-rpath=/home/rnanni/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ -std=c99 build/temp.linux-x86_64-3.7/mp_fourier_wrap.o -o /mnt/quasar2/rnanni/softwares/tractor/tractor/_mp_fourier.cpython-37m-x86_64-linux-gnu.so -g
make[2]: Leaving directory /mnt/quasar2/rnanni/softwares/tractor/tractor' make[1]: Leaving directory /mnt/quasar2/rnanni/softwares/tractor/tractor'
'''

When I try the test script I get:
'''(base) [rnanni@igm tractor]$ python examples/tractor-sdss-synth.py
Traceback (most recent call last):
File "examples/tractor-sdss-synth.py", line 20, in
from tractor import *
File "/mnt/quasar2/rnanni/softwares/tractor/tractor/init.py", line 11, in
from .version import *
ModuleNotFoundError: No module named 'tractor.version'
'''

[dstndstn]

The warnings are normal and not anything to worry about.

To fix the 'version' error, do

make version

[dstndstn]

I just pushed a commit that will hopefully address both of these issues for you.

[Rik-ad-astra]

Ok, thanks. I see that simply following your second option (python setup.py build_ext --inplace), the issue is solved without needing the "make version". I still cannot run the test example as it seems that it is not able to find astrometry.util:
'''
(base) [rnanni@igm tractor]$ python examples/tractor-sdss-synth.py
Traceback (most recent call last):
File "examples/tractor-sdss-synth.py", line 20, in
from tractor import *
File "/mnt/quasar2/rnanni/softwares/tractor/tractor/sfd.py", line 9, in
from astrometry.util.util import anwcs_t
File "/usr/local/astrometry/lib/python/astrometry/util/util.py", line 13, in
from . import _util
ImportError: cannot import name '_util' from 'astrometry.util' (/usr/local/astrometry/lib/python/astrometry/util/init.py)
'''

[dstndstn]

You did `make py` in the Astrometry.net code, right...?

…

On Thu, Nov 11, 2021 at 9:13 AM Rik-ad-astra ***@***.***> wrote: Ok, thanks. I see that simply following your second option (python setup.py build_ext --inplace), the issue is solved without needing the "make version". I still cannot run the test example as it seems that it is not able to find astrometry.util: ''' Traceback (most recent call last): File "examples/tractor-sdss-synth.py", line 20, in from tractor import * File "/mnt/quasar2/rnanni/softwares/tractor/tractor/sfd.py", line 9, in from astrometry.util.util import anwcs_t File "/usr/local/astrometry/lib/python/astrometry/util/util.py", line 13, in from . import _util ImportError: cannot import name '_util' from 'astrometry.util' (/usr/local/astrometry/lib/python/astrometry/util/*init*.py) (base) ***@***.*** tractor]$ python examples/tractor-sdss-synth.py Traceback (most recent call last): File "examples/tractor-sdss-synth.py", line 20, in from tractor import * File "/mnt/quasar2/rnanni/softwares/tractor/tractor/sfd.py", line 9, in from astrometry.util.util import anwcs_t File "/usr/local/astrometry/lib/python/astrometry/util/util.py", line 13, in from . import _util ImportError: cannot import name '_util' from 'astrometry.util' (/usr/local/astrometry/lib/python/astrometry/util/*init*.py) ''' — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#93 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIEH7JMTWF3TXNA6RRN5UTULPFQDANCNFSM4IOYETLA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[Rik-ad-astra]

Yeah, I think we did. Actually, I am able to import "from astrometry.util.starutil_numpy import radectolb", while "from astrometry.util.util import anwcs_t" is rising the aforementioned error. I thought that without the "make py" even the first import shouldn't work

[dstndstn]

No, the `make py` generates the *compiled* python modules. starutil_numpy is pure python, while util is compiled.

…

On Thu, Nov 11, 2021 at 9:45 AM Rik-ad-astra ***@***.***> wrote: Yeah, I think we did. Actually, I am able to import "from astrometry.util.starutil_numpy import radectolb", while "from astrometry.util.util import anwcs_t" is rising the aforementioned error. I thought that without the "make py" even the first import shouldn't work — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#93 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIEH7O4UNES3XHFIDHJS3DULPJHTANCNFSM4IOYETLA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.