remove linking to libpython on linux/OSX

[y0ast]

This allows building of dlib on linux without libpython and is the second step in enabling #138.

Depending on the merge order of this and #701 I'll refactor one or the other.

Also relevant for @matthew-brett

In order to build a wheel on manylinux:

wget http://www.cmake.org/files/v2.8/cmake-2.8.12.tar.gz --no-check-certificate
tar -zxvf cmake-2.8.12.tar.gz 
cd cmake-2.8.12
./bootstrap
make
make install
cd ..
wget -O boost.tar.gz https://dl.bintray.com/boostorg/release/1.64.0/source/boost_1_64_0.tar.gz --no-check-certificate
tar -zxvf boost.tar.gz
cd boost_1_64_0
./bootstrap.sh --with-python=/opt/python/cp27-cp27mu/bin/python
./b2 install
cd ..
git clone https://github.com/davisking/dlib.git
cd dlib
export CPATH=/opt/python/cp27-cp27mu/include/python2.7
export BOOST_ROOT=/usr/local/lib
/opt/python/cp27-cp27mu/bin/python setup.py bdist_wheel --yes DLIB_NO_GUI_SUPPORT

Output of auditwheel:

[root@451ff0031654 dist]# auditwheel show dlib-19.4.99-cp27-cp27mu-linux_x86_64.whl

dlib-19.4.99-cp27-cp27mu-linux_x86_64.whl is consistent with the
following platform tag: "linux_x86_64".

The wheel references external versioned symbols in these system-
provided shared libraries: libgcc_s.so.1 with versions {'GCC_4.0.0',
'GCC_3.0'}, libc.so.6 with versions {'GLIBC_2.3', 'GLIBC_2.2.5'},
libm.so.6 with versions {'GLIBC_2.2.5'}, libstdc++.so.6 with versions
{'CXXABI_1.3.1', 'CXXABI_1.3', 'GLIBCXX_3.4'}, libpthread.so.0 with
versions {'GLIBC_2.2.5', 'GLIBC_2.3.2'}

The following external shared libraries are required by the wheel:
{
    "libboost_python.so.1.64.0": "/usr/local/lib/libboost_python.so.1.64.0",
    "libc.so.6": "/lib64/libc-2.5.so",
    "libdl.so.2": "/lib64/libdl-2.5.so",
    "libgcc_s.so.1": "/lib64/libgcc_s-4.1.2-20080825.so.1",
    "libm.so.6": "/lib64/libm-2.5.so",
    "libnsl.so.1": "/lib64/libnsl-2.5.so",
    "libpthread.so.0": "/lib64/libpthread-2.5.so",
    "librt.so.1": "/lib64/librt-2.5.so",
    "libsqlite3.so.0": "/usr/local/lib/libsqlite3.so.0.8.6",
    "libstdc++.so.6": "/usr/lib64/libstdc++.so.6.0.8",
    "libutil.so.1": "/lib64/libutil-2.5.so"
}

In order to achieve the tag platform tag "manylinux1_x86_64" the
following shared library dependencies will need to be eliminated:

libboost_python.so.1.64.0, libsqlite3.so.0
[root@451ff0031654 dist]# auditwheel repair dlib-19.4.99-cp27-cp27mu-linux_x86_64.whl
Repairing dlib-19.4.99-cp27-cp27mu-linux_x86_64.whl
Grafting: /usr/local/lib/libboost_python.so.1.64.0 -> dlib/.libs/libboost_python-241685b5.so.1.64.0
Grafting: /usr/local/lib/libsqlite3.so.0.8.6 -> dlib/.libs/libsqlite3-0a3628e4.so.0.8.6
Setting RPATH: dlib/dlib.so to "$ORIGIN/.libs"
Previous filename tags: linux_x86_64
New filename tags: manylinux1_x86_64
Previous WHEEL info tags: cp27-cp27mu-linux_x86_64
New WHEEL info tags: cp27-cp27mu-manylinux1_x86_64

Fixed-up wheel written to /mnt/wheels/dlib/dist/wheelhouse/dlib-19.4.99-cp27-cp27mu-manylinux1_x86_64.whl

[davisking]

This makes regular builds fail to compile. Any proposed updates to dlib must not interfere with existing common usage.

[y0ast]

Ah, I am sorry, this indeed breaks the OS X build. So #701 would need to be merged first and this needs to be edited to only add the -shared flag in the case of linux.

[davisking]

It breaks on linux too. I haven't tried it on windows, but I would bet money it breaks it there too.

Please submit a PR that doesn't break anything. I also don't understand what this PR is accomplishing. Maybe because I'm not familiar with manylinux.

[y0ast]

This PR tries to accomplish fixing the building in a way that is compatible with python wheels (see PEP-427: https://www.python.org/dev/peps/pep-0427/). Wheels have become a very popular way to distribute complex python packages (see for example numpy/scipy/pandas/tensorflow/pytorch/scikit-learn who also distribute their package this way). It allows the user to install dlib from pypi in a matter of seconds instead of having to go through a compilation phase.

In order to be able to build a wheel one needs to precompile all assets, which is done on manylinux to be as widely compatible as possible.

The travis builds are passing and I am very curious to hear why it fails building for you.

I propose we do #701 first, so I can update this PR to work with every platform. If you prefer I can also take the changes of that PR and integrate it into this one.

[davisking]

The travis builds weren't building the pyhton extension. I just updated the travis config to include them though, so if you trigger a rebuild on travis you will see your build fail.

But yes, binary linux packages sound great. But whatever changes are made, it still needs to be possible for people to download dlib and run cmake or setup.py to build it themselves.

[y0ast]

Ah, that's great, thanks! I'll update this PR tomorrow to include the OSX bit, which will then automatically trigger the rebuild.

These changes are not meant to break running cmake/setup.py manually and I'm keen to fix them.

[davisking]

Cool, sounds good :)

[y0ast]

I cleaned up all the building files and ran:

export CPATH=/<path-to-python>/python/2.7.13/Frameworks/Python.framework/Versions/2.7/include/python2.7
cd tools/python
mkdir build
cd build
cmake ..
make install
cd ../../../python_examples
./svm_rank.py

output:

$ ./svm_rank.py
Ranking score for a relevant vector:     0.5
Ranking score for a non-relevant vector: -0.5
ranking_accuracy: 1  mean_ap: 1
Weights: 0.5
-0.5
Cross validation results: ranking_accuracy: 1  mean_ap: 1
Ranking score for a relevant vector:     0.5
Ranking score for a non-relevant vector: -0.5

[davisking]

It is not reasonable to expect users to put export CPATH=/<path-to-python>/python/2.7.13/Frameworks/Python.framework/Versions/2.7/include/python2.7 or any similar thing in front of their compile commands. Many users don't even know how many versions of python they have installed, let alone where those versions reside on their filesystems.

[y0ast]

I think I have found a way to allow users to build without specifying CPATH but also not requiring things that don't exist on a manylinux, by using find_package but not REQUIRE.

Please review the changes and let me know what you think.

[davisking]

Cool, looks good. Thanks for the PR :)

[matthew-brett]

Great - many thanks for all your hard work on this.