Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 1.1.1 of PyMEOS CFFI #47

Merged
merged 36 commits into from
Apr 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
73f5c5c
Extend testing workflow to test on mac (intel and arm) (#42)
Diviloper Apr 7, 2024
bf30011
Add pymeos cffi build workflow
Diviloper Apr 12, 2024
e1035b6
Always get meos sources from default branch
Diviloper Apr 12, 2024
a21adfc
Set working directory of wheel build step
Diviloper Apr 12, 2024
e210b3c
Add ld_path to exports before building wheels
Diviloper Apr 12, 2024
845f800
Install meos in linux wheel build container
Diviloper Apr 12, 2024
b11f880
Set working directory of build_sdist steps
Diviloper Apr 12, 2024
30e2d7b
Add checkout in build_wheels
Diviloper Apr 12, 2024
74dd8b6
Fix cibw linux script
Diviloper Apr 12, 2024
f98a940
Fix test command
Diviloper Apr 12, 2024
19fb8a7
Add log in sdist
Diviloper Apr 12, 2024
31177b8
Use proper build command for sdist
Diviloper Apr 12, 2024
908e588
Install build module
Diviloper Apr 12, 2024
b4430c1
Fix artifact path for sdist
Diviloper Apr 12, 2024
c0384db
Fix wheel artifact folder
Diviloper Apr 12, 2024
bd5b30e
Skip PyPy builds on linux
Diviloper Apr 12, 2024
1f8514a
Add logs
Diviloper Apr 12, 2024
d700f1b
Update library and include dir functions to check for existence of pa…
Diviloper Apr 12, 2024
5596e26
Merge macOS MEOS steps.
Diviloper Apr 12, 2024
7f4c022
Add QEMU to compile for aarch64
Diviloper Apr 12, 2024
f4632ee
Add wheel testing job
Diviloper Apr 12, 2024
9e8f88d
Remove LD exports from pytest step
Diviloper Apr 12, 2024
0e3632b
Add upload to pypi job
Diviloper Apr 12, 2024
11884cc
Remove aarch64 for linux
Diviloper Apr 12, 2024
27d7838
Create GitHub Release job
Diviloper Apr 12, 2024
379e987
Disable musllinux builds
Diviloper Apr 12, 2024
64cd3a8
Remove QEMU since no cross-compilation is being done anymore
Diviloper Apr 12, 2024
aa711ea
Copy PROJ data in setup.py when PACKAGE_DATA is set to be able to gen…
Diviloper Apr 13, 2024
ecf6cab
Pin PROJ versions and set necessary env variables to add proj data to…
Diviloper Apr 13, 2024
14459f2
Set PROJ_DATA and PROJ_LIB in meos_initialize if they are undefined
Diviloper Apr 13, 2024
3aa8411
Build json-c from sources to get version 0.17 instead of default vers…
Diviloper Apr 13, 2024
b6e18a7
Add json-c path to env for linux
Diviloper Apr 13, 2024
49fc7f5
Fix pypi publish action name
Diviloper Apr 13, 2024
d5d71ec
Only set PROJ_DATA if proj_data folder exists (only whould exist on a…
Diviloper Apr 13, 2024
aabf94d
Use real pypi server to make build releases
Diviloper Apr 13, 2024
e0e3d79
Run PyPI step only on MobilityDB/PyMEOS repo
Diviloper Apr 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
222 changes: 222 additions & 0 deletions .github/workflows/build_pymeos_cffi.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,222 @@
name: Build PyMEOS CFFI

on:
create:
tags:
- "pymeos-cffi-[0-9]+.[0-9]+.[0-9]+*"

jobs:
build_sdist:
name: Build PyMEOS CFFI source distribution
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Install Python
uses: actions/setup-python@v5
with:
python-version: 3.8
cache: "pip"

- name: Setup pip
run: |
python -m pip install --upgrade pip
python -m pip install build

- name: Build sdist
working-directory: pymeos_cffi
run: |
python -m build -s
ls -l dist

- uses: actions/upload-artifact@v4
with:
name: pymeos_cffi-sdist
path: ./pymeos_cffi/dist/pymeos_cffi-*.tar.gz

build_wheels:
name: Build PyMEOS CFFI for ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ ubuntu-latest, macos-13, macos-14 ]
include:
- ld_prefix: "/usr/local"
- os: macos-14
ld_prefix: "/opt/homebrew"

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Update brew
if: matrix.os == 'macos-13'
# Necessary to avoid issue with macOS runners. See
# https://github.com/actions/runner-images/issues/4020
run: |
brew reinstall [email protected] || brew link --overwrite [email protected]
brew reinstall [email protected] || brew link --overwrite [email protected]
brew update

- name: Get dependencies from homebrew (cache)
uses: tecolicom/actions-use-homebrew-tools@v1
if: runner.os == 'macOS'
with:
tools: cmake libpq proj json-c gsl geos

- name: Get PROJ version
id: proj_version
if: runner.os == 'macOS'
run: |
proj_version=$(brew list --versions proj)
proj_version=${proj_version#* }
echo "proj_version=$proj_version" >> $GITHUB_OUTPUT

- name: Install MEOS
if: runner.os == 'macOS'
run: |
git clone --depth 1 https://github.com/MobilityDB/MobilityDB
mkdir MobilityDB/build
cd MobilityDB/build
cmake .. -DMEOS=on
make -j
sudo make install

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: 3.8
cache: "pip"

- name: Install cibuildwheel
run: python -m pip install cibuildwheel==2.17.0

- name: Set PROJ_DATA (macOS)
if: runner.os == 'macOS'
run: |
PROJ_DATA=${{ matrix.ld_prefix }}/Cellar/proj/${{ steps.proj_version.outputs.proj_version }}/share/proj
echo "PROJ_DATA=$PROJ_DATA" >> $GITHUB_ENV

- name: Set PROJ_DATA and JSON-C path (Linux)
if: runner.os == 'Linux'
run: |
PROJ_DATA=/usr/proj81/share/proj
echo "PROJ_DATA=$PROJ_DATA" >> $GITHUB_ENV
echo "LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib64" >> $GITHUB_ENV

- name: Build wheels
working-directory: pymeos_cffi
run: |
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${{ matrix.ld_prefix }}/lib
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:${{ matrix.ld_prefix }}/lib
export PACKAGE_DATA=1
python -m cibuildwheel --output-dir wheelhouse
env:
# Disable PyPy builds on Linux since shapely has no built distributions for them
# Disable builds on musllinux
# Disable builds in linux architectures other than x86_64
CIBW_SKIP: "pp*-manylinux* *musllinux*"
CIBW_ARCHS_LINUX: "x86_64"
CIBW_ENVIRONMENT_PASS_LINUX: PACKAGE_DATA LD_LIBRARY_PATH PROJ_DATA
CIBW_BEFORE_ALL_LINUX: >
yum -y install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm &&
yum -y update &&
yum -y install gcc gcc-c++ make cmake postgresql13-devel proj81-devel geos39-devel gsl-devel &&
git clone --branch json-c-0.17 --depth 1 https://github.com/json-c/json-c &&
mkdir json-c-build &&
cd json-c-build &&
cmake ../json-c &&
make &&
make install &&
git clone --depth 1 https://github.com/MobilityDB/MobilityDB &&
mkdir MobilityDB/build &&
cd MobilityDB/build &&
cmake .. -DMEOS=on -DGEOS_INCLUDE_DIR=/usr/geos39/include/ -DGEOS_LIBRARY=/usr/geos39/lib64/libgeos_c.so -DGEOS_CONFIG=/usr/geos39/bin/geos-config -DPROJ_INCLUDE_DIRS=/usr/proj81/include/ -DPROJ_LIBRARIES=/usr/proj81/lib/libproj.so &&
make -j &&
make install

CIBW_TEST_COMMAND: "python -c \"from pymeos_cffi import meos_initialize, meos_finalize; meos_initialize('UTC'); meos_finalize()\""

- uses: actions/upload-artifact@v4
with:
name: pymeos_cffi-wheels-${{ matrix.os }}
path: ./pymeos_cffi/wheelhouse/*.whl

test_wheels:
name: Test PyMEOS CFFI wheel - Python ${{ matrix.python-version }} on ${{ matrix.os }}
needs: build_wheels
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
python-version: [ "3.8", "3.9", "3.10", "3.11", "3.12" ]
os: [ ubuntu-latest, macos-13, macos-14 ]
exclude:
# Necessary due to issue with macOS runners. See
# https://github.com/actions/setup-python/issues/808
# Can be removed once this PR is merged:
# https://github.com/actions/python-versions/pull/259
- os: macos-14
python-version: "3.8"
- os: macos-14
python-version: "3.9"

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Download wheels
uses: actions/download-artifact@v4
with:
name: pymeos_cffi-wheels-${{ matrix.os }}
path: ./pymeos_cffi_wheels

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"

- name: Install PyMEOS dependencies
run: |
python -m pip install --upgrade pip
pip install -f ./pymeos_cffi_wheels pymeos_cffi
pip install -r pymeos/dev-requirements.txt

- name: Test PyMEOS with pytest
working-directory: pymeos
run: pytest

upload_pypi:
name: Upload to PyPI
needs: [ test_wheels, build_sdist ]
runs-on: ubuntu-22.04
if: github.repository == 'MobilityDB/PyMEOS'
steps:
- uses: actions/download-artifact@v4
with:
path: ./dist
merge-multiple: true

- uses: pypa/gh-action-pypi-publish@release/v1
with:
user: __token__
password: ${{ secrets.TEST_PYPI_API_TOKEN }}
skip_existing: true

create_release:
name: Create GitHub Release
needs: [ test_wheels, build_sdist ]
runs-on: ubuntu-22.04
steps:
- uses: actions/download-artifact@v4
with:
path: ./dist
merge-multiple: true

- name: Create Release
uses: softprops/action-gh-release@v2
with:
files: ./dist/*
70 changes: 58 additions & 12 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,56 +3,102 @@ name: Test PyMEOS
on:
push:
branches: [ "develop", "master" ]
paths-ignore:
- "docs/**"
- ".readthedocs.yml"
- "README.md"
pull_request:
branches: [ "develop", "master" ]
paths-ignore:
- "docs/**"
- ".readthedocs.yml"
- "README.md"

jobs:
build:

runs-on: ubuntu-22.04
test:
name: Test PyMEOS - Python ${{ matrix.python-version }} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
python-version: [ "3.8", "3.9", "3.10", "3.11", "3.12" ]
os: [ ubuntu-latest, macos-13, macos-14 ]
exclude:
# Necessary due to issue with macOS runners. See
# https://github.com/actions/setup-python/issues/808
# Can be removed once this PR is merged:
# https://github.com/actions/python-versions/pull/259
- os: macos-14
python-version: "3.8"
- os: macos-14
python-version: "3.9"
include:
- ld_path: "/usr/local/lib"
- os: macos-14
ld_path: "/opt/homebrew/lib"

steps:
- uses: actions/checkout@v4
- name: Checkout
uses: actions/checkout@v4

- uses: awalsh128/cache-apt-pkgs-action@latest
- name: Get dependencies from apt (cache)
uses: awalsh128/cache-apt-pkgs-action@latest
if: runner.os == 'Linux'
with:
packages: build-essential cmake postgresql-server-dev-14 libproj-dev libjson-c-dev libgsl-dev libgeos-dev
version: 1.0

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
- name: Update brew
if: matrix.os == 'macos-13'
# Necessary to avoid issue with macOS runners. See
# https://github.com/actions/runner-images/issues/4020
run: |
brew reinstall [email protected] || brew link --overwrite [email protected]
brew reinstall [email protected] || brew link --overwrite [email protected]
brew update

- name: Get dependencies from homebrew (cache)
uses: tecolicom/actions-use-homebrew-tools@v1
if: runner.os == 'macOS'
with:
python-version: ${{ matrix.python-version }}
cache: "pip"
tools: cmake libpq proj json-c gsl geos

- name: Install MEOS
- name: Fetch MEOS sources
env:
BRANCH_NAME: ${{ github.base_ref || github.ref_name }}
run: |
git clone --branch ${{ env.BRANCH_NAME }} --depth 1 https://github.com/MobilityDB/MobilityDB

- name: Install MEOS
run: |
mkdir MobilityDB/build
cd MobilityDB/build
cmake .. -DMEOS=on
make -j
sudo make install

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"

- name: Install python dependencies
run: |
python -m pip install --upgrade pip
pip install -r pymeos_cffi/dev-requirements.txt
pip install -r pymeos/dev-requirements.txt

- name: Install pymeos_cffi
run: |
cd pymeos_cffi
python ./pymeos_cffi/builder/build_header.py
python ./pymeos_cffi/builder/build_pymeos_functions.py
pip install .
cd ..

- name: Test PyMEOS with pytest
run: |
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${{ matrix.ld_path }}
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:${{ matrix.ld_path }}
cd pymeos
pytest
Loading
Loading