A deep learning framework to help you build your species distribution models

View framework · Report Bug · Request Feature

Malpolon

Developed as part of the European GUARDEN (ID: 101060693) and MAMBO (ID: 101060639) projects, Malpolon is a framework facilitating the training and sharing of Deep Species Distribution models using various types of input covariates including bioclimatic rasters, remote sensing images, land-use rasters, etc...

If you're not a deep learning or PyTorch expert but nonetheless want to use visual deep learning models on satellite and/or bioclimatic rasters to predict the presence of species on a given area, this framework is for you.

🧭 Usage

Malpolon is destined to be used by various user profiles, some more experimented than others. To this end, we provide several examples of usage of the framework, organized by use case or scenarios. These examples can be found in the examples folder of the repository, each with a README file for more details on how to use the scripts. Additionally, check out our guide "Getting started with examples".

Here is a list of the currently available scenarios:

• Benchmarks : I want to compare the performance of different models on a given known dataset;
  or I am a potential kaggle participant on the GeoLifeClef challenge. I want to train a model on the provided datasets without having to worry about the data loading, starting from a plug-and-play example.
  • GeoLifeClef2022 : contains a fully functional example of a model training on the GeoLifeClef2022 dataset, from data download, to training and prediction.
  • GeoLifeClef2023 : contains dataloaders for the GeoLifeClef2023 dataset (different from the GLCgeolifeclef/2022 dataloaders). The training and prediction scripts are not provided.
  • GeoLifeClef2024 (pre-extracted) : contains a fully functional example of a multimodal ensemble model used to provide a strong baseline for the GeoLifeClef2024 kaggle competition. The example uses unique dataloaders and models to handle pre-extracted values from satellite patches, satellite time series and bioclimatic time series.
• Train (custom datasets) : I have a dataset of my own and I want to train a model on it. I want to be able to easily customize the training process and the model architecture.
  • Drop and play : I have an observations file (.csv) and I want to train a model on different environmental variables (rasters, satellite imagery) without having to worry about the data loading.
  • Custom dataset : I have my own dataset consisting of pre-extracted image patches and/or rasters and I want to train a model on it.
• Inference : I have an observations file (.csv) and I want to predict the presence of species on a given area using a model I trained previously and a selected dataset or a shapefile I would provide.

⚙️ Installation

To install malpolon, you will first need to install Python ≥ 3.10, and several python packages. To do so, it is best practice to create a virtual environment containing all these packages locally.

⚠️ macOS installation does not yet include instructions on how to properly set up GPU usage for GPU-enabled mac. For training purposes we recommend sticking to Linux for the time being. ⚠️

Click here to expand instructions

Requirements

Before proceeding, please make sure the following packages are installed on your system:

• Python ≥ 3.10
• pip
• git
• libgeos-dev (dependency of Python library Cartopy)
  • On Linux (Ubuntu): sudo apt install libgeos-dev
  • On MacOS: brew install geos
• cmake
  • On Linux (Ubuntu): sudo apt install cmake
  • On MacOS: brew install cmake
• cuda (if you intend to run your models on GPU)
  • CUDA Installation guide
  • CuDNN Installation guide

The following instructions show installation commands for Python 3.10, but can be adapted for any of the compatible Python versions mentioned above by simply changing the version number.

Install from PyPi

The backend side of malpolon is distributed as a package on PyPi. To install it, simply run the following command:

pip install malpolon

However, versions available on PyPi are non-experimental and possibly behind the repository's main and dev branches. To know which version you want download, please refer to the tags section of the repository and match it with PyPi. Furthermore, the PyPi package does not include the examples and the documentation. If you want to install the full repository, follow the next steps.

Install from GitHub

1. Clone the repository

Clone the Malpolon repository using git in the directory of your choice:

git clone https://github.com/plantnet/malpolon.git

---

2. Create your virtual environment

• Via virtualenv

We recommend handling your virtual environment using virtualenv (or similar) and installing the packages via pip.

First create your virtual environment using the proper python version, and activate it (note that in this example, the virtual environment "malpolon_env" will be installed in the current directory).

virtualenv -p /usr/bin/python3.10 ./malpolon_3.10
source ./malpolon_3.10/bin/activate

Once the env is activated, install the python packages listed in requirements_python3.10.txt:

pip install --upgrade setuptools
pip install -r requirements_python3.10.txt

• Via conda

⚠️ Be aware that conda recently changed its licensing and you may subject to fees, or be limited in downloads. Sources: anaconda website, datacamp blog recap ⚠️

You can also use conda to install your packages.

conda env create -f environment_python3.10.yml
conda activate malpolon_3.10

---

3. Install Malpolon as a python package

The malpolon repository can also be installed in your virtual environment as a package. This allows you to import malpolon anywhere in your scripts without having to worry about file paths. It can be installed via pip using:

cd malpolon
pip install -e .

To check that the installation went well, use the following command

python -m malpolon.check_install

which, if you have CUDA properly installed, should output something similar to

Using PyTorch version 2.1.0+cu121
CUDA available: True (version: 12.1)
cuDNN available: True (version: 8902)
Number of CUDA-compatible devices found: 1

---

The dev branch is susceptible to have more up-to-date content such as newer examples and experimental features. To switch to the dev branch locally, run:

git checkout dev

📄 Documentation

An online code documentation is available via GitHub pages at this link. This documentation is updated each time new content is pushed to the main branch.

Alternatively, you can generate the documention locally by following these steps :

1. Install the additional dependencies contained in docs/docs_requirements.txt must be installed

pip install -r docs/docs_requirements.txt

2. Generated the documentation

make -C docs html

The result can be found in docs/_build/html.

⚒️ Troubleshooting

Commonly encountered errors when using the framework are compiled here.

🚀 Contributing

Guidelines

Issues and PR templates are provided to help you start a contribution to the project.

A checking script is also provided and can run checks relative to the 2 next sections with the following command:

./checkMyCode all

Unit tests

Click here to expand instructions

When submitting, make sure the unit tests all pass without errors. These tests are located at malpolon/tests/ and can be ran all at once, with a code coverage estimation, via command line:

./checkMyCode.sh t  # or `pytest malpolon/tests/`

Specify a file path as argument to run a single test file:

./checkMyCode.sh malpolon/tests/<TEST_FILE>.py  # or `pytest malpolon/tests/<TEST_FILE>.py`

Run individual test functions via python malpolon/tests/test_<module>.py by modifying the files beforehand to call the functions you want to test with:

if __name__ == '__main__':
  test_my_function()

This is especially useful for malpolon/tests/test_examples.py which tests all the provided examples, ensuring they do not crash. However, these require having all the datasets and take a while to run. Some data you might not have local access to.
To skip a test function, add a decorator @pytest.mark.skip() above the function definition.

Linting

Click here to expand instructions

Likewise, do care about writing a clean code. The project uses flake8, Pylint and Pydocstyle to check the good formatting and documentation of your code. To run linters check on your code you can either run each of these library independently or use the checking script:

./checkMyCode.sh l

Run linters on non-test file(s) :

./checkMyCode.sh <FILE_PATH_1> <FILE_PATH_2>

🚆 Roadmap

This roadmap outlines the planned features and milestones for the project. Please note that the roadmap is subject to change and may be updated as the project progress.

Click here to toggle roadmap

• Data support
  • Images (pre-extracted patches)
  • Rasters
  • Time series
    • Via GLC23 loaders (.csv)
    • Via generic loader
  • Shapefiles
  • Fuse several data types in one training
• Deep learning tasks
  • Binary classification
  • Multi-class classification
  • Multi-label classification
  • Regression (abundance prediction)
  • Ordinal
• Supported models
  • CNN
  • LSTM
  • Transformers
• Training flexibility
  • Add model head/tail modifiers
  • Allow easy (un-)freeze of layers
  • Allow dataset intersections and unions
• Allow data parallel training
  • Multithreading
  • Multiprocessing
    • Issues may arise depending on hardware

Libraries

Here is an overview of the main Python libraries used in this project.

• - To handle deep learning loops and dataloaders
• - Deep learning framework which simplifies the usage of PyTorch elements
• - For common computational operations
• - To handle data rasters
• - For displaying purposes
• - To handle models' hyperparameters
• - To handle geographical data

Acknowledgments

This work is made possible through public financing by the European Commission on european projects MAMBO and GUARDEN.

This work is currently under development and maintained by the Pl@ntNet team within the INRIA research institute.

Licensing

This framework is distributed under the MIT license, as is the Pl@ntNet project. See LICENSE.md for more information.

Citation & credits

Malpolon is a project developed by the Pl@ntNet team within the INRIA research institute. If you use this framework in your research, please cite our software paper:

@misc{larcher2024malpolonframeworkdeepspecies,
      title={MALPOLON: A Framework for Deep Species Distribution Modeling},
      author={Theo Larcher and Lukas Picek and Benjamin Deneu and Titouan Lorieul and Maximilien Servajean and Alexis Joly},
      year={2024},
      eprint={2409.18102},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2409.18102},
}

Authors include :

• Théo Larcher (current lead developer) (email)
• Maximilien Servajean
• Alexis Joly

Former developpers include :

• Titouan Lorieul (former lead developer) (email)