Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare python publish #27

Merged
merged 13 commits into from
Nov 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Publish Python 🐍 distribution 📦 to PyPI

on:
release:
types: [published]

permissions:
contents: read

jobs:
publish-to-pypi:
name: >-
Publish Python 🐍 distribution 📦 to PyPI
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/remotebmi
permissions:
id-token: write # IMPORTANT: mandatory for trusted publishing
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: "pip"
- name: Install build
run: pip install build
- name: Build package
run: python3 -m build
- name: Publish distribution 📦 to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
59 changes: 54 additions & 5 deletions .github/workflows/python.yml
BSchilperoort marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,30 +1,36 @@
name: Python
name: Python 🐍

on:
push:
paths:
- "openapi.yaml"
- "python/remotebmi/**"
- "python/tests/**"
- "python/pyproject.toml"
- .github/workflows/python.yml
pull_request:
paths:
- "openapi.yaml"
- "python/remotebmi/**"
- "python/tests/**"
- "python/pyproject.toml"
- .github/workflows/python.yml
types: [opened, reopened]

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

permissions:
id-token: write # This is required for codecov

defaults:
run:
working-directory: python

jobs:
lint-format:
name: formatting and type checking 🔍️
runs-on: ubuntu-latest

steps:
Expand All @@ -33,37 +39,80 @@ jobs:
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
python-version: "3.13"
cache: "pip"
- name: Install ruff
run: pip install ruff
- name: Run ruff check
run: ruff check
- name: Run ruff format
run: ruff format --check
typing:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.13"
cache: "pip"
- name: Install remotebmi
run: pip install .[dev] # dev too so tests can be type checked
- name: Install mypy
run: pip install mypy numpy
- name: Run mypy
run: mypy .
test:
name: run test suite 🐛
runs-on: ${{ matrix.os }}
strategy:
fail-fast: true
matrix:
os: ['ubuntu-latest', 'macos-latest', 'windows-latest']
python-version: ['3.10', '3.13']
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"
- name: Install dependencies
run: pip install -e .[dev]
- name: Run tests
run: pytest
cov:
name: compute and upload coverage
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
python-version: "3.13"
cache: "pip"
- name: Install dependencies
run: pip install -e .[dev]
- name: Run tests
run: pytest tests
run: pytest --cov --cov-report=xml
- name: Upload coverage to Codecov
BSchilperoort marked this conversation as resolved.
Show resolved Hide resolved
uses: codecov/codecov-action@v4
with:
file: coverage.xml
use_oidc: true
build:
name: build the package 🛠️
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
python-version: "3.13"
cache: "pip"
- name: Install build
run: pip install build
Expand Down
26 changes: 26 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# YAML 1.2
---
authors:
-
affiliation: "Netherlands eScience Center"
family-names: Verhoeven
given-names: Stefan
orcid: "https://orcid.org/0000-0002-5821-2060"
-
affiliation: "Netherlands eScience Center"
family-names: Schilperoort
given-names: Bart
orcid: "https://orcid.org/0000-0003-4487-9822"

cff-version: "1.2.0"
keywords:
- bmi
license: "Apache-2.0"
message: "If you use this software, please cite it using these metadata."
repository-code: "https://github.com/eWaterCycle/remotebmi"
title: remotebmi Python/Julia/R packages
identifiers:
- description: Latest version of software
type: doi
value: "TODO fill with concept DOI after fisrt release"
...
78 changes: 78 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Contributor Covenant Code of Conduct

## Our Pledge

In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our
project and our community a harassment-free experience for everyone,
regardless of age, body size, disability, ethnicity, gender identity and
expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity and
orientation.

## Our Standards

Examples of behavior that contributes to creating a positive environment
include:

- Using welcoming and inclusive language
- Being respectful of differing viewpoints and experiences
- Gracefully accepting constructive criticism
- Focusing on what is best for the community
- Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

- The use of sexualized language or imagery and unwelcome sexual
attention or advances
- Trolling, insulting/derogatory comments, and personal or political
attacks
- Public or private harassment
- Publishing others\' private information, such as a physical or
electronic address, without explicit permission
- Other conduct which could reasonably be considered inappropriate in
a professional setting

## Our Responsibilities

Project maintainers are responsible for clarifying the standards of
acceptable behavior and are expected to take appropriate and fair
corrective action in response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit,
or reject comments, commits, code, wiki edits, issues, and other
contributions that are not aligned to this Code of Conduct, or to ban
temporarily or permanently any contributor for other behaviors that they
deem inappropriate, threatening, offensive, or harmful.

## Scope

This Code of Conduct applies both within project spaces and in public
spaces when an individual is representing the project or its community.
Examples of representing a project or community include using an
official project e-mail address, posting via an official social media
account, or acting as an appointed representative at an online or
offline event. Representation of a project may be further defined and
clarified by project maintainers.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may
be reported by contacting the project team at
<[email protected]>. All complaints will be reviewed and
investigated and will result in a response that is deemed necessary and
appropriate to the circumstances. The project team is obligated to
maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted
separately.

Project maintainers who do not follow or enforce the Code of Conduct in
good faith may face temporary or permanent repercussions as determined
by other members of the project\'s leadership.

## Attribution

This Code of Conduct is adapted from the [Contributor
Covenant](https://www.contributor-covenant.org), version 1.4, available
at
<https://www.contributor-covenant.org/version/1/4/code-of-conduct.html>
88 changes: 88 additions & 0 deletions python/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Remote BMI

[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](CODE_OF_CONDUCT.md)
[![Codecov test coverage](https://codecov.io/gh/eWaterCycle/remotebmi/graph/badge.svg)](https://app.codecov.io/gh/eWaterCycle/remotebmi)

The [Basic Model Interface (BMI)](https://bmi.readthedocs.io/en/stable/) is a standard interface for models.
The interface is available in different languages and a [language agnosting version in SIDL](https://github.com/csdms/bmi/blob/stable/bmi.sidl).

To have a consumer of the model and the provider of the model seperated you can use [grpc4bmi](https://grpc4bmi.readthedocs.io/), but this only works on languages that have a grpc implementation.
This Python package replaced the gRPC protocol with an REST API.
The [REST API specification](https://github.com/eWaterCycle/remotebmi/blob/main/openapi.yaml) is in the [OpenAPI](https://swagger.io/specification/) format.

Remotebmi is available in other languages see [here](https://github.com/eWaterCycle/remotebmi?tab=readme-ov-file#structure).

## Difference from BMI

- Request body and response body are in JSON format
- On errors you get 4xx and 5xx responses with [Problem Details](https://tools.ietf.org/html/rfc7807) as response body
- Variable names must be URL safe
BSchilperoort marked this conversation as resolved.
Show resolved Hide resolved
- Variable type must be in enum.
- get_value_ptr function is not available

## Consumer

Installation

```shell
pip install remotebmi
```

A client can connect to a running server with the following code.

```python
from remotebmi.client.client import RemoteBmiClient

model = RemoteBmiClient('http://localhost:50051')
# Now you can use the BMI methods on model
# for example
model.initialize('config.file')
model.update()
model.get_value('var_name')
```

A client can also start a [Apptainer](https://apptainer.org) container containing the model and the server.

```python
from remotebmi.client.apptainer import BmiClientApptainer

model = BmiClientApptainer('my_model.sif', work_dir='/tmp')
```

The client picks a random port and expects the container to run the BMI web server on that port.
The port is passed to the container using the `BMI_PORT` environment variable.

A client can also start a [Docker](https://docs.docker.com/engine/) container containing the model and the server.

```python
from remotebmi.client.docker import BmiClientDocker

model = BmiClientDocker('ewatercycle/wflowjl:0.7.3', work_dir='/tmp')
```

The BMI web server inside the Docker container should be running on port 50051.
If the port is different, you can pass the port as the `image_port` argument to the `BmiClientDocker` constructor.

## Provider

Given you have a model class called `MyModel` in a package `mypackage` then the web service can be started with the following command.

```shell
BMI_MODULE=mypackage BMI_CLASS=MyModel run-bmi-server
```

For example [leakybucket](https://github.com/eWaterCycle/leakybucket-bmi):

```shell
pip install leakybucket
BMI_MODULE=leakybucket.leakybucket_bmi BMI_CLASS=LeakyBucketBmi run-bmi-server
```

and the client can connect to it with the following code.

```python
> from remotebmi.client.client import RemoteBmiClient
> client = RemoteBmiClient('http://localhost:50051')
> client.get_component_name()
leakybucket
```
Loading