Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding omero-test-infra CI #2

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/workflows/omero_test_infra.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Install and test and OMERO plugin e.g. a Web app, a CLI plugin or a library
#
# This workflow will install omero-test-infra, start an OMERO environment
# including database, server and web deployment, configure the OMERO plugin
# and run integration tests.
#
# 1. Set up the stage variable depending on the plugin. Supported stages
# are: app, cli, scripts, lib, srv
#
# 2. Adjust the cron schedule as necessary

name: OMERO
on:
push:
pull_request:
schedule:
- cron: '0 0 * * 0'

jobs:
test:
name: Run integration tests against OMERO
runs-on: ubuntu-latest
env:
STAGE: app
steps:
- uses: actions/checkout@v4
- name: Checkout omero-test-infra
uses: actions/checkout@master
with:
repository: ome/omero-test-infra
path: .omero
ref: ${{ secrets.OMERO_TEST_INFRA_REF }}
- name: Build and run OMERO tests
run: .omero/docker $STAGE
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.properties
107 changes: 107 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,110 @@ This repository contains the ONTOP module for the Image Data Resource (IDR) that
This project was developed with support from the Biohackathon 2024.

<img src="https://2024.biohackathon.org/images/bh24-logo.png" alt="Biohackathon 2024" width="200">

## Howto Ontop
### Software
- Protege: https://protege.stanford.edu/software.php#desktop-protege
- Unpack on your system into a writable directory
- Download ontop command line client (ontop-cli-x.y.z.zip) and ontop plugin for Protege (it.unibz.inf.ontop.protege-x.y.z.jar) from https://github.com/ontop/ontop/releases/latest
- Unpack ontop-cli zip
- Save ontop plugin into your Protege's plugin directory.
- Download postgresql jdbc driver for your system's java version: https://jdbc.postgresql.org/download/
- Save the jdbc driver into the jdbc/ subdirectory of the unpacked ontop-cli archive.

### Configure omero postgres db
If you run ontop on a different host than your omero instance, you have to allow remote tcp/ip access to the latter via port 5432. By default, this is disabled and only connections from localhost are permitted.
To this end, edit these two files (paths may vary according to host OS, on debian/ubuntu servers it will likely be */etc/postgresql/VERSION/main/*).
- *postgres.conf*
either allow all connections
```
listen_addresses = '*' # what IP address(es) to listen on;
```
or specify clients as comma separated values:
```
listen_addresses = '192.168.1.10,172.5.16.4,localhost' # what IP address(es) to listen on;
```

- *pg_hba.conf*
Adding these two lines worked for me but may be too permissive in your situation (in particular if your omero is not behind a firewall):
```
host all all 0.0.0.0/0 md5
host all all ::/0 md5
```

- restart your postgres service, i.e.
```
sudo service postgresql restart
```

### Test the connection
- Launch Protege
- If not present, add ontop Tabs (Menu -> Window -> Tabs -> Ontop Mappings)
- Configure postgres jdbc driver:
- Select Menu -> File -> Preferences -> JDBC Drivers
- Click Add
- Select "org.postgresql.Driver" from Class name dropdown menu
- Select postgres jdbc driver jar file downloaded earlier from the ontop-cli jdbc directory
- Setup connection:
- In the Ontop Mappings Tab, select "Connection parameters" subtab
- Enter connection URL: "jdbc:postgresql://your.omero.url:5432/omero"
- Enter a db username with at least read access to all tables in the omero db.
- Enter db user's password
- Select "org.postgresql.Driver" as JDBC driver class.
- Click test connection
- On error: Check postgres config on omero host and connection details.
- On success: Hurray!

### Crunch time: Your first mapping
#### Add entity and property to active ontology
Add one class (e.g. `:Dataset`) and one data property (e.g. `:name`) to the ontology.

More general, tables are mapped to classes, table index columns are mapped to
class instances (triple subjects), table columns headers are mapped to
properties (table predicates), table column values are mapped to property values
(table subjects).

#### Define mapping
- Select "Mapping manager" tab
- Click "New"
- Enter
```ttl
:dataset/{id} a :Dataset;
:name {name} .
```
into the field "Target" and
```sql
select id, name from dataset;
```
into the "Source" field.

You may want to test your query by clicking the "Execute the sql query" which will populate the "SQL Query results" field.

### SPARQL query
Select the "Ontop SPARQL" tab. Click "Prefixes" and select the base prefix and further prefixes as needed.
In the query editor enter
```sparql
select * where {
?subj a :Dataset;
:name ?name .
}
```
And click "Execute". Find your omero dataset ids and names in the SPARQL results field and observe the generated SQL query in the "SQL tranlation" tab.

### Save mapping, ontology, and properties
Select Menu -> Save (or Save as). Select "Turtle syntax" for the file format and
select a file name. This filename will serve as the basis for three files:
*<filename>.properties* (db connection settings), *<filename>.ttl* (ontology),
and *<filename>.obda* (mapping definition in ontop format).

### Launch ontop SPARQL endpoint
In a terminal, navigate to the directory where you just saved your mapping definition files to.
In that directory, run

```console
path/to/ontop-cli/ontop endpoint -m <filename>.obda -t <filename>.ttl -p <filename>.properties
```

**replace** *<filename>* with the actual filename from above.

Open your webbrowser at http://localhost:8080 where a beautiful SPARQL frontend awaits your queries. Sparqly happy queries!
33 changes: 33 additions & 0 deletions mpi_sandbox/omemap.obda
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
[PrefixDeclaration]
: https://www.openmicroscopy.org/omemap/
owl: http://www.w3.org/2002/07/owl#
rdf: http://www.w3.org/1999/02/22-rdf-syntax-ns#
xml: http://www.w3.org/XML/1998/namespace
xsd: http://www.w3.org/2001/XMLSchema#
obda: https://w3id.org/obda/vocabulary#
rdfs: http://www.w3.org/2000/01/rdf-schema#
omemap: https://www.openmicroscopy.org/omemap#
ome_core: http://www.openmicroscopy.org/rdf/2016-06/ome_core/

[MappingDeclaration] @collection [[
mappingId MAPID-ad72dafe19404fc2a460e734b0a2a0ee
target ome_core:Dataset/{id} a ome_core:Dataset ; rdfs:label {name}^^xsd:string ; rdfs:comment {description}^^xsd:string ; ome_core:annotation ome_core:Annotation/{child} .
source select dataset.id, datasetannotationlink.child, dataset.name, dataset.description from datasetannotationlink join dataset on datasetannotationlink.parent = dataset.id;

mappingId MAPID-a4cb3fa530c84c5b90ddb5e81cbbcb72
target ome_core:Annotation/{id} a ome_core:Annotation ; rdfs:comment {textvalue}^^xsd:string .
source select id, textvalue from annotation;

mappingId MAPID-a5fcd77a0a164d7fb12ac427c70e93a0
target ome_core:TagAnnotation/{id} a ome_core:TagAnnotation ; rdfs:comment {textvalue}^^xsd:string .
source select * from annotation where discriminator='/basic/text/tag/' order by id

mappingId MAPID-e7112763f8224ae6b61c7ce5665fe204
target ome_core:Image/{id} a ome_core:Image ; rdfs:label {name}^^xsd:string ; rdfs:comment {description}^^xsd:string ; ome_core:annotation ome_core:Annotation/{child} .
source select image.id, image.name, image.description, imageannotationlink.child from image join imageannotationlink on image.id=imageannotationlink.parent;

mappingId MAPID-3d5abba8f4834197b0edbd8d667802b3
target ome_core:Project/{id} a ome_core:Project ; rdfs:label {name}^^xsd:string ; rdfs:comment {description}^^xsd:string ; ome_core:dataset ome_core:Dataset/{child} .
source select project.id, project.name, project.description, projectdatasetlink.child from project join projectdatasetlink on project.id = projectdatasetlink.parent;
]]

25 changes: 25 additions & 0 deletions mpi_sandbox/omemap.ttl
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
@prefix : <https://www.openmicroscopy.org/omemap/> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix xml: <http://www.w3.org/XML/1998/namespace> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix obda: <https://w3id.org/obda/vocabulary#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix omemap: <https://www.openmicroscopy.org/omemap#> .
@prefix ome_core: <http://www.openmicroscopy.org/rdf/2016-06/ome_core/> .
@base <https://www.openmicroscopy.org/omemap/> .

<https://www.openmicroscopy.org/omemap> rdf:type owl:Ontology ;
owl:imports <https://gitlab.com/openmicroscopy/incubator/ome-owl/-/raw/master/ontology/owl/ome_core/ome_core.owl.ttl> .


#################################################################
# Data properties
#################################################################

### https://www.openmicroscopy.org/omemap#annotation_type
omemap:annotation_type rdf:type owl:DatatypeProperty ;
rdfs:domain ome_core:Annotation .


### Generated by the OWL API (version 4.5.29.2024-05-13T12:11:03Z) https://github.com/owlcs/owlapi
79 changes: 79 additions & 0 deletions mpi_sandbox/queries.org
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
#+begin_src sparql :url http://localhost:8080/sparql
PREFIX : <https://www.openmicroscopy.org/omemap/>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX xml: <http://www.w3.org/XML/1998/namespace>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX obda: <https://w3id.org/obda/vocabulary#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX omemap: <https://www.openmicroscopy.org/omemap#>
PREFIX ome_core: <http://www.openmicroscopy.org/rdf/2016-06/ome_core/>

select * where {
?ant a ome_core:Annotation ;
rdfs:comment ?cmnt .
?ds a ome_core:Dataset ;
ome_core:annotation ?ant .
}
limit 10
#+end_src

#+RESULTS:
| ant | cmnt | ds |
|--------------------------------------------------------------------+-------------------------+----------------------------------------------------------------|
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Annotation/3222 | pseudomonas fluorescens | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/602 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Annotation/3219 | mat formation | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/712 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Annotation/3221 | mirror | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/709 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Annotation/3217 | AxioZoom | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/602 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Annotation/3218 | darkfield | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/703 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Annotation/3221 | mirror | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/603 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Annotation/3224 | upright | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/752 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Annotation/3220 | microscopy | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/707 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Annotation/2106 | Lateral | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/606 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Annotation/2105 | Top | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/603 |

#+begin_src sparql :url http://localhost:8080/sparql
PREFIX : <https://www.openmicroscopy.org/omemap/>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX xml: <http://www.w3.org/XML/1998/namespace>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX obda: <https://w3id.org/obda/vocabulary#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX omemap: <https://www.openmicroscopy.org/omemap#>
PREFIX ome_core: <http://www.openmicroscopy.org/rdf/2016-06/ome_core/>

select * where {
?proj a ome_core:Project ;
rdfs:label ?lbl .
?ds a ome_core:Dataset .
?proj ome_core:dataset ?ds .
}
limit 20
#+end_src

#+RESULTS:
| proj | lbl | ds |
|-----------------------------------------------------------------+---------------------------------+-----------------------------------------------------------------|
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/865 | MouseCT | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/3654 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/1303 | Elio shading test | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/4544 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/904 | Imaging for Elisa Brambilla | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/2962 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/655 | MouseCT_backup | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/1442 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/865 | MouseCT | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/2870 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/865 | MouseCT | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/2571 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/1308 | 3D Cell Atlas | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/4612 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/1002 | Mat_imaging | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/3759 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/865 | MouseCT | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/3193 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/151 | 2016 | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/331 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/865 | MouseCT | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/3383 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/655 | MouseCT_backup | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/1834 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/804 | 2020_2021_VPL_microscopy | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/2200 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/865 | MouseCT | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/2890 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/1256 | Omnipose_Training_2D | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/4448 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/1223 | Alejandro Microscopy | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/4073 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/1223 | Alejandro Microscopy | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/4427 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/655 | MouseCT_backup | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/2024 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/1007 | Microscopy_2024 | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/4556 |
| http://www.openmicroscopy.org/rdf/2016-06/ome_core/Project/1052 | minD oscillation in DrodA cells | http://www.openmicroscopy.org/rdf/2016-06/ome_core/Dataset/4537 |


Empty file.
9 changes: 9 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "omero-ontop-mappings"
version = "0.1"
description = "Placeholder pyproject.toml for omero-ontop-mappings"

4 changes: 4 additions & 0 deletions test/test_mock.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
def test_mock():
a = True
if a:
return True