Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor startup scripts, envvars for all omero parameter #1

Merged
merged 17 commits into from
Jul 13, 2017
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions 50-config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/usr/bin/env python
# Set omero config properties from CONFIG_ envvars
# Variable names should replace "." with "_" and "_" with "__"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not particularly worried but there's obviously some danger of a __ conflict here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know. Do you have another suggestion?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how evil this is but:

$ env "a.b=1" python -c "import os; print os.environ['a.b']"
1
$ docker run -ti --rm -e a.b=1 centos python -c "import os; print os.environ['a.b']"
1

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortuntately it's also used in the bash script60-database.sh

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Continuing discussion under #3

# E.g. CONFIG_omero_web_public_enabled=false

import os
from subprocess import call
from re import sub


for (k, v) in os.environ.iteritems():
omero = '/opt/omero/server/OMERO.server/bin/omero'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need not be in loop

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and probably should be caps

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

if k.startswith('CONFIG_'):
prop = k[7:]
prop = sub('([^_])_([^_])', r'\1.\2', prop)
prop = sub('__', '_', prop)
value = v
rc = call([omero, 'config', 'set', '--', prop, value])
assert rc == 0
43 changes: 43 additions & 0 deletions 60-database.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#!/bin/bash
# 50-config.py or equivalent must be run first to set omero.db.*

set -eu

omero=/opt/omero/server/OMERO.server/bin/omero
omego=/opt/omero/omego/bin/omego
cd /opt/omero/server

CONFIG_omero_db_host=${CONFIG_omero_db_host:-}
if [ -n "$CONFIG_omero_db_host" ]; then
DBHOST="$CONFIG_omero_db_host"
else
DBHOST=db
$omero config set omero.db.host "$DBHOST"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be outside the if block?

Copy link
Member Author

@manics manics Jun 8, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, CONFIG_omero_db_host will already be converted into omero.db.host by 50-config.py- I'll update the comment at the top of the file. The only reason for special handling here is that the default of omero.db.host=localhost will never work, instead it defaults to db which should be set by docker run --link ....

fi
DBUSER="${CONFIG_omero_db_user:-omero}"
DBNAME="${CONFIG_omero_db_name:-omero}"
DBPASS="${CONFIG_omero_db_pass:-omero}"
ROOTPASS="${ROOTPASS:-omero}"

export PGPASSWORD="$DBPASS"

i=0
while ! psql -h "$DBHOST" -U "$DBUSER" "$DBNAME" >/dev/null 2>&1 < /dev/null; do
i=$(($i+1))
if [ $i -ge 50 ]; then
echo "$(date) - postgres:5432 still not reachable, giving up"
exit 1
fi
echo "$(date) - waiting for postgres:5432..."
sleep 1
done
echo "postgres connection established"

psql -w -h "$DBHOST" -U "$DBUSER" "$DBNAME" -c \
"select * from dbpatch" 2> /dev/null && {
echo "Upgrading database"
$omego db upgrade --serverdir=OMERO.server
} || {
echo "Initialising database"
$omego db init --rootpass "$ROOTPASS" --serverdir=OMERO.server
}
16 changes: 16 additions & 0 deletions 99-run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash

set -eu

omero=/opt/omero/server/OMERO.server/bin/omero
cd /opt/omero/server

if stat -t /config/* > /dev/null 2>&1; then
for f in /config/*; do
echo "Loading $f"
$omero load "$f"
done
fi

echo "Starting OMERO.server"
exec $omero admin start --foreground
14 changes: 8 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,15 @@ RUN yum -y install epel-release \
ARG OMERO_VERSION=latest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're using the main release version (i.e. 5.3.2) for tags of this repo, then I'm assuming:

  • latest images are considered worst-practice
  • on each OMERO release, we'll branch from master and bump this string to include the version (5.3.3)
  • for bug fixes before an OMERO release, we'll suffix 5.3.3-1

Note: this doesn't yet give us a strategy for breaking changes in the image itself

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming this has everyone's implicit 👍 but now with 5.4 replacing 5.3

RUN ansible-playbook playbook.yml -e omero_server_release=$OMERO_VERSION

USER omero-server
RUN curl -L -o /usr/local/bin/dumb-init \
https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64 && \
chmod +x /usr/local/bin/dumb-init
ADD run-exec.sh /usr/local/bin/
ADD 50-config.py 60-database.sh 99-run.sh /startup/

# default.xml may be modified at runtime for a multinode configuration
RUN cp /opt/omero/server/OMERO.server/etc/templates/grid/default.xml /opt/omero/server/OMERO.server/etc/templates/grid/default.xml.orig
USER omero-server

EXPOSE 4061 4063 4064
EXPOSE 4063 4064
VOLUME ["/OMERO", "/opt/omero/server/OMERO.server/var"]

ADD slave.cfg run.sh process_defaultxml.py /opt/omero/server/
ENTRYPOINT ["/opt/omero/server/run.sh"]
ENTRYPOINT ["/usr/local/bin/run-exec.sh"]
76 changes: 76 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
OMERO.server Docker
===================

A CentOS 7 based Docker image for OMERO.server.


Running the images
------------------

To run the Docker images start a postgres DB:

docker run -d --name postgres -e POSTGRES_PASSWORD=postgres postgres

Then run OMERO.server passing the the database configuration parameters if they differ from the defaults:

docker run -d --name omero-server --link postgres:db
-e CONFIG_omero_db_user=postgres \
-e CONFIG_omero_db_pass=postgres \
-e CONFIG_omero_db_name=postgres \
-e ROOTPASS=omero-root-password \
-p 4063:4063 -p 4064:4064 \
-e ROOTPASS=omero openmicroscopy/omero-server
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ROOTPASS twice?



Configuration variables
-----------------------

All [OMERO configuration properties](www.openmicroscopy.org/site/support/omero/sysadmins/config.html) can be set be defining environment variables `CONFIG_omero_property_name=`.
Since `.` is not allowed in a variable name `.` must be replaced by `_`, and `_` by `__`, for example

-e CONFIG_omero_web_public_enabled=false


Configuration files
-------------------

Additional configuration files for OMERO can be provided by mounting a directory `/config`.
Files will be loaded with `omero load`.
For example:

docker run -d -v /config:/config:ro openmicroscopy/omero-server

Parameters required for initialising the server such as database configuration *must* be set using environment variables.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

initializing

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed



Default volumes
---------------

- `/opt/omero/server/OMERO.server/var`: The OMERO.server `var` directory, including logs
- `/OMERO`: The OMERO data directory


Exposed ports
-------------

- 4063
- 4064


Example with named volumes
--------------------------

docker volume create --name omero-db
docker volume create --name omero-data

docker run -d --name postgres -e POSTGRES_PASSWORD=postgres \
-v omero-db:/var/lib/postgresql/data postgres
docker run -d --name omero-server --link postgres:db
-e CONFIG_omero_db_pass=dbpassword -v omero-data:/OMERO \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example isn't working for me:

FATAL:  password authentication failed for user "omero"
DETAIL:  Connection matched pg_hba.conf line 95: "host all all 0.0.0.0/0 md5"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't want to repeat all the -e CONFIG_omero_db_* params from the previous example, I'll try and make that clearer

-p 4063:4063 -p 4064:4064 openmicroscopy/omero-server


Running without links
---------------------

As an alternative to running with `--link` the address of the database can be specified using the variable `CONFIG_omero_db_host`
73 changes: 0 additions & 73 deletions process_defaultxml.py

This file was deleted.

10 changes: 10 additions & 0 deletions run-exec.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/local/bin/dumb-init /bin/bash

set -e

for f in /startup/*; do
if [ -f "$f" -a -x "$f" ]; then
echo "Running $f $@"
"$f" "$@"
fi
done
105 changes: 0 additions & 105 deletions run.sh

This file was deleted.

8 changes: 0 additions & 8 deletions slave.cfg

This file was deleted.

Loading