True. Things have grown historically. Though there are several categories of commands, broadly:

• one-time system setup
• database management (manage.py, models.py)
• running GHC: healthchecks (models.py) or HTTP server (app.py)
• maintain system: like updating translations

Have to think what the best strategy is. We could move everything to pavement.py. That seems the most flexible though may be overwhelming. Or do you recommend other ways, maybe even another CLI tool like http://click.pocoo.org/?

Not for this issue: Would things help if we move to Django at some point? Looks like lots GHC stuff is explicitly programmed (forms, DB upgrades, possibly other commands) that is automated or web-based in Django.

+1 to move everything in one file... with the possibilities to have includes

Have to think what the best strategy is. We could move everything to pavement.py. That seems the most flexible though may be overwhelming. Or do you recommend other ways, maybe even another CLI tool like http://click.pocoo.org/?

I don't think using paver is a good idea. It's purpose is a bit different, and while it can be used to manipulate the app, it's not suited well to build composed apps. Flask offers cli application facilities, uses click under the hood, so I guess this would be more proper direction.

Not for this issue: Would things help if we move to Django at some point? Looks like lots GHC stuff is explicitly programmed (forms, DB upgrades, possibly other commands) that is automated or web-based in Django.

In general yes, although it's better to do it in early stage. Django takes away a lot of project infrastructure decisions from developers, offering solid help there (migrations ootb, custom commands infrastructure, separate test configuration, auth and user management, pluggable apps etc) , but requires keeping to specific project architecture.

I browsed through pavement.py and the documentation and I came to this list of CLI commands:

Paver

• setup: setup plugin dependencies
• create_secret_key
• create: db and superuser account
• create_hash
• upgrade: upgrade db
• refresh_docs: sphinx build
• clean: clean environment
• extract_translations
• add_language_catalogue
• compile translations
• update_translations
• runner_deamon
• run_healthchecks
• run_tests

From docs

(And not mentioned with paver):

• python GeoHealthCheck/models.py create: init database
• python GeoHealthCheck/app.py: start webapp (optional host and port)
• python GeoHealthCheck/scheduler.py &: start runner in background
• python GeoHealthCheck/models.py drop: drop database
• python GeoHealthCheck/models.py load <datafile.json> [y/n]: load data
• SQL for resetting user password
• build documentation with sphinx (has already paver command)

So most of the commands are already accessible with paver and my instinct would be to add/modify the commands in the docs that aren't covered yet. Or would it be better to drop paver altogether and move to click as suggested (or something else) ?

Some of the paver commands use sh and I assume that gives issues on windows machines. How important is that?

IMHO: moving to Django would be an entire different discussion with more sides to it than CLI.

@borrob thanks for looking into this. Bit hard for me to estimate. For new projects I (others) usually start with Click. This is the only project I use Paver. I am no expert in this, but purposes of these two seem overlapping, with Paver more geared at the setup/install phase and some development like translations. Also Paver shields from OS/Filesystem-dependencies, so in theory sh() should work also on Win, though not completely. (Maybe even Paver could be used within Click?).

We don't have a real (pip) installation yet for GHC as a Python package. A typical Python package installation/setup would usually install a (click) CLI in a global or venv bin location. See for example pygeoapi.
From there a typical scenario is to install a server instance via that CLI. See for example mapproxy where the CLI mapproxy-util is installed.

In the longer term I would like to see a GHC install from PyPi and instance(s) with this scenario, all commands with options/flags as to run in automatic installs like Docker. Each subcommand handles a set of related actions:

pip install geohealthcheck
# installs GHC CLI e.g. `geohc` (`ghc` is already for Haskell..)
geohc create-instance . # creates server instance, possibly interactively

geohc manage-db create
geohc manage-db init
geohc manage-db upgrade
geohc manage-db export
geohc manage-db import

geohc client ... # commands to talk to (future) API, e.g. bulk import Resources.

geohc serve ...

But this will be a longer path, we need project-discussion for. In the short term is it an idea to keep Paver for the development and install (e.g. it downloads .zip for JS libs) phase and introduce Click for daily/admin management? As part of the Paver install a Click CLI could be installed where we could put all DB/'runserver'-related actions. Just thinking out loud. Replacing all by Click is quite disrupting, there's quite some tricky stuff in pavement.py...

As for Django: yes, seen in many Flask-projects, one starts building/adding components to finally get to what Django has built-in or provides in Django-app-extensions: migrations, (registration-) forms, (DB-)views, REST, translations, encrypted passwords, multiple configs etc. My ideal architecture for the future would be a Django-REST app with a Vue.js (or any other) frontend that only interacts via the GHC-API (no Jinja templating). Maybe, if we leave the DB structure as-is and wrapping all "business/healthcheck" logic, we could develop that as a side project, also a PSC issue.

To summarise the discussion and actions:

Short term solution

• Keep paver and all current paver commands (in order to not brake any existing tool chains)
• Introduce click for all "non-install" related tasks
• copy/shortcut/link as many paver commands under click

Long term solution

• discuss with PSC on migration of Flask to Django framework
• discuss with PSC on creating a PyPi (and/or conda) package
• discuss with PSC on CLI tools and interface
• discuss with PSC on frontend: jinja vs Vue/Angular/...

Actions

I can start with the short term items. Where do we document the PSC items? New issues? A wiki page?

@borrob Good summary! Agree you go ahead with Short Term Solution.

GHC does not have a PSC yet, was also a trigger to start one...Also needed for OSGeo.org incubation AFAIK. We can model after pygeoapi: https://pygeoapi.io/community/psc/. Will put forward on GHC Gitter. @tomkralidis : or have I missed something?