diff --git a/docs/how-tos/clickhouse_cluster.rst b/docs/how-tos/clickhouse_cluster.rst new file mode 100644 index 0000000..3c28304 --- /dev/null +++ b/docs/how-tos/clickhouse_cluster.rst @@ -0,0 +1,24 @@ +.. clickhouse-cluster: + +How To Run Aspects With ClickHouse Cluster +****************************************** + +ClickHouse clusters are an advanced way of running ClickHouse, but offer many benefits for large scale installations. Deciding whether to run in a clustered environment is a big decision with many cost and administrative impacts and should be carefully considered before launching a production environment. In most cases we expect Aspects to perform well without cluster scaling. + +.. warning:: + Please fully understand the nuances of ClickHouse replication before choosing this route! While we have done some testing to ensure basic functionality of Aspects in a clustered environment, it may be expected to fail in novel and spectacular ways until we have a fully tested release running under load. + + +It is strongly suggested that if you are considering clustering you use a service provider to manage the cluster for you. `ClickHouse Cloud `_ and `Altinity `_ both offer ClickHouse as a service. Altinity also offers `management services `_ for running ClickHouse in your own Kubernetes environment. + +Configuring Aspects to run in clustered mode is simple: + +- Follow the directions in :ref:`remote-clickhouse` to set up your ClickHouse connection +- Set the ``CLICKHOUSE_CLUSTER_NAME`` Tutor configuration variables as appropriate for your installation +- Save your Tutor configuration: ``tutor config save`` + +When you initialize your environment the following changes will happen from a single server install: + +- All users, databases, tables, views, functions, etc. will be created with ``ON CLUSTER 'your-cluster-name'``, including dbt managed views. +- All tables will be created with the Replicated versions of their table engines (ex: ``ReplicatedReplacingMergeTree``) +- ClickHouse traffic will be split among your nodes, as described in `their documentation `_ . diff --git a/docs/how-tos/index.rst b/docs/how-tos/index.rst index 7624845..5c527ba 100644 --- a/docs/how-tos/index.rst +++ b/docs/how-tos/index.rst @@ -5,6 +5,7 @@ How-Tos :maxdepth: 2 :caption: Content: + Upgrade Aspects Changing the xAPI actor identifier Backfill old or missing data Superset extra assets @@ -15,3 +16,4 @@ How-Tos Clickhouse extra SQL Connect to external Clickhouse database Extending DBT + Run Aspects in a ClickHouse cluster diff --git a/docs/how-tos/remote_clickhouse.rst b/docs/how-tos/remote_clickhouse.rst index 1b31446..fcb9537 100644 --- a/docs/how-tos/remote_clickhouse.rst +++ b/docs/how-tos/remote_clickhouse.rst @@ -51,8 +51,8 @@ Clickhouse instance. To do so, set the following variables in your **config.yaml CLICKHOUSE_SECURE_CONNECTION: true|false CLICKHOUSE_ADMIN_USER: CLICKHOUSE_ADMIN_PASSWORD: - CLICKHOUSE_CLIENT_PORT: # default to 9000 | 9440 used by initialization tasks - CLICKHOUSE_PORT: # default to 8123 | 8443 used by other services such as ralph and the lms + CLICKHOUSE_HOST_HTTP_PORT: # defaults to 8123 or 8443 for secure TLS connections + CLICKHOUSE_HOST_NATIVE_PORT: # default to 9000 | 9440 used by initialization tasks Additionally, there are some other variables that are calculated based on the above variables, but can be customized if needed: diff --git a/docs/how-tos/upgrade.rst b/docs/how-tos/upgrade.rst new file mode 100644 index 0000000..341b70c --- /dev/null +++ b/docs/how-tos/upgrade.rst @@ -0,0 +1,15 @@ +.. upgrade-aspects: + +How-to Upgrade Aspects +********************** + +At least in the early phases of existence, Aspects is intended to have a faster upgrade cycle than Open edX named releases. It's expected that many operators will want to upgrade between releases to get features as they become available, but the upgrade process should be the same whether it's happening as part of a named release upgrade or in between. + +As for any upgrade you should take a backup snapshot of your environment before beginning, especially make sure you have a recent backup of your ClickHouse database. Then follow these steps, which are basically the same as installation: + +- Install the version you would like from tutor-contrib-aspects, or for the latest: ``pip install --upgrade tutor-contrib-aspects`` +- Save your tutor configuration: ``tutor config save`` +- Build your openedx image (in case there are upgrades to LMS/Studio plugins): ``tutor images build openedx --no-cache`` +- Initialize Aspects to get the latest schema and reports, for a tutor local install: ``tutor local do init -l aspects`` + +In a case where the release has special instructions, such as when new xAPI transforms have been added and you may need to replay tracking logs, they will be included in the release announcement.