Merge pull request #96 from openedx/bmtcril/beta_doc_updates

docs: Add upgrading and clustering docs, update port setting names

docs/how-tos/clickhouse_cluster.rst

@@ -0,0 +1,24 @@
+.. clickhouse-cluster:
+
+How To Run Aspects With ClickHouse Cluster
+******************************************
+
+ClickHouse clusters are an advanced way of running ClickHouse, but offer many benefits for large scale installations. Deciding whether to run in a clustered environment is a big decision with many cost and administrative impacts and should be carefully considered before launching a production environment. In most cases we expect Aspects to perform well without cluster scaling.
+
+.. warning::
+    Please fully understand the nuances of ClickHouse replication before choosing this route! While we have done some testing to ensure basic functionality of Aspects in a clustered environment, it may be expected to fail in novel and spectacular ways until we have a fully tested release running under load.
+
+
+It is strongly suggested that if you are considering clustering you use a service provider to manage the cluster for you. `ClickHouse Cloud <https://clickhouse.com/cloud>`_ and `Altinity <https://altinity.com/cloud-database/>`_ both offer ClickHouse as a service. Altinity also offers `management services <https://altinity.com/cloud-database/#anywhere>`_ for running ClickHouse in your own Kubernetes environment.
+
+Configuring Aspects to run in clustered mode is simple:
+
+- Follow the directions in :ref:`remote-clickhouse` to set up your ClickHouse connection
+- Set the ``CLICKHOUSE_CLUSTER_NAME`` Tutor configuration variables as appropriate for your installation
+- Save your Tutor configuration: ``tutor config save``
+
+When you initialize your environment the following changes will happen from a single server install:
+
+- All users, databases, tables, views, functions, etc. will be created with ``ON CLUSTER 'your-cluster-name'``, including dbt managed views.
+- All tables will be created with the Replicated versions of their table engines (ex: ``ReplicatedReplacingMergeTree``)
+- ClickHouse traffic will be split among your nodes, as described in `their documentation <https://clickhouse.com/docs/en/architecture/cluster-deployment>`_ .

docs/how-tos/index.rst

@@ -5,6 +5,7 @@ How-Tos
    :maxdepth: 2
    :caption: Content:
 
+   Upgrade Aspects <upgrade>
    Changing the xAPI actor identifier <changing_actor_identifier>
    Backfill old or missing data <backfill>
    Superset extra assets <superset_extra_assets>
@@ -15,3 +16,4 @@ How-Tos
    Clickhouse extra SQL <clickhouse_sql>
    Connect to external Clickhouse database <remote_clickhouse>
    Extending DBT <dbt_extensions>
+   Run Aspects in a ClickHouse cluster <clickhouse_cluster>

docs/how-tos/remote_clickhouse.rst

@@ -51,8 +51,8 @@ Clickhouse instance. To do so, set the following variables in your **config.yaml
     CLICKHOUSE_SECURE_CONNECTION: true|false
     CLICKHOUSE_ADMIN_USER: <clickhouse_admin_user>
     CLICKHOUSE_ADMIN_PASSWORD: <clickhouse_admin_password>
-    CLICKHOUSE_CLIENT_PORT: <clickhouse_client_port> # default to 9000 | 9440 used by initialization tasks
-    CLICKHOUSE_PORT: <clickhouse_port> # default to 8123 | 8443 used by other services such as ralph and the lms
+    CLICKHOUSE_HOST_HTTP_PORT: <clickhouse_http_port> # defaults to 8123 or 8443 for secure TLS connections
+    CLICKHOUSE_HOST_NATIVE_PORT: <clickhouse_client_port> # default to 9000 | 9440 used by initialization tasks
 
 Additionally, there are some other variables that are calculated based on the above variables, but
 can be customized if needed:

docs/how-tos/upgrade.rst

@@ -0,0 +1,15 @@
+.. upgrade-aspects:
+
+How-to Upgrade Aspects
+**********************
+
+At least in the early phases of existence, Aspects is intended to have a faster upgrade cycle than Open edX named releases. It's expected that many operators will want to upgrade between releases to get features as they become available, but the upgrade process should be the same whether it's happening as part of a named release upgrade or in between.
+
+As for any upgrade you should take a backup snapshot of your environment before beginning, especially make sure you have a recent backup of your ClickHouse database. Then follow these steps, which are basically the same as installation:
+
+- Install the version you would like from tutor-contrib-aspects, or for the latest: ``pip install --upgrade tutor-contrib-aspects``
+- Save your tutor configuration: ``tutor config save``
+- Build your openedx image (in case there are upgrades to LMS/Studio plugins): ``tutor images build openedx --no-cache``
+- Initialize Aspects to get the latest schema and reports, for a tutor local install: ``tutor local do init -l aspects``
+
+In a case where the release has special instructions, such as when new xAPI transforms have been added and you may need to replay tracking logs, they will be included in the release announcement.