From 3c53c9a72180fe2202d790ff3b422931b159f78b Mon Sep 17 00:00:00 2001 From: Jillian Vogel Date: Fri, 12 Jan 2024 22:24:43 +1030 Subject: [PATCH 1/3] docs: supersedes ADR 0006 with 0008 --- .../0006_areas_of_responsibility.rst | 5 +- docs/decisions/0008_project_structure.rst | 110 ++++++++++++++++++ 2 files changed, 114 insertions(+), 1 deletion(-) create mode 100644 docs/decisions/0008_project_structure.rst diff --git a/docs/decisions/0006_areas_of_responsibility.rst b/docs/decisions/0006_areas_of_responsibility.rst index ef29dfa..6794af3 100644 --- a/docs/decisions/0006_areas_of_responsibility.rst +++ b/docs/decisions/0006_areas_of_responsibility.rst @@ -4,7 +4,7 @@ Status ****** -Accepted +Superseded by `ADR 8 Project structure`_. Context ******* @@ -106,3 +106,6 @@ ClickHouse schema which Ralph writes to part of, but the rest is managed elsewhe This would also leak Aspects implementation details into all of the plugins, reducing their flexibility for other use cases. + + +.. _ADR 8 Project structure: 0008_project_structure.rst diff --git a/docs/decisions/0008_project_structure.rst b/docs/decisions/0008_project_structure.rst new file mode 100644 index 0000000..3b9d6f8 --- /dev/null +++ b/docs/decisions/0008_project_structure.rst @@ -0,0 +1,110 @@ +8. Project Structure +#################### + +Status +****** + +Accepted + +Context +******* + +The Aspects Analytics system (Aspects) consists of several pieces of technology working together via +some configuration and scripting. Decisions around where to store data and configuration need to be +made so that developers understand where to make changes and look for the causes of issues. These +decisions can have wide ranging impact on things such as extensibility and configurability of the +system as a whole. + +This ADR offers guiding principles to address these issues: + +* The number of plugins and systems in play can make it difficult to know where to look for + configuration or add new features. +* Initialization steps for the third-party systems are sometimes intricate and dependent on other + systems, so Aspects deployment needs to control exactly what happens when. +* In the future, Aspects will offer a non-Tutor deployment option for experienced operators. + +Decisions +********* + +**Aspects repo** + +This repo (`openedx-aspects`_) remains the primary Aspects repository where the project decisions +and documentation are stored. + +This repo should also be used to store any common code or configuration between Tutor and non-Tutor +Aspects deployments. + +**Single Aspects Tutor plugin** + +All of the third-party services that Aspects requires can be deployed and configured by a single +`tutor-contrib-aspects`_ plugin. + +This plugin employs the following structure to delineate the different areas of responsibility:: + + tutoraspects/ + ├── patches + └── templates + ├── aspects + │   ├── apps + │   │   ├── aspects + │   │   ├── clickhouse + │   │   ├── ralph + │   │   ├── superset + │   │   └── vector + │   ├── build + │   │   ├── aspects + │   │   └── aspects-superset + │   │   └── openedx-assets + │   └── jobs + │   └── init + │   ├── aspects + │   ├── clickhouse + │   ├── lms + │   ├── mysql + │   └── superset + └── openedx-assets + +This structure is guided by the following principles: + +* Configuration or functionality specific to a single service should live under a directory named + for that service. + + E.g. `templates/aspects/apps/clickhouse` contains deployment templates related to Clickhouse, + and `templates/aspects/jobs/init/clickhouse` contains the Clickhouse initialization scripts. + +* Configuration or functionality that spans multiple services should live under an `aspects` directory. + + E.g. `templates/aspects/apps/aspects` contains Aspects-specific database migrations (Clickhouse + migrations handled by Alembic) and dbt profiles (data transformation layer). + +* Templates for Superset assets (dashboards, charts, etc.) live under the + `templates/aspects/build/aspects-superset/openedx-assets` directory to facilitate localization of + this user-facing content. + + These asset templates utilize shared, non-localized files stored under `templates/openedx-assets`. + +**LMS plugins** + +Aspects depends on a few Django plugins to move events from the LMS into its processing pipeline +(currently `openedx-event-sink-clickhouse`_ and `event-routing-backends`_). These plugins should +live in their own repositories, and be installed to the LMS/CMS as "extra requirements", using the +chosen deployment's best practices. + +Consequences +************ + +* The service-based Tutor plugins (``tutor-contrib-clickhouse``, ``tutor-contrib-ralph``, ``tutor-contrib-superset``) are consolidated into `tutor-contrib-aspects`_ and archived under `openedx-unsupported`_. + +Rejected Alternatives +********************* + +**Separate plugins for each third-party service** + +Supercedes `ADR 6 Areas of responsibility`_. + +.. _ADR 6 Areas of responsibility: 0006_areas_of_responsibility.rst +.. _event-routing-backends: https://github.com/openedx/event-routing-backends +.. _openedx-aspects: https://github.com/openedx/openedx-aspects +.. _openedx-event-sink-clickhouse: https://github.com/openedx/openedx-event-sink-clickhouse +.. _openedx-unsupported: https://github.com/openedx-unsupported +.. _tutor-contrib-aspects: https://github.com/openedx/tutor-contrib-aspects From 5ad47bd7726c3f5fa0e1506d48c82b129a5e6a67 Mon Sep 17 00:00:00 2001 From: Jillian Vogel Date: Fri, 12 Jan 2024 22:27:48 +1030 Subject: [PATCH 2/3] docs: adds links to the README for rendered documentation and the tutor plugin. --- README.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 7bcb5f4..3a575b7 100644 --- a/README.md +++ b/README.md @@ -8,8 +8,12 @@ Aspects is an optional implementation of analytics for the Open edX LMS. It is t ## What _isn't_ Aspects? -A deployable application in-and-of itself, it helps deploy other applications together using Tutor plugins and customizations to the Open edX platform to connect well-supported, existing, third party applications. +A deployable application in-and-of itself, it helps deploy other applications together using a Tutor plugin and customizations to the Open edX platform to connect well-supported, existing, third party applications. ## Status -Aspects is in development by efforts from Axim Collaborative and Open edX community contributors. You can follow design, development, and implementation at the [Data Working Group issue board](https://github.com/orgs/openedx/projects/5/views/1). +Aspects is in development by efforts from Axim Collaborative and Open edX community contributors. For more information, see: + +* [Data Working Group issue board](https://github.com/orgs/openedx/projects/5/views/1): project design, development, and implementation +* [Aspects Documentation](https://docs.openedx.org/projects/openedx-aspects): project documentation +* [tutor-contrib-aspects](https://github.com/openedx/tutor-contrib-aspects): deploys Aspects using Tutor. From 05786e764b0c93fdc7fd3999e35336ac8c2a0bb8 Mon Sep 17 00:00:00 2001 From: Jillian Vogel Date: Thu, 18 Jan 2024 11:50:47 +1030 Subject: [PATCH 3/3] fix: adds details about LMS plugin install. --- docs/decisions/0008_project_structure.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/decisions/0008_project_structure.rst b/docs/decisions/0008_project_structure.rst index 3b9d6f8..3453503 100644 --- a/docs/decisions/0008_project_structure.rst +++ b/docs/decisions/0008_project_structure.rst @@ -85,10 +85,11 @@ This structure is guided by the following principles: **LMS plugins** -Aspects depends on a few Django plugins to move events from the LMS into its processing pipeline -(currently `openedx-event-sink-clickhouse`_ and `event-routing-backends`_). These plugins should -live in their own repositories, and be installed to the LMS/CMS as "extra requirements", using the -chosen deployment's best practices. +Aspects depends on specific versions of a few Django plugins to move events from the LMS into its +processing pipeline (currently `openedx-event-sink-clickhouse`_ and `event-routing-backends`_). +These plugins should live in their own repositories, and be installed to the LMS/CMS as "extra +requirements", using the chosen deployment's best practices. For example, the Tutor plugin installs +these dependencies to the LMS+CMS using the `OPENEDX_EXTRA_PIP_REQUIREMENTS` variable. Consequences ************