Merge pull request #216 from openedx/cag/ttl

docs: add data lifecycly policy

docs/concepts/data_lifecycle_policy.rst

@@ -0,0 +1,14 @@
+.. _data-lifecycle-policy:
+
+Data Lifecycle Policy
+*********************
+
+What it is
+##########
+
+Aspects is a data pipeline that captures, transforms, and aggregates tracking logs from the Open edX platform into xAPI statements and stores them in a ClickHouse database.
+However, the data is not stored indefinitely by default. The data is keep for 1 year by default, but this can be adjusted by the site operator via the setting `ASPECTS_DATA_TTL_EXPRESSION` in the tutor plugin.
+
+The setting value is a ClickHouse expression that defines the time-to-live policy (TTL) for the data. The expression is evaluated for each row in the table and should return a date. Rows with a date in the past are deleted. You can read more about the TTL policy in the ClickHouse documentation: https://clickhouse.tech/docs/en/engines/table-engines/mergetree-family/mergetree/#ttl
+
+The data is partioned by month this way the TLL policy is applied per partition. Make sure to set the TTL policy to a date that is compatible with the partitioning policy. e.g. `ASPECTS_DATA_TTL_EXPRESSION: toDateTime(emission_time) + INTERVAL 2 MONTH` or `ASPECTS_DATA_TTL_EXPRESSION: toDateTime(emission_time) + INTERVAL 2 YEAR`.

docs/concepts/index.rst

@@ -9,6 +9,7 @@ Concepts
    xAPI <xapi_concepts>
    Tracking Logs <tracking_logs>
    Clickhouse <clickhouse>
+   Data Lifecycle Policy <data_lifecycle_policy>
    dbt <dbt>
    Ralph <ralph>
    Vector <vector>