Skip to content
This repository has been archived by the owner on Jun 20, 2023. It is now read-only.

Integrates Superset course-based permissions filters with Clickhouse xAPI data #6

Merged
merged 8 commits into from
Mar 7, 2023

Conversation

pomegranited
Copy link
Contributor

@pomegranited pomegranited commented Feb 27, 2023

Updates Superset initialization and filters for openedx/openedx-aspects#30:

  • Loads the "Open edX" role and default role permissions during superset init.
    This role is used to mark the row-level security settings which restrict access to xAPI data by course, see Creates OARS datasets, charts, and dashboards in Superset openedx/tutor-contrib-aspects#3
  • Fixes our custom Jinja can_view_courses filter to allow it to work on Clickhouse xAPI data.
  • Refactors our custom security manager to encapsulate its functionality, and improve caching of users' allowed courses list.
  • Parameterizes some tutor config that we need to use in tutor-config-oars

Miscellaneous fixes:

  • [nit] rename the superset-app service to just superset -- it's annoying, and keeps tripping me up.
  • PII: Remove usernames from debug/error logging

Testing instructions

See openedx/tutor-contrib-aspects#3.

@openedx-webhooks openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Feb 27, 2023
@openedx-webhooks
Copy link

openedx-webhooks commented Feb 27, 2023

Thanks for the pull request, @pomegranited! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

  • supporting documentation
  • Open edX discussion forum threads
  • timeline information ("this must be merged by XX date", and why that is)
  • partner information ("this is a course on edx.org")
  • any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

and shifts Jinja filter to a separate module.
This role is assigned to all Open edX SSO users, and is used to mark
row-based permissions.
so we can use it in the OARS plugin too
@pomegranited pomegranited force-pushed the jill/superset-xapi-permissions branch from 867c4f1 to 945780a Compare March 1, 2023 08:32
@pomegranited pomegranited changed the title Jill/superset xapi permissions Integrates Superset course-based permissions filters with Clickhouse xAPI data Mar 2, 2023
The user's role list isn't returned as an array of strings, it's an
array of Roles, so updated the check accordingly.
@pomegranited pomegranited marked this pull request as ready for review March 5, 2023 04:29
Copy link
Collaborator

@bmtcril bmtcril left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, couple of non-blocking comments / questions. I'm ok with merging as is if we can move the TODOs out to tickets to be followed up on.

tutorsuperset/templates/superset/jobs/init/init-openedx.sh Outdated Show resolved Hide resolved
"""
Fetches the given user's access details from the Open edX User database
(since Open edX doesn't have an API for this).

NOTE: Open edX JWT seems to provide this info with the "profile" scope.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow is this all we're using the mysql db for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeup, that's it right now.

But it's likely that people will want to query things in MySQL to put in their charts, right? Which raises an operational note: does Tutor do read replicas yet? We don't really want people to have to slow down their LMS/Studio running expensive queries..

Copy link
Contributor Author

@pomegranited pomegranited Mar 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found the method that gives us the extra user data from the OAuth2 JWT token: openedx:auth_backends/backends.py

In that dict are "administrator" (is_staff) and "superuser" (is_superuser) values.

I think we'll need to get a JWT token in order to use this (we've currently just got an OAuth2 token): https://github.com/openedx/edx-platform/blob/master/openedx/core/djangoapps/oauth_dispatch/docs/decisions/0003-use-jwt-as-oauth-tokens-remove-openid-connect.rst#requesting-jwt-tokens

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah neat, I'm not necessarily against using MySQL here but it would be nice to go through OAuth for all of this. I don't know that Tutor supports MySQL read replicas, I'd guess not since it might depend a lot on how / where you're hosting things. In theory we could use ClickHouse MySQL tables to connect to replicas if they exist and push the configuration there: https://clickhouse.com/docs/en/sql-reference/table-functions/mysql

I'm not sure that's the best option though. There's this task for figuring out whether it is feasible for permissions, though that seems unnecessary right now. It would probably be worth having for a lot of other things, though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up task: openedx/openedx-aspects#42

@pomegranited pomegranited merged commit 6cdb82c into main Mar 7, 2023
@pomegranited pomegranited deleted the jill/superset-xapi-permissions branch March 7, 2023 03:40
@openedx-webhooks
Copy link

@pomegranited 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
open-source-contribution PR author is not from Axim or 2U
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants