Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Altinity Cloud & ClickHouse clustering #218

Closed
bmtcril opened this issue May 3, 2024 · 3 comments
Closed

Fix Altinity Cloud & ClickHouse clustering #218

bmtcril opened this issue May 3, 2024 · 3 comments
Assignees

Comments

@bmtcril
Copy link
Contributor

bmtcril commented May 3, 2024

Currently there are a few blocking issues with clustering and Altinity Cloud that would prevent operators from using them. They are mostly fixed in these largely untested branches:

Upstream fix "ON CLUSTER" for dictionaries:
https://github.com/bmtcril/dbt-clickhouse/tree/bmtcril/cluster_dictionaries

Various other fixes to make sure we're using the cluster correctly for DDLs:
openedx/aspects-dbt#84
openedx/tutor-contrib-aspects#767

I've left off this work in favor of Redwood critical tasks, but at current Altinity wasn't able to load our 300M xapi dataset due to memory errors, and quite possible had some other issues that I couldn't test due to the above.

@bmtcril bmtcril self-assigned this Jul 2, 2024
@bmtcril bmtcril moved this to Doing in Data Working Group Jul 2, 2024
@bmtcril
Copy link
Contributor Author

bmtcril commented Jul 2, 2024

PR for dbt-clickhouse is now here, but I'm waiting for approval to sign the CLA: ClickHouse/dbt-clickhouse#307 (comment)

@bmtcril
Copy link
Contributor Author

bmtcril commented Jul 2, 2024

These fixes have been tested against Altinity Cloud and edunext's k8s cluster, as well as checked against local CH and CH Cloud.

@bmtcril bmtcril moved this from Doing to Done in Data Working Group Aug 8, 2024
@bmtcril
Copy link
Contributor Author

bmtcril commented Aug 8, 2024

This merged

@bmtcril bmtcril closed this as completed Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

No branches or pull requests

1 participant