Docs: How-to - Dbt packages

[bmtcril]

No description provided.

[pomegranited]

@bmtcril Does #111 address this issue sufficiently?

I'm happy to expand on the Extending dbt page if we want to provide a more detailed tutorial.

[bmtcril]

@pomegranited I think we may want a step-by-step for that, unless @Ian2012 has already covered that elsewhere or has a starter doc?

[Ian2012]

@pomegranited That's the only thing we need to expand DBT, for the conference we can include a video tutorial and a demo package that includes some useful models for custom stuff. I can consider this as closed

[pomegranited]

@Ian2012 Cool, I think a demo package and a tutorial will help people, and it would help me to go through the whole process to understand it better.

But I don't think we should include a video -- they're not very accessible and are hard to translate. Would a step-by-step tutorial on this docs site be ok?

[Ian2012]

Yes, sure

[pomegranited]

@bmtcril @Ian2012 I have hit an issue with dbt that I can't figure out from the docs.. can you help? Here's what happened:

1. Created this aspects-dbt-example @ jill/demo and configured my tutor dev stack to use it.

2. The first time I ran it (using tutor dev do dbt -c "run"), I used the wrong schema + table name, specifying reporting.problem_events which does not exist, and so dbt accurately reported the error:

   06:05:28    Database Error in model average_attempts (models/problem_responses/average_attempts.sql)
   :HTTPDriver for http://clickhouse:8123 returned response code 404)
   Code: 60. DB::Exception: Table reporting.problem_events does not exist. (UNKNOWN_TABLE) (version 23.8.9.54 (official build))
   compiled Code at target/run/aspects_dbt_example/models/problem_responses/average_attempts.sql
   

3. I tried changing the table to xapi.problem_events, which does exist, but I couldn't get rid of the error.

   Clickhouse seems to have stored the relationship; I can see this line at the top of this file $(tutor config printroot)/data/clickhouse/store/e4a/e4a29e0d-0e2f-44fb-9e2b-0d62549aaa57/problem_events_mv.sql:

   ATTACH MATERIALIZED VIEW _ UUID 'e25be6d2-1ed2-481a-a47b-457b6a989c6f' TO reporting.problem_events`
   

I've tried dbt clean, and adding +full_refresh: true to the model config in dbt_project.yml, but neither helped.

This is just my devstack, so it's no big deal to blow away and start again. But since I'm writing a tutorial on how to create a custom dbt package, it would be really nice to resolve this without extreme measures 😄

Edit have since changed over to using ref('int_problem_results'), and that worked, but I'd still like to know how I could have made xapi.problem_events work.

[bmtcril]

@pomegranited I think you have a version with the dbt bug that didn't get fixed until 0.73.1 yesterday. Then the build broke on those changes, so you'd have to build aspects / aspects-superset yourself until the next fix goes out, hopefully today. Sorry about that!

[pomegranited]

@bmtcril Ooh cool! That's great timing, thank you :)