How to load dbt source description and metadata into Dagster

[csaba-pusztai]

I have successfully integrated a dbt model into Dagster. Description and metadata are correctly pulled in by Dagster for my tables and visible both in the graph (the description) and the corresponding asset detail pages . However, source details do not seem to be populated when I click on the asset details. The model runs nicely via Dagster and sources appear in the graph.

My dummy dbt schema.yml is as follows:

version: 2

models:
  - name: test_model_1
    description: "This model contains user-level revenue data aggregated at the monthly level."
    columns:
      - name: n_of_rows
    config:
      meta:
        dagster:
          group: revenue

sources:
  - name: external
    description: "This is an external source"
    database: ext
    schema: ext
    tables:
    - name: payments
      description: "User payment data"
      meta:
        dagster:
          group: revenue

More specifically:

• I do get to see "This model contains user-level revenue data aggregated at the monthly level." for my test_model_1 in the graph node and on the asset profile.
• But I do not see "User payment data" either in the graph node or the asset profile.

[rexledesma]

Details like description and column metadata are not automatically pulled in for dbt sources. By default, Dagster only takes the dbt source's asset key and displays it in the asset graph.

If you'd like these details to be visible for a dbt source, then you should:

1. Define your dbt source as a Dagster asset (using @asset or @multi_asset)
2. Define the metadata for that Dagster asset using the dbt manifest.

References

• https://docs.dagster.io/integrations/dbt/reference#upstream-dependencies

[sotte]

From a usability perspective it's not great that descriptions of dbt sources are handled differently than dbt models. I really wish descriptions and metadata columns would be pulled automatically.