You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Building a model that was created by doing a clone command that has a PK constraint fails with an error that the PK already exists when dbt tries to apply an alter table to create the PK.
Found the issue when doing a slim CI setup in order to support setting PK/FK constraints I am executing:
dbt clone --select @State:modified,state:old
I then execute
dbt build --select state:modified+
This way all parent models for any model downstream of an impacted model are present, this is helpful for being able to have the FK references work, otherwise the referenced table is potentially not present, and currently FK does not honor deferral.
When building a model that was created by the above clone, if that model has a PK defined, the model fails due to the PK already existing.
Database Error in model model_b (models/model_b.sql)
[DELTA_CONSTRAINT_ALREADY_EXISTS] Constraint 'model_b_pk' already exists. Please delete the old constraint first.
Old constraint:
model_b_pk PRIMARY KEY (`pk_col`)
compiled code at target/run/databricks_analytics_dbt/models/model_b.sql
Steps To Reproduce
Create a model with a PK defined:
{{ config(materialized = 'table')}}
select
'a' as pk_col
If you directly build this and rebuild it works fine. If you create this model in a different target schema, and then clone it to your current schema when you try to build it will fail with the constraint already exists error.
I've tried without naming the PK and using the hash. On clone Databricks is automatically generating the PK name as
'tablename'_pk
Using the same name does not get dbt to recognize the already existing PK and so it tries to do an alter to apply it which then fails.
Expected behavior
DBT should recognize an existing PK and either remove it and replace with the current model definition, or especially if the name matches alter the existing version rather than attempting to apply it as a new PK.
Screenshots and log output
Cloned table exists:
Create or replace statement executes:
create
or replace table `model_b` using delta
as
select
pk_col
from
(
select
'a' as pk_col
) as model_subq
Then tries to alter with the PK
alter table
`model_b`
add
constraint model_b_pk primary key(pk_col)
Describe the bug
Building a model that was created by doing a clone command that has a PK constraint fails with an error that the PK already exists when dbt tries to apply an alter table to create the PK.
Found the issue when doing a slim CI setup in order to support setting PK/FK constraints I am executing:
dbt clone --select @State:modified,state:old
I then execute
dbt build --select state:modified+
This way all parent models for any model downstream of an impacted model are present, this is helpful for being able to have the FK references work, otherwise the referenced table is potentially not present, and currently FK does not honor deferral.
When building a model that was created by the above clone, if that model has a PK defined, the model fails due to the PK already existing.
Steps To Reproduce
Create a model with a PK defined:
If you directly build this and rebuild it works fine. If you create this model in a different target schema, and then clone it to your current schema when you try to build it will fail with the constraint already exists error.
I've tried without naming the PK and using the hash. On clone Databricks is automatically generating the PK name as
'tablename'_pk
Using the same name does not get dbt to recognize the already existing PK and so it tries to do an alter to apply it which then fails.
Expected behavior
DBT should recognize an existing PK and either remove it and replace with the current model definition, or especially if the name matches alter the existing version rather than attempting to apply it as a new PK.
Screenshots and log output
Cloned table exists:
Create or replace statement executes:
Then tries to alter with the PK
System information
DBT Cloud using 'Latest Version'
16:16:32 Registered adapter: databricks=1.9.0-post8+5e20eeaef43e671913f995d8079d4ec2b8a1da6d
16:16:32 checksum: 429d12d886761424119078b59c6289cc952f90a4f6bdc27a6fe96500a5717cce, vars: {}, profile: user, target: , version: 2024.12.18+447b71a
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: