Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

createIndex if 'NOT indexExists' fails (version 4.15) #283

Open
jmayday opened this issue Aug 22, 2022 · 8 comments
Open

createIndex if 'NOT indexExists' fails (version 4.15) #283

jmayday opened this issue Aug 22, 2022 · 8 comments

Comments

@jmayday
Copy link

jmayday commented Aug 22, 2022

Demo project: https://github.com/jmayday/liquibase-demo
Liquibase version: 4.15.0 #4001 built at 2022-08-05 16:17+0000

I'm adding Liquibase integration to project and I would like to prepare a change set creating indexes if they are missing only. Someone already asked about indexExists precondition here #69 but there was no answer.

This is change set without any precondition and it fails with error below (which is all correct, as there is already index with such name):

<changeSet id="changeset_20220822_1250_1" author="jmayday">
    <ext:createIndex collectionName="testCollection">
        <ext:keys>
            { userId: 1, type: 1}
        </ext:keys>
        <ext:options>
            { unique: false, name: "userId_1" }
        </ext:options>
    </ext:createIndex>
</changeSet>

The error message (again - all is fine, index is already existing so we can't 2nd one with same name):

Caused by: com.mongodb.MongoCommandException: Command failed with error 86 (IndexKeySpecsConflict): 'Index must have unique name.The existing index: { v: 2, key: { userId: 1 }, name: "userId_1", ns: "dev1.testCollection" } has the same name as the requested index: { v: 2, key: { userId: 1, type: 1 }, name: "userId_1", ns: "dev1.testCollection" }' on server localhost:27017. The full response is {"ok": 0.0, "errmsg": "Index must have unique name.The existing index: { v: 2, key: { userId: 1 }, name: \"userId_1\", ns: \"dev1.testCollection\" } has the same name as the requested index: { v: 2, key: { userId: 1, type: 1 }, name: \"userId_1\", ns: \"dev1.testCollection\" }", "code": 86, "codeName": "IndexKeySpecsConflict"}
	at com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:198)

But if I extend the changeSet with preConditions, then an exception is being thrown:

<changeSet id="changeset_20220822_1250_1" author="jmayday">
    <preConditions onFail="MARK_RAN">
        <not>
            <indexExists indexName="userId_1"/>
        </not>
    </preConditions>
    <ext:createIndex collectionName="testCollection">
        <ext:keys>
            { userId: 1, type: 1}
        </ext:keys>
        <ext:options>
            { unique: false, name: "userId_1" }
        </ext:options>
    </ext:createIndex>
</changeSet>

The error:

Exception in thread "main" liquibase.exception.LiquibaseException: liquibase.exception.MigrationFailedException: Migration failed for changeset changesets/changeset_20220819_1428.xml::changeset_20220822_1250_1::jmayday:
     Reason: 
          changelog.xml : Index Exists Precondition: userId_1 : class liquibase.ext.mongodb.database.MongoConnection cannot be cast to class liquibase.database.jvm.JdbcConnection (liquibase.ext.mongodb.database.MongoConnection and liquibase.database.jvm.JdbcConnection are in unnamed module of loader 'app')

Dependency tree:

[INFO] com.myproject:liquibase:jar:1.0.0-SNAPSHOT
[INFO] +- org.liquibase.ext:liquibase-mongodb:jar:4.15.0:compile
[INFO] |  +- org.liquibase:liquibase-core:jar:4.15.0:compile
[INFO] |  |  +- javax.xml.bind:jaxb-api:jar:2.3.1:compile
[INFO] |  |  |  \- javax.activation:javax.activation-api:jar:1.2.0:compile
[INFO] |  |  +- org.yaml:snakeyaml:jar:1.27:compile
[INFO] |  |  \- com.opencsv:opencsv:jar:5.6:compile
[INFO] |  |     +- org.apache.commons:commons-lang3:jar:3.12.0:compile
[INFO] |  |     +- org.apache.commons:commons-text:jar:1.9:compile
[INFO] |  |     \- org.apache.commons:commons-collections4:jar:4.4:compile
[INFO] |  +- org.mongodb:mongodb-driver-sync:jar:4.6.1:compile
[INFO] |  +- org.mongodb:mongodb-driver-core:jar:4.6.1:compile
[INFO] |  |  \- org.mongodb:bson-record-codec:jar:4.6.1:runtime
[INFO] |  +- org.mongodb:bson:jar:4.6.1:compile
[INFO] |  +- com.fasterxml.jackson.core:jackson-databind:jar:2.13.3:compile
[INFO] |  |  +- com.fasterxml.jackson.core:jackson-annotations:jar:2.13.3:compile
[INFO] |  |  \- com.fasterxml.jackson.core:jackson-core:jar:2.13.3:compile
[INFO] |  \- org.projectlombok:lombok:jar:1.18.22:compile
[INFO] +- commons-cli:commons-cli:jar:1.5.0:compile
[INFO] \- ch.qos.logback:logback-classic:jar:1.2.11:compile
[INFO]    +- ch.qos.logback:logback-core:jar:1.2.11:compile
[INFO]    \- org.slf4j:slf4j-api:jar:1.7.30:compile

When debugging I find method liquibase.snapshot.JdbcDatabaseSnapshot#getMetaDataFromCache to do casting which throws exception because getDatabase().getConnection() is a MongoConnection:

public CachingDatabaseMetaData getMetaDataFromCache() throws SQLException {
    if (cachingDatabaseMetaData == null) {
        DatabaseMetaData databaseMetaData = null;
        if (getDatabase().getConnection() != null) {
            databaseMetaData = ((JdbcConnection) getDatabase().getConnection()).getUnderlyingConnection().getMetaData();
        }

        cachingDatabaseMetaData = new CachingDatabaseMetaData(this.getDatabase(), databaseMetaData);
    }
    return cachingDatabaseMetaData;
}

Whole stacktrace below:

Exception in thread "main" liquibase.exception.LiquibaseException: liquibase.exception.MigrationFailedException: Migration failed for changeset changesets/changeset_20220819_1428.xml::changeset_20220822_1250_1::jmayday:
     Reason: 
          changelog.xml : Index Exists Precondition: userId_1 : class liquibase.ext.mongodb.database.MongoConnection cannot be cast to class liquibase.database.jvm.JdbcConnection (liquibase.ext.mongodb.database.MongoConnection and liquibase.database.jvm.JdbcConnection are in unnamed module of loader 'app')

	at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:126)
	at liquibase.Liquibase.lambda$null$0(Liquibase.java:263)
	at liquibase.Scope.lambda$child$0(Scope.java:180)
	at liquibase.Scope.child(Scope.java:189)
	at liquibase.Scope.child(Scope.java:179)
	at liquibase.Scope.child(Scope.java:158)
	at liquibase.Scope.child(Scope.java:243)
	at liquibase.Liquibase.lambda$update$1(Liquibase.java:262)
	at liquibase.Scope.lambda$child$0(Scope.java:180)
	at liquibase.Scope.child(Scope.java:189)
	at liquibase.Scope.child(Scope.java:179)
	at liquibase.Scope.child(Scope.java:158)
	at liquibase.Liquibase.runInScope(Liquibase.java:2414)
	at liquibase.Liquibase.update(Liquibase.java:209)
	at liquibase.Liquibase.update(Liquibase.java:195)
	at liquibase.Liquibase.update(Liquibase.java:191)
	at liquibase.Liquibase.update(Liquibase.java:183)
	at com.myproject.Liqui.main(Liqui.java:62)
Caused by: liquibase.exception.MigrationFailedException: Migration failed for changeset changesets/changeset_20220819_1428.xml::changeset_20220822_1250_1::jmayday:
     Reason: 
          changelog.xml : Index Exists Precondition: userId_1 : class liquibase.ext.mongodb.database.MongoConnection cannot be cast to class liquibase.database.jvm.JdbcConnection (liquibase.ext.mongodb.database.MongoConnection and liquibase.database.jvm.JdbcConnection are in unnamed module of loader 'app')

	at liquibase.changelog.ChangeSet.execute(ChangeSet.java:632)
	at liquibase.changelog.visitor.UpdateVisitor.visit(UpdateVisitor.java:56)
	at liquibase.changelog.ChangeLogIterator$2.lambda$null$0(ChangeLogIterator.java:113)
	at liquibase.Scope.lambda$child$0(Scope.java:180)
	at liquibase.Scope.child(Scope.java:189)
	at liquibase.Scope.child(Scope.java:179)
	at liquibase.Scope.child(Scope.java:158)
	at liquibase.changelog.ChangeLogIterator$2.lambda$run$1(ChangeLogIterator.java:112)
	at liquibase.Scope.lambda$child$0(Scope.java:180)
	at liquibase.Scope.child(Scope.java:189)
	at liquibase.Scope.child(Scope.java:179)
	at liquibase.Scope.child(Scope.java:158)
	at liquibase.Scope.child(Scope.java:243)
	at liquibase.changelog.ChangeLogIterator$2.run(ChangeLogIterator.java:93)
	at liquibase.Scope.lambda$child$0(Scope.java:180)
	at liquibase.Scope.child(Scope.java:189)
	at liquibase.Scope.child(Scope.java:179)
	at liquibase.Scope.child(Scope.java:158)
	at liquibase.Scope.child(Scope.java:243)
	at liquibase.Scope.child(Scope.java:247)
	at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:65)
	... 17 more
Caused by: liquibase.exception.PreconditionErrorException: Precondition Error
	at liquibase.precondition.core.IndexExistsPrecondition.check(IndexExistsPrecondition.java:123)
	at liquibase.precondition.core.NotPrecondition.check(NotPrecondition.java:35)
	at liquibase.precondition.core.AndPrecondition.check(AndPrecondition.java:40)
	at liquibase.precondition.core.PreconditionContainer.check(PreconditionContainer.java:213)
	at liquibase.changelog.ChangeSet.execute(ChangeSet.java:589)
	... 37 more
Caused by: liquibase.exception.DatabaseException: java.lang.ClassCastException: class liquibase.ext.mongodb.database.MongoConnection cannot be cast to class liquibase.database.jvm.JdbcConnection (liquibase.ext.mongodb.database.MongoConnection and liquibase.database.jvm.JdbcConnection are in unnamed module of loader 'app')
	at liquibase.snapshot.jvm.IndexSnapshotGenerator.snapshotObject(IndexSnapshotGenerator.java:304)
	at liquibase.snapshot.jvm.JdbcSnapshotGenerator.snapshot(JdbcSnapshotGenerator.java:66)
	at liquibase.snapshot.SnapshotGeneratorChain.snapshot(SnapshotGeneratorChain.java:49)
	at liquibase.snapshot.DatabaseSnapshot.include(DatabaseSnapshot.java:312)
	at liquibase.snapshot.DatabaseSnapshot.init(DatabaseSnapshot.java:105)
	at liquibase.snapshot.DatabaseSnapshot.<init>(DatabaseSnapshot.java:58)
	at liquibase.snapshot.JdbcDatabaseSnapshot.<init>(JdbcDatabaseSnapshot.java:34)
	at liquibase.snapshot.SnapshotGeneratorFactory.createSnapshot(SnapshotGeneratorFactory.java:215)
	at liquibase.snapshot.SnapshotGeneratorFactory.createSnapshot(SnapshotGeneratorFactory.java:244)
	at liquibase.snapshot.SnapshotGeneratorFactory.has(SnapshotGeneratorFactory.java:133)
	at liquibase.precondition.core.IndexExistsPrecondition.check(IndexExistsPrecondition.java:103)
	... 41 more
Caused by: java.lang.ClassCastException: class liquibase.ext.mongodb.database.MongoConnection cannot be cast to class liquibase.database.jvm.JdbcConnection (liquibase.ext.mongodb.database.MongoConnection and liquibase.database.jvm.JdbcConnection are in unnamed module of loader 'app')
	at liquibase.snapshot.JdbcDatabaseSnapshot.getMetaDataFromCache(JdbcDatabaseSnapshot.java:45)
	at liquibase.snapshot.jvm.IndexSnapshotGenerator.snapshotObject(IndexSnapshotGenerator.java:159)
	... 51 more
@jmayday jmayday changed the title Can't make indexExists precondition working Can't make indexExists precondition working (version 4.15) Aug 22, 2022
@jmayday jmayday changed the title Can't make indexExists precondition working (version 4.15) createIndex if 'NOT indexExists' (version 4.15) Aug 22, 2022
@jmayday jmayday changed the title createIndex if 'NOT indexExists' (version 4.15) createIndex if 'NOT indexExists' fails (version 4.15) Aug 23, 2022
@r-michal-ah
Copy link

hello, any update on it?

@kb-mendozaACN
Copy link

Can we have this fixed pls?

@jmayday
Copy link
Author

jmayday commented Sep 21, 2023

@r-michal-ah @kb-mendozaACN I don't think it's worth waiting. I've started using Mongock, maybe it will be good choice for you guys.

@kb-mendozaACN
Copy link

kb-mendozaACN commented Sep 21, 2023

@jmayday unfortunately, we can not introduce more tools. have you tried creating via mongodb:runCommand? I wanted to try it as well, but checking here first, just in case someone already tried and failed too.

@jmayday
Copy link
Author

jmayday commented Sep 21, 2023

Sorry, I don't understand. What you mean exactly?

@jmayday
Copy link
Author

jmayday commented Sep 21, 2023

I think I tried. But I don't remember exactly, as it was year ago :). I think I exhausted options to use liquibase with Mongo, and there was no support at all (as we see in this thread), so I started checking alternatives and ended up using Mongock.

@pcalouche
Copy link

I would love to see this implemented to. The workaround I found was not fail the change set if there was an error. Better than nothing.

<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
        https://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-latest.xsd
        http://www.liquibase.org/xml/ns/dbchangelog-ext https://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd">

  <!--
    Mongo Liquibase does not have a precondition to check if an index exists, so set
    failOnError to be false. See https://github.com/liquibase/liquibase-mongodb/issues/283.
  -->
  <changeSet id="4" author="psc" failOnError="false">
    <ext:dropIndex collectionName="testers">
      <ext:keys>{firstName: 1}</ext:keys>
    </ext:dropIndex>
    <rollback>
      <ext:createIndex collectionName="testers">
        <ext:keys>{firstName: 1}</ext:keys>
        <ext:options>{name: "firstName_asc"}</ext:options>
      </ext:createIndex>
    </rollback>
    <comment>
      Drops the index on the firstName field in the testers collection
    </comment>
  </changeSet>

</databaseChangeLog>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants