Multiple entries with same key exception

[LoverAndrew]

• Version: 0.5.1
• Storage Backend: Hbase
• Mixed Index Backend: elasticsearch 6.6.0
• Steps to Reproduce: Using Atlas to create typedefs, delete it, then create it again, finally can not delete it with exception. Both at Atlas and Gremlin .

Stack Trace (if you have one)

2883594 [gremlin-server-exec-7] ERROR org.janusgraph.graphdb.database.StandardJanusGraph  - Could not commit transaction [1] due to exception
java.lang.IllegalArgumentException: Multiple entries with same key: department.department_code=org.janusgraph.diskstorage.indexing.StandardKeyInformation@478572cd and department.department_code=org.janusgraph.diskstorage.indexing.StandardKeyInformation@7cf9c550
        at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:150)
        at com.google.common.collect.RegularImmutableMap.checkNoConflictInBucket(RegularImmutableMap.java:104)
        at com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:70)
        at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:254)
        at org.janusgraph.graphdb.database.IndexSerializer$IndexInfoRetriever$1.get(IndexSerializer.java:165)
        at org.janusgraph.diskstorage.indexing.IndexTransaction.getIndexMutation(IndexTransaction.java:82)
        at org.janusgraph.diskstorage.indexing.IndexTransaction.delete(IndexTransaction.java:75)
        at org.janusgraph.graphdb.database.StandardJanusGraph.prepareCommit(StandardJanusGraph.java:649)
        at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:731)
        at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1425)
        at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:297)
        at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104)
        at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.lambda$commitAll$2(DefaultGraphManager.java:131)        at java.base/java.util.concurrent.ConcurrentHashMap$EntrySetView.forEach(ConcurrentHashMap.java:4864)
        at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.commitAll(DefaultGraphManager.java:128)
        at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.attemptCommit(AbstractOpProcessor.java:371)
        at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.handleIterator(AbstractOpProcessor.java:97)
        at org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor.lambda$evalOpInternal$5(AbstractEvalOpProcessor.java:264)
        at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:278)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
2883824 [gremlin-server-exec-7] WARN  org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor  - Exception processing a script on request [RequestMessage{, requestId=4faabff3-b0f0-4837-a6f3-1aa958989a97, op='eval', processor='', args={gremlin=g.V().has('__type_servicetype','government').drop(), userAgent=Gremlin Console/3.4.6, batchSize=64}}].
java.lang.IllegalArgumentException: Multiple entries with same key: department.department_code=org.janusgraph.diskstorage.indexing.StandardKeyInformation@478572cd and department.department_code=org.janusgraph.diskstorage.indexing.StandardKeyInformation@7cf9c550
        at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:150)
        at com.google.common.collect.RegularImmutableMap.checkNoConflictInBucket(RegularImmutableMap.java:104)
        at com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:70)
        at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:254)
        at org.janusgraph.graphdb.database.IndexSerializer$IndexInfoRetriever$1.get(IndexSerializer.java:165)
        at org.janusgraph.diskstorage.indexing.IndexTransaction.getIndexMutation(IndexTransaction.java:82)
        at org.janusgraph.diskstorage.indexing.IndexTransaction.delete(IndexTransaction.java:75)
        at org.janusgraph.graphdb.database.StandardJanusGraph.prepareCommit(StandardJanusGraph.java:649)
        at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:731)
        at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1425)
        at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:297)
        at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104)
        at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.lambda$commitAll$2(DefaultGraphManager.java:131)        at java.base/java.util.concurrent.ConcurrentHashMap$EntrySetView.forEach(ConcurrentHashMap.java:4864)
        at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.commitAll(DefaultGraphManager.java:128)
        at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.attemptCommit(AbstractOpProcessor.java:371)
        at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.handleIterator(AbstractOpProcessor.java:97)
        at org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor.lambda$evalOpInternal$5(AbstractEvalOpProcessor.java:264)
        at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:278)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
2935324 [gremlin-server-exec-8] WARN  org.janusgraph.graphdb.transaction.StandardJanusGraphTx  - Query requires iterating over all vertices [()]. For better performance, use indexes

[li-boxuan]

@LoverAndrew Can you give a code example using gremlin which can reproduce this bug?

[LoverAndrew]

@LoverAndrew Can you give a code example using gremlin which can reproduce this bug?

Actually, I used Apache Atlas to create typeDefs using RestAPI(/v2/types/typedefs) , after do create, delete , create, then fiailed to delete it, the finally exception coming from Janusgraph. also failed to delete it using gremlin g.V().has('__type_servicetype','government').drop().

I will try to trace the final request data from Atlas to JanusGraph later.

The attached file is the request API.

government_department_office_duty.txt

[li-boxuan]

I will try to trace the final request data from Atlas to JanusGraph later.

Yeah that will be helpful. I presume most people here don't know much about Atlas.

[nshntarora]

Hey @LoverAndrew,

It doesn't look like an issue with Janusgraph but one with Atlas. I faced the same issue which made me land on this thread, I think I've found the cause for the issue. Here's how this can be reproduced.

1. Create a Business Metadata named “Test”, and add an attribute “Attr” to it
2. Then add the business metadata attribute to any asset
3. Now, go and delete the BM, “Test” you created in step 1
4. Now create a new business metadata with the same name “Test” and the same attribute name “Attr”

Atlas is now in an unusable state, and all writes to Janusgraph now fail.

So, it looks like Atlas doesn't support deleting Business Metadata

[li-boxuan]

This issue has been converted to GitHub discussion. If we can reproduce it in JanusGraph using gremlin, we can open a new bug issue.