CosmosDB state stores changes order of operations in transaction

[artursouza]

Expected Behavior

When an app performs the following transaction in state store:

upsert key1 value1
delete key1
upsert key2 value2

The expected result in the database is that key1 is not present and key2 is present, respecting the order of the operations.

Actual Behavior

The result in the database is that key1 and key2 are present, not respecting the order of the operations.

Steps to Reproduce the Problem

Perform a transaction operation in CosmosDB state store.

Release Note

RELEASE NOTE: FIX CosmosDb transaction API to respect order of operations

[artursouza]

This requires the stored procedure to be rewritten.

[dillenmeister]

1. If state operations with the same key appears more than once, isn't it always only the last operation that will matter? Would this be a solution instead of respecting the order, or are there use cases where you want to update the same state twice? The suggested conformance test (New conf test scenario for state stores: transaction order #1210) would pass if only the last operation per key was executed.

upsert key1 value1 // skip
delete key1
upsert key2 value2
delete key3 // skip
upsert key3 value3

2. Can two operations with the same key ever be in the same transaction if etags are included (first write concurrency mode)?
   Wouldn't this fail on the second operation? (note related issue Cosmos state store ignores ETags for multi operations #1244)

upsert key1 value1 etag1
delete key1 etag1 // fails because state with key1 has a new etag
upsert key2 value2 etag2

[shubham1172]

I was able to reproduce the issue with the following snippet. On every alternate run, I see that both k1 and k2 are present (problematic), and the other times only k2 is present (as expected).

/**
 * To reproduce #1209, perform the following in a transaction:
 * upsert k1, v1
 * delete k1
 * upsert k2, v2
 * 
 * Expected: k1 is not present, k2 is present with value v2
 * Actual: k1 and k2 are present with values v1 and v2
 * 
 * @param client DaprClient
 */
async function performOperations(client: DaprClient) {
    var operations = [
        {
            operation: "upsert",
            request: {
                key: "key1",
                value: {
                    part: "partition",
                    zipcode: 100
                }
            }
        },
        {
            operation: "delete",
            request: {
                key: "key1"
            }
        },
        {
            operation: "upsert",
            request: {
                key: "key2",
                value: {
                    part: "partition",
                    zipcode: 200
                }
            }
        }
    ];

    var metadata = {
        partitionKey: "partition"
    }

    await client.state.transaction(STATE_STORE_NAME, operations, metadata);
}

[shubham1172]

/assign

[mukundansundar]

@artursouza @yaron2 @shubham1172 In this case for transactions with first write concurrency, shouldn't all those transactions fail if the same key is updated more than once in the transaction?

[shubham1172]

#1244 seems to be covering it. Looking at the code, Cosmos DB transaction API completely ignores ETags today.