Draft: table level RAW #524

terry-xuan-gao · 2024-08-11T17:20:45Z

Related Issue(s) & Descriptions

Checklist

Tests were added or are not required
Documentation was added or is not required

Signed-off-by: Terry Gao <[email protected]>

gerayking · 2024-09-12T03:35:48Z

go/vt/vtgate/scatter_conn.go

@@ -296,6 +296,9 @@ func (stc *ScatterConn) ExecuteMultiShard(
 			if qr.SessionStateChanges != "" {
 				session.SetReadAfterWriteGTID(qr.SessionStateChanges)
 				stc.gateway.AddGtid(qr.SessionStateChanges)
+
+				tableName := primitive.GetTableName()
+				session.latestGTIDForTable.UpdateGTID(tableName, qr.SessionStateChanges)


Is it possible for network issues to cause a GTID rollback here?

For example:

SQL1 with GTID 1 is sent.
SQL2 with GTID 2 is sent.
MySQL server completes SQL1.
MySQL server completes SQL2.

Network error occurs.

Result 2 arrives at WeScale and triggers update_gtid(gtid2).
Result 1 arrives at WeScale and triggers update_gtid(gtid1).

The resulting GTID is 1.

Since session.GetReadAfterWrite().GetReadAfterWriteGtid() always stores the latest GTID at the session level. So, directly taking session.GetReadAfterWrite().GetReadAfterWriteGtid() to update latestGTIDForTable may avoid the problem of GTID rollback?

But I think what's more important here is that we should judge the sequence of the existing GTID and the GTID to be updated in latestGTIDForTable.
Since there is only one MySQL instance performing write operations, I understand that the returned GTIDs should all have the same UUID. Can I directly take the transaction number in the GTID for comparison?

Signed-off-by: Terry Gao <[email protected]>

terry-xuan-gao · 2024-09-26T16:58:12Z

when session1 has a long transaction on t1, other sessions can achieve blocking when querying t1 and non-blocking when querying t2.
However, set @@table_level ="off"; has not taken effect. After closing the table-level RAW, querying t2 is still not blocked.

* add: latest_gtid_for_table.go Signed-off-by: Terry Gao <[email protected]> * deal: safe_session.go with LatestGTIDForTable Signed-off-by: Terry Gao <[email protected]> * deal: scatter_conn.go with LatestGTIDForTable Signed-off-by: Terry Gao <[email protected]> * fix: e2e test Signed-off-by: Terry Gao <[email protected]> * fix: tools version latest->0.24.0 Signed-off-by: Terry Gao <[email protected]> * fix: tools version latest->0.24.0 Signed-off-by: Terry Gao <[email protected]> * feat: update gtid Signed-off-by: Terry Gao <[email protected]> * update: query.proto Signed-off-by: Terry Gao <[email protected]> * update: .pb.go Signed-off-by: Terry Gao <[email protected]> * fix: go.mod go.sum Signed-off-by: Terry Gao <[email protected]> * feat: get gtid map Signed-off-by: Terry Gao <[email protected]> * update: vtgate.proto Signed-off-by: Terry Gao <[email protected]> * add: getReadAfterWriteGtid() Signed-off-by: Terry Gao <[email protected]> * update: init opts.TableReadAfterWriteGtidMap Signed-off-by: Terry Gao <[email protected]> * fix: gtid rollback issue Signed-off-by: Terry Gao <[email protected]> * fix: init latestGTIDForTable in safeSession Signed-off-by: Terry Gao <[email protected]> * fix: add latest_gtid_for_table in ReadAfterWrite Signed-off-by: Terry Gao <[email protected]> * fix: scatter_conn.go Signed-off-by: Terry Gao <[email protected]> * fix: vtgate.proto Signed-off-by: Terry Gao <[email protected]> * fix: vtgate.proto Signed-off-by: Terry Gao <[email protected]> * fix: vtgate.proto Signed-off-by: Terry Gao <[email protected]> * fix: query_executor.go Signed-off-by: Terry Gao <[email protected]> * fix: query_executor.go Signed-off-by: Terry Gao <[email protected]> * fix: query_executor.go Signed-off-by: Terry Gao <[email protected]> * fix: query_executor.go Signed-off-by: Terry Gao <[email protected]> * fix: query_executor.go Signed-off-by: Terry Gao <[email protected]> * fix: query_executor.go Signed-off-by: Terry Gao <[email protected]> * fix: scattor_conn.go Signed-off-by: Terry Gao <[email protected]> * fix: scattor_conn.go Signed-off-by: Terry Gao <[email protected]> * fix: scattor_conn.go Signed-off-by: Terry Gao <[email protected]> * fix: safe_session.go Signed-off-by: Terry Gao <[email protected]> * fix: safe_session.go Signed-off-by: Terry Gao <[email protected]> * fix: safe_session.go Signed-off-by: Terry Gao <[email protected]> * fix: getRAWGtid Signed-off-by: Terry Gao <[email protected]> * feat: support set table_level Signed-off-by: Terry Gao <[email protected]> * fix: getRAWGtid Signed-off-by: Terry Gao <[email protected]> --------- Signed-off-by: Terry Gao <[email protected]>

Signed-off-by: terry-xuan-gao <[email protected]>

terry-xuan-gao added 6 commits August 11, 2024 17:16

add: latest_gtid_for_table.go

9a69582

Signed-off-by: Terry Gao <[email protected]>

deal: safe_session.go with LatestGTIDForTable

7e760f4

Signed-off-by: Terry Gao <[email protected]>

deal: scatter_conn.go with LatestGTIDForTable

db2b0a4

Signed-off-by: Terry Gao <[email protected]>

fix: e2e test

3c1c947

Signed-off-by: Terry Gao <[email protected]>

fix: tools version latest->0.24.0

2ffe951

Signed-off-by: Terry Gao <[email protected]>

fix: tools version latest->0.24.0

8f50f5f

Signed-off-by: Terry Gao <[email protected]>

gerayking requested review from earayu, gerayking and newborn22 September 12, 2024 07:03

terry-xuan-gao added 4 commits September 12, 2024 15:59

feat: update gtid

7241a8e

Signed-off-by: Terry Gao <[email protected]>

update: query.proto

9417b7c

Signed-off-by: Terry Gao <[email protected]>

update: .pb.go

d013467

Signed-off-by: Terry Gao <[email protected]>

fix: go.mod go.sum

4843ef7

Signed-off-by: Terry Gao <[email protected]>

gerayking reviewed Sep 14, 2024

View reviewed changes

terry-xuan-gao added 16 commits September 15, 2024 16:53

feat: get gtid map

042d799

Signed-off-by: Terry Gao <[email protected]>

update: vtgate.proto

db1c431

Signed-off-by: Terry Gao <[email protected]>

add: getReadAfterWriteGtid()

c4a81bf

Signed-off-by: Terry Gao <[email protected]>

update: init opts.TableReadAfterWriteGtidMap

55ca658

Signed-off-by: Terry Gao <[email protected]>

fix: gtid rollback issue

f969358

Signed-off-by: Terry Gao <[email protected]>

fix: init latestGTIDForTable in safeSession

f3fd223

Signed-off-by: Terry Gao <[email protected]>

fix: add latest_gtid_for_table in ReadAfterWrite

0819854

Signed-off-by: Terry Gao <[email protected]>

fix: scatter_conn.go

5f4a287

Signed-off-by: Terry Gao <[email protected]>

fix: vtgate.proto

7269c30

Signed-off-by: Terry Gao <[email protected]>

fix: vtgate.proto

43f38f6

Signed-off-by: Terry Gao <[email protected]>

fix: vtgate.proto

ee20ef2

Signed-off-by: Terry Gao <[email protected]>

fix: query_executor.go

9948c39

Signed-off-by: Terry Gao <[email protected]>

fix: query_executor.go

c655033

Signed-off-by: Terry Gao <[email protected]>

fix: query_executor.go

a256a89

Signed-off-by: Terry Gao <[email protected]>

fix: query_executor.go

337e205

Signed-off-by: Terry Gao <[email protected]>

fix: query_executor.go

0f8711f

Signed-off-by: Terry Gao <[email protected]>

terry-xuan-gao added 10 commits September 25, 2024 15:08

fix: query_executor.go

51d7751

Signed-off-by: Terry Gao <[email protected]>

fix: scattor_conn.go

6d44dfb

Signed-off-by: Terry Gao <[email protected]>

fix: scattor_conn.go

05363b6

Signed-off-by: Terry Gao <[email protected]>

fix: scattor_conn.go

01478cb

Signed-off-by: Terry Gao <[email protected]>

fix: safe_session.go

5b55897

Signed-off-by: Terry Gao <[email protected]>

fix: safe_session.go

905d205

Signed-off-by: Terry Gao <[email protected]>

fix: safe_session.go

72432fc

Signed-off-by: Terry Gao <[email protected]>

fix: getRAWGtid

63916e7

Signed-off-by: Terry Gao <[email protected]>

feat: support set table_level

f34339d

Signed-off-by: Terry Gao <[email protected]>

fix: getRAWGtid

a27e2cb

Signed-off-by: Terry Gao <[email protected]>

terry-xuan-gao added 5 commits September 28, 2024 19:10

fix: reuse Parse(sql)

2d40c7f

Signed-off-by: terry-xuan-gao <[email protected]>

Merge branch 'feature/table_level_RAW' into ospp-dev

d391498

fix: reuse Parse(sql)

fe5575b

Signed-off-by: terry-xuan-gao <[email protected]>

fix: function name

d9b4d32

Signed-off-by: terry-xuan-gao <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Draft: table level RAW #524

Draft: table level RAW #524

terry-xuan-gao commented Aug 11, 2024

gerayking Sep 12, 2024

terry-xuan-gao Sep 22, 2024

terry-xuan-gao commented Sep 26, 2024

Draft: table level RAW #524

Are you sure you want to change the base?

Draft: table level RAW #524

Conversation

terry-xuan-gao commented Aug 11, 2024

Related Issue(s) & Descriptions

Checklist

gerayking Sep 12, 2024

Choose a reason for hiding this comment

terry-xuan-gao Sep 22, 2024

Choose a reason for hiding this comment

terry-xuan-gao commented Sep 26, 2024