txn: make the unique key lock behaviour consistent for all executors

[cfzjywxk]

Development Task

After #21483, we'll try to lock all the related unique keys for pessimistic transactions if the executor is pointGet or batchPointGet. For other coprocessor related executors, we should also implement this for them, and make the lock behaviours consistent.
Tasks are:

• Try to lock all unique keys for existing data for all executors, make them behave the same.
• Make a detailed and improved document about the pessimistic lock behaviours of TiDB.

Another possible solution is to push down the pessimistic lock operations into the kv requests doing data read, the difference behaviours could be resolved, check rfc for more details.

[zhaoxugang]

I want to join to resolve this issue, what can i do for this issue?@cfzjywxk

[cfzjywxk]

I want to join to resolve this issue, what can i do for this issue?@cfzjywxk

@zhaoxugang
I think maybe we need to to find out a way to lock the existing unique index keys for pessimistic transaction in selectLockExec executor, currently only row keys are locked.

[zhaoxugang]

the query by unique index keys will be executed by PointGetExec ot BatchPointGetExec, so i don't know what can i do for resolve the issue@you06,@cfzjywxk

[you06]

the query by unique index keys will be executed by PointGetExec ot BatchPointGetExec, so i don't know what can i do for resolve the issue@you06,@cfzjywxk

In a short word, current PointGetExec and BatchPointGetExec will lock exist/unexist keys, however other executors will lock exist keys only. With the executor choice in planner module, it would be confusing for users.

eg.

-- id is a unique key, val is not
select * from t where id = 1 and val = 1; -- sql1, lock unexist key(id  = 1)
select * from t where val = 1; -- sql2, does not lock any keys if val = 1 not exist

Notice that sql2 takes a looser condition, it should cover the effects which sql1 makes. However, it does not lock any keys actually. Besides, the push-down requests scan need-lock keys in coprocessor in TiKV, there will be a great performance impact if we easily lock the query range for non-unique keys.

In a short word, this issue doesn't have a well design yet.

[zhaoxugang]

got it,thx

[zhaoxugang]

I have an idea , It is i choose which index to lock dependency access path of data source . If there not exists index that was choosing , i will lock the table .
@you06

[you06]

I have an idea , It is i choose which index to lock dependency access path of data source . If there not exists index that was choosing , i will lock the table .
@you06

Lock table sounds terrible, that will introduce a great performance impact.

[dwangxxx]

the query by unique index keys will be executed by PointGetExec ot BatchPointGetExec, so i don't know what can i do for resolve the issue@you06,@cfzjywxk

In a short word, current PointGetExec and BatchPointGetExec will lock exist/unexist keys, however other executors will lock exist keys only. With the executor choice in planner module, it would be confusing for users.

eg.

-- id is a unique key, val is not
select * from t where id = 1 and val = 1; -- sql1, lock unexist key(id  = 1)
select * from t where val = 1; -- sql2, does not lock any keys if val = 1 not exist

Notice that sql2 takes a looser condition, it should cover the effects which sql1 makes. However, it does not lock any keys actually. Besides, the push-down requests scan need-lock keys in coprocessor in TiKV, there will be a great performance impact if we easily lock the query range for non-unique keys.

In a short word, this issue doesn't have a well design yet.

@you06
The execution plan of select * from t where id = 1 and v = 1 and val = 30 for update; is like this:

It will not lock any unexist keys. So in this situation, the task of this issue is to lock the unexist key(id = 1 and v = 1) in SelectLockExec or IndexLookUpExec to block other txn to update or insert at key(id = 1 and v = 1)?

[you06]

@dwangxxx It seems your case talks about the table like create table t(id int, v int, val int, unique index u(id ,v));.

I don't mean it needs to lock the unexist key when with predicate id = 1 and v = 1 and val = 30 with empty result. Because id = 1 and v = 1 is wider than id = 1 and v = 1 and val = 30, so id = 1 and v = 1 locks the unexist key and id = 1 and v = 1 and val = 30 does not lock the unexist key is ok to me(just my personal opinion).
The ideal model for me is supporting range lock, so that when using predicate v = 1 will lock more keys than id = 1 and v = 1.

[dwangxxx]

@dwangxxx It seems your case talks about the table like create table t(id int, v int, val int, unique index u(id ,v));.

I don't mean it needs to lock the unexist key when with predicate id = 1 and v = 1 and val = 30 with empty result. Because id = 1 and v = 1 is wider than id = 1 and v = 1 and val = 30, so id = 1 and v = 1 locks the unexist key and id = 1 and v = 1 and val = 30 does not lock the unexist key is ok to me(just my personal opinion). The ideal model for me is supporting range lock, so that when using predicate v = 1 will lock more keys than id = 1 and v = 1.

@you06
So the ideal model you mentioned is like this?

/* t1 */ begin pessimistic;
/* t2 */ begin pessimistic;
/* t1 */ select * from t where id = 1 for update;  -- this will lock all keys id = 1
/* t2 */ insert into t values(1, 2, 3); -- this will be blocked until t1 commit

[you06]

@you06 So the ideal model you mentioned is like this?

/* t1 */ begin pessimistic;
/* t2 */ begin pessimistic;
/* t1 */ select * from t where id = 1 for update;  -- this will lock all keys id = 1
/* t2 */ insert into t values(1, 2, 3); -- this will be blocked until t1 commit

Yes.

[dwangxxx]

@you06
I'm a little confused. How about the non-unique key? I list some situations, could you help figure out which should be blocked or not? Suppose table t is empty. Thanks~

• test1

/* t1 */ begin pessimistic;
/* t2 */ begin pessimistic;
/* t1 */ select * from t where id = 1 for update; 
/* t2 */ insert into t values(1, x, x); 

• test2

/* t1 */ begin pessimistic;
/* t2 */ begin pessimistic;
/* t1 */ select * from t where id = 1 and v = 1 for update; 
/* t2 */ insert into t values(1, 1, x); 

• test3

/* t1 */ begin pessimistic;
/* t2 */ begin pessimistic;
/* t1 */ select * from t where id = 1 and v = 1 and val = 3 for update; 
/* t2 */ insert into t values(1, 1, x); 

• test4

/* t1 */ begin pessimistic;
/* t2 */ begin pessimistic;
/* t1 */ select * from t where val = 3 for update; 
/* t2 */ insert into t values(x, x, 3); 

• test5

/* t1 */ begin pessimistic;
/* t2 */ begin pessimistic;
/* t1 */ select * from t where id = 1 and val = 3 for update; 
/* t2 */ insert into t values(1, x, x);