Skip to content

Concurrent upsert causes deadlock. Is it normal? #12366

@VasilyMelnik

Description

@VasilyMelnik

Hello everyone!
I conduct small perftest on my local deplyment of ignite3, with 2 threads making 50% upsert and 50% get operations on random index:

public static void main(String[] args) throws Exception
    {
        String[] addresses = {
                "localhost:10800",
                "localhost:10801",
                "localhost:10802"
        };       
        try (IgniteClient client = IgniteClient.builder()
                .addresses(addresses)
                .connectTimeout(30000)
                .build()) {

            Runnable runnable = new Runnable() {
                @Override
                public void run() {
                    Random randomOp = new Random();
                    Random randomIndex = new Random();
                    Table perftest =client.tables().table("perftest");
                    long start = System.currentTimeMillis();
                    int N = 70000;
                    for (int i=0;i<N; i++) {
                        // 50% upsert, 50% get
                        if (randomOp.nextInt(10) <5 )
                                perftest.recordView().upsert(null, Tuple.create().set("id", randomIndex.nextLong(1000)).set("val", RandGeneratedStr(1000)));
                            else
                                perftest.recordView().get(null,Tuple.create().set("id", randomIndex.nextLong(1000)));
                    }
                    long end =System.currentTimeMillis();
                    System.out.println("Rate: "+N*1.0*1000/(end-start));
                }
            };
          Thread t1 = new Thread(runnable);
          Thread t2 = new Thread(runnable);

          t1.start();
          t2.start();

          t1.join();
          t2.join();
        }
    }

And when i run two concurrent threads, i get one of them failed with:
IGN-TX-4 TraceId:205ebf31-614f-4ab1-bb3c-c017ab57ec09 Failed to acquire a lock due to a possible deadlock
I see, that each upsert requires X lock on key (smth like this LockKey [ctx=42_part_11, key=RowId [partitionId=11, uuid=00000199-7235-8748-3904-4e30efc61428]]) and get requires S lock on key. All upserts by one key should be serialized? getiing X-lock one by one. So what is the reason of deadlock and how can i prevent it in concurrent workloads?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions