NVRM: Xid (PCI:0000:01:00): 119, pid=950, name=Xorg, Timeout after 45s of waiting for RPC response from GPU0 GSP!

[zhangboin]

The NVIDIA GeForce GTX 1660s GPU uses the 580.82.07 open-source driver module. Occasionally, when executing the NVIdia-sleep.sh suspend command to enter standby, errors such as Xid 119 and Xid 120 GSP timeouts occur. How can this kind of problem be solved?

The detailed kernel log is as follows:
Nov 15 18:04:45 localhost kernel: [26702.334337] NVRM: kgspHealthCheck_TU102: ****************************** GSP-CrashCat Report ******************************* Nov 15 18:04:45 localhost kernel: [26702.334344] NVRM: GPU at PCI:0000:01:00: GPU-0c1b77a3-5728-c13a-5865-1918d8edc88d Nov 15 18:04:45 localhost kernel: [26702.334346] NVRM: Xid (PCI:0000:01:00): 120, pid=950, name=Xorg, GSP task exception: load access fault (cause:0x5) @ pc:0x527fc30, task:1 Nov 15 18:04:45 localhost kernel: [26702.334351] NVRM: Reported by libos task:0 v2.0 [0] @ ts:1763201085 Nov 15 18:04:45 localhost kernel: [26702.334352] NVRM: RISC-V CSR State: Nov 15 18:04:45 localhost kernel: [26702.334353] NVRM: mstatus:0x000000001e000000 mscratch:0x0000000000000000 mie:0x0000000000000880 mip:0x0000000000000000 Nov 15 18:04:45 localhost kernel: [26702.334354] NVRM: mepc:0x000000000527fc30 mbadaddr:0x00000000000001c0 mcause:0x0000000000000005 Nov 15 18:04:45 localhost kernel: [26702.334356] NVRM: RISC-V GPR State: Nov 15 18:04:45 localhost kernel: [26702.334356] NVRM: ra:0x000000000527fc30 sp:0x0000000005d6a160 gp:0x0000000000000000 tp:0x0000000000000000 Nov 15 18:04:45 localhost kernel: [26702.334357] NVRM: a0:0x0000000000000000 a1:0x0000000000335775 a2:0x00000000040219c8 a3:0x0000000000f8f965 Nov 15 18:04:45 localhost kernel: [26702.334359] NVRM: a4:0x0000000004057fc0 a5:0x00000000040219c8 a6:0x0000000005d6a100 a7:0x0000000000000008 Nov 15 18:04:45 localhost kernel: [26702.334360] NVRM: s0:0x0000000005d6a2c0 s1:0x0000000000335775 s2:0x00000000041f3000 s3:0x0000000000002000 Nov 15 18:04:45 localhost kernel: [26702.334361] NVRM: s4:0x8000000000187b20 s5:0x0000000000000004 s6:0x000000000000000f s7:0x0000000000000065 Nov 15 18:04:45 localhost kernel: [26702.334362] NVRM: s8:0x00000000048678f0 s9:0x8000000000322c70 s10:0x0000000000002000 s11:0x80000000001694f0 Nov 15 18:04:45 localhost kernel: [26702.334363] NVRM: t0:0x0000000000000012 t1:0x0000000005c0b694 t2:0x0000000000000000 t3:0x0000000001000001 Nov 15 18:04:45 localhost kernel: [26702.334365] NVRM: t4:0x80000000001674f0 t5:0x0000000000000038 t6:0x0000000000000000 Nov 15 18:04:45 localhost kernel: [26702.334366] NVRM: Stack Trace: Nov 15 18:04:45 localhost kernel: [26702.334366] NVRM: 0x000000000527fc30 Nov 15 18:04:45 localhost kernel: [26702.334367] NVRM: 0x0000000005293d78 Nov 15 18:04:45 localhost kernel: [26702.334367] NVRM: 0x000000000529423c Nov 15 18:04:45 localhost kernel: [26702.334368] NVRM: 0x000000000525ca50 Nov 15 18:04:45 localhost kernel: [26702.334369] NVRM: 0x000000000525cbd0 Nov 15 18:04:45 localhost kernel: [26702.334369] NVRM: 0x000000000525cc98 Nov 15 18:04:45 localhost kernel: [26702.334370] NVRM: 0x0000000005561948 Nov 15 18:04:45 localhost kernel: [26702.334370] NVRM: 0x0000000005af8604 Nov 15 18:04:45 localhost kernel: [26702.334371] NVRM: 0x0000000005a0b190 Nov 15 18:04:45 localhost kernel: [26702.334372] NVRM: 0x00000000059fc3d4 Nov 15 18:04:45 localhost kernel: [26702.334372] NVRM: 0x00000000059f5be4 Nov 15 18:04:45 localhost kernel: [26702.334373] NVRM: 0x00000000059f84ac Nov 15 18:04:45 localhost kernel: [26702.334373] NVRM: 0x0000000004d8e268 Nov 15 18:04:45 localhost kernel: [26702.334374] NVRM: 0x0000000005c086a4 Nov 15 18:04:45 localhost kernel: [26702.334374] NVRM: 0x0000000005c21c14 Nov 15 18:04:45 localhost kernel: [26702.334375] NVRM: 0x0000000005c221f0 Nov 15 18:04:45 localhost kernel: [26702.334376] NVRM: 0x0000000005c25d4c Nov 15 18:04:45 localhost kernel: [26702.334376] NVRM: PC Trace: Nov 15 18:04:45 localhost kernel: [26702.334377] NVRM: 0x0000000004018064 0x000000000401a8a8 0x0000000004018d0c 0x000000000400e35c 0x0000000004018c4c Nov 15 18:04:45 localhost kernel: [26702.334378] NVRM: 0x000000000401832c 0x000000000400e35c 0x00000000040183a0 0x0000000004018c3c 0x0000000004018110 Nov 15 18:04:45 localhost kernel: [26702.334379] NVRM: 0x0000000004018c1c 0x000000000401aa84 0x0000000004018a84 0x00000000040181d8 0x0000000004018b90 Nov 15 18:04:45 localhost kernel: [26702.334380] NVRM: 0x000000000401832c 0x000000000400e35c 0x00000000040183a0 0x0000000004018bc8 0x00000000040181d8 Nov 15 18:04:45 localhost kernel: [26702.334381] NVRM: 0x0000000004018b90 0x000000000401832c Nov 15 18:04:45 localhost kernel: [26702.334382] NVRM: External I/O Register State: Nov 15 18:04:45 localhost kernel: [26702.334382] NVRM: 0x00111360:0x00000000 0x00111364:0xbadf5041 0x00111368:0x0010ec10 0x0011136c:0x00000000 Nov 15 18:04:45 localhost kernel: [26702.334384] NVRM: 0x001112b4:0x00040040 0x001112b8:0x00000040 0x001112bc:0x00000000 0x00111344:0x11100000 Nov 15 18:04:45 localhost kernel: [26702.334386] NVRM: 0x00110008:0x00000010 0x0011010c:0x00000000 0x00110118:0x00012022 0x00110110:0x01296410 Nov 15 18:04:45 localhost kernel: [26702.334387] NVRM: 0x00110128:0x00000000 0x00110114:0x0000e09c 0x0011011c:0x00000020 Nov 15 18:04:45 localhost kernel: [26702.334389] NVRM: ------------[ end crash report ]------------ Nov 15 18:05:30 localhost kernel: [26747.335912] NVRM: _kgspLogXid119: ********************************* GSP Timeout ********************************** Nov 15 18:05:30 localhost kernel: [26747.335915] NVRM: _kgspLogXid119: Note: Please also check logs above. Nov 15 18:05:30 localhost kernel: [26747.335965] NVRM: Xid (PCI:0000:01:00): 119, pid=950, name=Xorg, Timeout after 45s of waiting for RPC response from GPU0 GSP! Expected function 10 (FREE) sequence 6483824 (0x15070 0x0). Nov 15 18:05:30 localhost kernel: [26747.335988] NVRM: GPU0 GSP RPC buffer contains function 10 (FREE) sequence 6483824 and data 0x0000000000015070 0x0000000000000000. Nov 15 18:05:30 localhost kernel: [26747.335990] NVRM: GPU0 RPC history (CPU -> GSP): Nov 15 18:05:30 localhost kernel: [26747.335990] NVRM: entry function sequence data0 data1 ts_start ts_end duration actively_polling Nov 15 18:05:30 localhost kernel: [26747.335992] NVRM: 0 10 FREE 6483824 0x0000000000015070 0x0000000000000000 0x0006439f3e6033f1 0x0000000000000000 y Nov 15 18:05:30 localhost kernel: [26747.335995] NVRM: -1 76 GSP_RM_CONTROL 6483823 0x00000000c3720101 0x0000000000000800 0x0006439f3e602bb7 0x0006439f3e603325 1902us Nov 15 18:05:30 localhost kernel: [26747.335998] NVRM: -2 76 GSP_RM_CONTROL 6483822 0x000000000073010b 0x0000000000000014 0x0006439f3e602af4 0x0006439f3e602ba0 172us Nov 15 18:05:30 localhost kernel: [26747.336000] NVRM: -3 76 GSP_RM_CONTROL 6483821 0x00000000c3700104 0x0000000000000014 0x0006439f3e5feb5d 0x0006439f3e5febb6 89us Nov 15 18:05:30 localhost kernel: [26747.336002] NVRM: -4 76 GSP_RM_CONTROL 6483820 0x00000000c3700104 0x0000000000000014 0x0006439f3e5feafa 0x0006439f3e5feb56 92us Nov 15 18:05:30 localhost kernel: [26747.336003] NVRM: -5 76 GSP_RM_CONTROL 6483819 0x00000000c3700104 0x0000000000000014 0x0006439f3e5fea97 0x0006439f3e5feaf2 91us Nov 15 18:05:30 localhost kernel: [26747.336005] NVRM: -6 76 GSP_RM_CONTROL 6483818 0x00000000c3700104 0x0000000000000014 0x0006439f3e5fea34 0x0006439f3e5fea90 92us Nov 15 18:05:30 localhost kernel: [26747.336007] NVRM: -7 76 GSP_RM_CONTROL 6483817 0x00000000c3700104 0x0000000000000014 0x0006439f3e5fe9d1 0x0006439f3e5fea2d 92us Nov 15 18:05:30 localhost kernel: [26747.336008] NVRM: GPU0 RPC event history (CPU <- GSP): Nov 15 18:05:30 localhost kernel: [26747.336009] NVRM: entry function sequence data0 data1 ts_start ts_end duration during_incomplete_rpc Nov 15 18:05:30 localhost kernel: [26747.336010] NVRM: 0 4098 GSP_RUN_CPU_SEQUENCER 0 0x00000000000001ea 0x0000000000003fe2 0x0006439f3d742cad 0x0006439f3d742fd9 812us Nov 15 18:05:30 localhost kernel: [26747.336013] NVRM: -1 4111 PERF_BRIDGELESS_INFO_ 0 0x0000000000000000 0x0000000000000000 0x0006439f3aa54a6a 0x0006439f3aa54a6a Nov 15 18:05:30 localhost kernel: [26747.336014] NVRM: -2 4098 GSP_RUN_CPU_SEQUENCER 0 0x00000000000001ea 0x0000000000003fe2 0x0006439f39a3cc5b 0x0006439f39a3cf84 809us Nov 15 18:05:30 localhost kernel: [26747.336016] NVRM: -3 4111 PERF_BRIDGELESS_INFO_ 0 0x0000000000000000 0x0000000000000000 0x0006439f36d4c0cf 0x0006439f36d4c0cf Nov 15 18:05:30 localhost kernel: [26747.336018] NVRM: -4 4098 GSP_RUN_CPU_SEQUENCER 0 0x00000000000001ea 0x0000000000003fe2 0x0006439f35d2f753 0x0006439f35d2fa82 815us Nov 15 18:05:30 localhost kernel: [26747.336020] NVRM: -5 4111 PERF_BRIDGELESS_INFO_ 0 0x0000000000000000 0x0000000000000000 0x0006439f32e4c0cf 0x0006439f32e4c0cf Nov 15 18:05:30 localhost kernel: [26747.336021] NVRM: -6 4098 GSP_RUN_CPU_SEQUENCER 0 0x00000000000001ea 0x0000000000003fe2 0x0006439f31e3eeef 0x0006439f31e3f225 822us Nov 15 18:05:30 localhost kernel: [26747.336023] NVRM: -7 4111 PERF_BRIDGELESS_INFO_ 0 0x0000000000000000 0x0000000000000000 0x0006439f2f158643 0x0006439f2f158643 Nov 15 18:05:30 localhost kernel: [26747.336025] CPU: 1 PID: 950 Comm: Xorg Kdump: loaded Tainted: G U W OE 5.19.17+ #1 Nov 15 18:05:30 localhost kernel: [26747.336028] Hardware name: Advantech EBC-GF68/EBC-GF68, BIOS GF68000Q060X019 10/11/2024 Nov 15 18:05:30 localhost kernel: [26747.336029] Call Trace: Nov 15 18:05:30 localhost kernel: [26747.336030] <TASK> Nov 15 18:05:30 localhost kernel: [26747.336032] dump_stack_lvl+0x38/0x4d Nov 15 18:05:30 localhost kernel: [26747.336036] dump_stack+0x10/0x16 Nov 15 18:05:30 localhost kernel: [26747.336039] os_dump_stack+0xe/0x14 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.336209] _kgspRpcRecvPoll+0x650/0x810 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.336404] _issueRpcAndWait+0xdd/0x970 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.336555] ? osGetCurrentThread+0x26/0x60 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.336740] ? rmDeviceGpuLockIsOwner+0x29/0x90 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.336931] ? os_mem_set+0x14/0x20 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.337016] rpcRmApiFree_GSP+0x11b/0x200 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.337163] serverFreeResourceRpcUnderLock+0x32c/0x400 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.337320] ? os_alloc_mem+0xc6/0xe0 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.337405] ? _portMemAllocNonPagedUntracked+0x2c/0x40 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.337538] ? _portMemAllocatorAlloc+0x2b/0xe0 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.337671] ? _tlsEntryAcquire+0x93/0xd0 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.337808] ? multimapFirstItem_IMPL+0x1a/0x80 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.337948] clientFreeResource_IMPL+0x779/0x890 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.338080] ? _rmGpuLocksAcquire.constprop.0+0x36b/0xb50 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.338271] ? serverResLock_Prologue+0x283/0x2f0 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.338427] ? nvDbg_vPrintf+0x60/0x160 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.338619] rmclientFreeResource_IMPL+0x5f/0x6b [nvidia] Nov 15 18:05:30 localhost kernel: [26747.338776] serverFreeResourceTreeUnderLock+0xb8/0x1a0 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.338906] serverFreeResourceTree+0x7ca/0x980 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.339036] ? mark_page_accessed+0x18/0x50 Nov 15 18:05:30 localhost kernel: [26747.339040] ? __find_get_block+0xee/0x360 Nov 15 18:05:30 localhost kernel: [26747.339044] ? os_acquire_spinlock+0x12/0x30 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.339129] ? _raw_spin_unlock_irqrestore+0x22/0x40 Nov 15 18:05:30 localhost kernel: [26747.339132] ? os_release_spinlock+0x1a/0x20 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.339218] ? _tlsThreadEntryGet+0x82/0x90 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.339347] ? tlsEntryGet+0x20/0x70 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.339476] ? down_write+0x13/0x60 Nov 15 18:05:30 localhost kernel: [26747.339478] ? os_acquire_rwlock_write+0x31/0x40 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.339563] rmapiFreeWithSecInfo+0x12b/0x2b0 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.339721] rmapiFreeWithSecInfoTls+0x49/0x70 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.339879] Nv01FreeWithSecInfo+0x4e/0x60 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.340034] RmIoctl+0x449/0xe00 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.340218] ? preempt_count_add+0x82/0xc0 Nov 15 18:05:30 localhost kernel: [26747.340221] ? os_acquire_spinlock+0x12/0x30 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.340307] ? _raw_spin_unlock_irqrestore+0x22/0x40 Nov 15 18:05:30 localhost kernel: [26747.340310] rm_ioctl+0x66/0x4f0 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.340497] nvidia_unlocked_ioctl+0x628/0x940 [nvidia] Nov 15 18:05:30 localhost kernel: [26747.340579] ? __fget_light+0xad/0x140 Nov 15 18:05:30 localhost kernel: [26747.340582] __x64_sys_ioctl+0x95/0xd0 Nov 15 18:05:30 localhost kernel: [26747.340584] do_syscall_64+0x3b/0x90 Nov 15 18:05:30 localhost kernel: [26747.340586] entry_SYSCALL_64_after_hwframe+0x63/0xcd Nov 15 18:05:30 localhost kernel: [26747.340589] RIP: 0033:0x7f0d6111a9bf Nov 15 18:05:30 localhost kernel: [26747.340591] Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <41> 89 c0 3d 00 f0 ff ff 77 1f 48 8b 44 24 18 64 48 2b 04 25 28 00 Nov 15 18:05:30 localhost kernel: [26747.340593] RSP: 002b:00007ffcddb84540 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 Nov 15 18:05:30 localhost kernel: [26747.340595] RAX: ffffffffffffffda RBX: 0000000000000010 RCX: 00007f0d6111a9bf Nov 15 18:05:30 localhost kernel: [26747.340596] RDX: 00007ffcddb84650 RSI: 00000000c0104629 RDI: 000000000000000f Nov 15 18:05:30 localhost kernel: [26747.340597] RBP: 00007ffcddb84650 R08: 00007ffcddb84650 R09: 00007ffcddb8465c Nov 15 18:05:30 localhost kernel: [26747.340598] R10: 0000000000000000 R11: 0000000000000246 R12: 000000000000000f Nov 15 18:05:30 localhost kernel: [26747.340599] R13: 00000000c0104629 R14: 0000000000000029 R15: 00007ffcddb845a0 Nov 15 18:05:30 localhost kernel: [26747.340601] </TASK> Nov 15 18:05:30 localhost kernel: [26747.340609] NVRM: _kgspLogXid119: ******************************************************************************** Nov 15 18:05:30 localhost kernel: [26747.340611] NVRM: _issueRpcAndWait: rpcRecvPoll timedout for fn 10 sequence 6483824! Nov 15 18:05:30 localhost kernel: [26747.340613] NVRM: rpcRmApiFree_GSP: GspRmFree failed: hClient=0xc1d00003; hObject=0x00015070; paramsStatus=0x00000000; status=0x00000065 Nov 15 18:05:30 localhost kernel: [26747.340615] NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_client.c:844 Nov 15 18:05:30 localhost kernel: [26747.340629] NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_server.c:259 Nov 15 18:06:15 localhost kernel: [26792.343955] NVRM: Xid (PCI:0000:01:00): 119, pid=950, name=Xorg, Timeout after 45s of waiting for RPC response from GPU0 GSP! Expected function 10 (FREE) sequence 6483825 (0x15071 0x0). Nov 15 18:06:15 localhost kernel: [26792.343970] NVRM: _issueRpcAndWait: rpcRecvPoll timedout for fn 10 sequence 6483825! Nov 15 18:06:15 localhost kernel: [26792.343971] NVRM: rpcRmApiFree_GSP: GspRmFree failed: hClient=0xc1d00003; hObject=0x00015071; paramsStatus=0x00000000; status=0x00000065 Nov 15 18:06:15 localhost kernel: [26792.343974] NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_client.c:844 Nov 15 18:06:15 localhost kernel: [26792.344034] NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_server.c:259 Nov 15 18:07:00 localhost kernel: [26837.347910] NVRM: Xid (PCI:0000:01:00): 119, Timeout after 45s of waiting for RPC response from GPU0 GSP! Expected function 76 (GSP_RM_CONTROL) sequence 6483826 (0x2080a097 0x490). Nov 15 18:07:00 localhost kernel: [26837.347923] NVRM: nvAssertFailedNoLog: Assertion failed: Back to back GSP RPC timeout detected! GPU marked for reset @ kernel_gsp.c:2366 Nov 15 18:07:00 localhost kernel: [26837.347932] NVRM: _issueRpcAndWait: rpcRecvPoll timedout for fn 76 sequence 6483826! Nov 15 18:07:00 localhost kernel: [26837.347994] NVRM: Xid (PCI:0000:01:00): 154, GPU recovery action changed from 0x0 (None) to 0x1 (GPU Reset Required) Nov 15 18:07:00 localhost kernel: [26837.348259] show_signal_msg: 13 callbacks suppressed Nov 15 18:07:00 localhost kernel: [26837.348261] screencoverserv[26109]: segfault at 18 ip 00007fffef7d6c22 sp 00007fffffffded0 error 4 in libhal.so.1.0[7fffef400000+521000] Nov 15 18:07:00 localhost kernel: [26837.348270] Code: 84 10 d5 ff 8b 40 08 83 f8 06 0f 85 8c 01 00 00 e8 73 10 d5 ff 48 8b 00 48 8b 10 48 81 c2 68 01 00 00 48 8b 12 48 89 c7 ff d2 <8b> 40 18 85 c0 75 4a e8 52 10 d5 ff 48 8b 00 48 8b 10 48 81 c2 68 Nov 15 18:07:00 localhost kernel: [26837.348278] NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_client.c:844 Nov 15 18:07:00 localhost kernel: [26837.348298] NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_server.c:259 Nov 15 18:07:00 localhost kernel: [26837.348332] NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_client.c:844 Nov 15 18:07:00 localhost kernel: [26837.348353] NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_server.c:259 Nov 15 18:07:00 localhost kernel: [26837.348534] NVRM: dispswConstruct_IMPL: RPC error, can't get the displaymask and number of heads Nov 15 18:07:00 localhost kernel: [26837.348915] nvidia-modeset: ERROR: GPU:0: The requested configuration of display devices (BEA CL23-DP (DP-0), DELL U2412M (HDMI-2)) is not supported on this GPU. Nov 15 18:07:00 localhost kernel: [26837.363525] NVRM: dispswConstruct_IMPL: RPC error, can't get the displaymask and number of heads Nov 15 18:07:00 localhost kernel: [26837.363751] nvidia-modeset: ERROR: GPU:0: The requested configuration of display devices (BEA CL23-DP (DP-0), DELL U2412M (HDMI-2), BEA CL15-HDMI (HDMI-1)) is not supported on this GPU. Nov 15 18:07:00 localhost kernel: [26837.380802] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57d:0:0:0x00000062 Nov 15 18:07:00 localhost kernel: [26837.391125] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:0:0:0x00000062 Nov 15 18:07:00 localhost kernel: [26837.401472] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:1:0:0x00000062 Nov 15 18:07:00 localhost kernel: [26837.411764] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:00 localhost kernel: [26837.422095] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:3:0:0x00000062 Nov 15 18:07:00 localhost kernel: [26837.432368] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:00 localhost kernel: [26837.442709] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:5:0:0x00000062 Nov 15 18:07:00 localhost kernel: [26837.453030] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:00 localhost kernel: [26837.463307] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:7:0:0x00000062 Nov 15 18:07:00 localhost kernel: [26837.474771] nvidia-modeset: ERROR: GPU:0: The requested configuration of display devices (DELL U2412M (HDMI-2), BEA CL15-HDMI (HDMI-1)) is not supported on this GPU. Nov 15 18:07:00 localhost kernel: [26837.490045] nvidia-modeset: ERROR: GPU:0: The requested configuration of display devices (BEA CL23-DP (DP-0), BEA CL15-HDMI (HDMI-1)) is not supported on this GPU. Nov 15 18:07:00 localhost kernel: [26837.505009] nvidia-modeset: ERROR: GPU:0: The requested configuration of display devices (BEA CL23-DP (DP-0), DELL U2412M (HDMI-2)) is not supported on this GPU. Nov 15 18:07:00 localhost kernel: [26837.564757] pcie_rcdrv 0000:03:00.0: sono_pcie_ep_fasync Nov 15 18:07:00 localhost kernel: [26837.570798] sonokbd: kbd close! Nov 15 18:07:01 localhost kernel: [26837.948139] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:0:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26837.958473] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26837.968795] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26837.979067] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26837.989444] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26837.999722] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.010046] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.020366] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.030647] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.040969] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.051245] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.061571] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.071902] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.082176] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.092504] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.102824] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.113099] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.123424] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.133709] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.144032] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.154358] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.164639] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.174985] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.185311] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.195587] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.205930] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.216204] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.226527] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.236854] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.247127] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.257457] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.267735] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.278053] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.288377] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.298653] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.308978] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.319299] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.329577] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.339896] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.350241] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.360559] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.370885] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.381156] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.391503] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.401829] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.412105] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.422431] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.432707] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.443032] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.453353] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.463629] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.473954] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.484231] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.494555] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.504876] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.515151] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.525475] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.535798] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.546075] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.556399] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.566678] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.577000] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.587325] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.597596] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.607934] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.618259] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.628735] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.639057] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.649328] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.659656] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.669983] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.680263] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.690604] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:01 localhost kernel: [26838.700881] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.711203] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.721524] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.731801] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.742131] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.752460] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.762740] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.773066] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.783341] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.793671] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.803999] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.814274] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.824597] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.834920] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.845193] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.855529] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.865809] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.876138] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.886462] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.896736] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.907060] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.917337] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.927663] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.937982] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.948257] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.958585] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.968907] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.979180] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.989505] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26838.999783] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.010108] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.020429] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.030704] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.041029] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.051306] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.061632] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.071958] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.082240] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.092567] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.102893] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.113169] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.123495] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.133767] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.144092] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.154418] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.164692] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.175022] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.185346] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.195620] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.205935] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.216212] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.226539] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.236863] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.247140] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.257463] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.267739] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.278062] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.288387] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.298658] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.308987] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.319307] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.329582] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.339909] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.350185] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.360507] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.370832] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.381107] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.391432] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.401751] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.412028] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.422353] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.432632] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.442956] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.453274] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.463551] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.473878] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.484151] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.494484] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.504813] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.515092] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.525424] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.535757] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.546034] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.556371] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.566653] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.576979] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.587313] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.597594] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.607934] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.618284] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.628572] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.638869] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.649161] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.659435] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.669715] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.679993] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.690269] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.700539] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:02 localhost kernel: [26839.710814] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.721091] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.731369] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.741648] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.751929] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.762204] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.772482] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.782759] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.793033] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.803309] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.813589] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.823868] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.834147] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.844430] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.854704] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.864984] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.875265] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.885541] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.895822] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.906100] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.916375] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.926656] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.936935] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.947211] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.957492] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.967768] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.978050] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.988326] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26839.998604] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:2:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26840.008882] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:4:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26840.019164] nvidia-modeset: ERROR: GPU:0: Failed to query display engine channel state: 0x0000c57e:6:0:0x00000062 Nov 15 18:07:03 localhost kernel: [26840.029430] nvidia-modeset: WARNING: GPU:0: Timeout while waiting for idle. Nov 15 18:07:06 localhost kernel: [26843.029533] nvidia-modeset: WARNING: GPU:0: Lost display notification (0:0x00000000); continuing. Nov 15 18:10:01 localhost kernel: [27018.108804] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 18:20:01 localhost kernel: [27618.131815] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 18:30:01 localhost kernel: [28218.140386] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 18:40:01 localhost kernel: [28818.149192] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 18:49:46 localhost kernel: [29402.783899] INFO: task nvidia-modeset/:551 blocked for more than 1835 seconds. Nov 15 18:49:46 localhost kernel: [29402.791140] Tainted: G U W OE 5.19.17+ #1 Nov 15 18:49:46 localhost kernel: [29402.796566] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Nov 15 18:49:46 localhost kernel: [29402.804415] task:nvidia-modeset/ state:D stack: 0 pid: 551 ppid: 2 flags:0x00004000 Nov 15 18:49:46 localhost kernel: [29402.804418] Call Trace: Nov 15 18:49:46 localhost kernel: [29402.804419] <TASK> Nov 15 18:49:46 localhost kernel: [29402.804421] __schedule+0x2f8/0x930 Nov 15 18:49:46 localhost kernel: [29402.804427] schedule+0x6b/0xf0 Nov 15 18:49:46 localhost kernel: [29402.804428] schedule_timeout+0x11c/0x160 Nov 15 18:49:46 localhost kernel: [29402.804430] ? schedule_timeout+0x11c/0x160 Nov 15 18:49:46 localhost kernel: [29402.804433] __down_common+0xfd/0x1e0 Nov 15 18:49:46 localhost kernel: [29402.804434] ? _raw_spin_lock_irq+0x18/0x50 Nov 15 18:49:46 localhost kernel: [29402.804437] __down+0x1d/0x30 Nov 15 18:49:46 localhost kernel: [29402.804439] down+0x51/0x70 Nov 15 18:49:46 localhost kernel: [29402.804441] nvkms_kthread_q_callback+0xa9/0x180 [nvidia_modeset] Nov 15 18:49:46 localhost kernel: [29402.804465] _main_loop+0x8c/0x140 [nvidia_modeset] Nov 15 18:49:46 localhost kernel: [29402.804483] ? nvkms_get_per_open_data+0x60/0x60 [nvidia_modeset] Nov 15 18:49:46 localhost kernel: [29402.804500] kthread+0xfd/0x130 Nov 15 18:49:46 localhost kernel: [29402.804503] ? kthread_complete_and_exit+0x20/0x20 Nov 15 18:49:46 localhost kernel: [29402.804506] ret_from_fork+0x22/0x30 Nov 15 18:49:46 localhost kernel: [29402.804509] </TASK> Nov 15 18:50:01 localhost kernel: [29418.158035] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 19:00:01 localhost kernel: [30018.166589] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 19:10:01 localhost kernel: [30618.175322] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 19:20:01 localhost kernel: [31218.188062] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 19:20:21 localhost kernel: [31237.787856] INFO: task nvidia-modeset/:551 blocked for more than 3670 seconds. Nov 15 19:20:21 localhost kernel: [31237.795105] Tainted: G U W OE 5.19.17+ #1 Nov 15 19:20:21 localhost kernel: [31237.800533] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Nov 15 19:20:21 localhost kernel: [31237.808386] task:nvidia-modeset/ state:D stack: 0 pid: 551 ppid: 2 flags:0x00004000 Nov 15 19:20:21 localhost kernel: [31237.808390] Call Trace: Nov 15 19:20:21 localhost kernel: [31237.808391] <TASK> Nov 15 19:20:21 localhost kernel: [31237.808394] __schedule+0x2f8/0x930 Nov 15 19:20:21 localhost kernel: [31237.808399] schedule+0x6b/0xf0 Nov 15 19:20:21 localhost kernel: [31237.808401] schedule_timeout+0x11c/0x160 Nov 15 19:20:21 localhost kernel: [31237.808403] ? schedule_timeout+0x11c/0x160 Nov 15 19:20:21 localhost kernel: [31237.808405] __down_common+0xfd/0x1e0 Nov 15 19:20:21 localhost kernel: [31237.808407] ? _raw_spin_lock_irq+0x18/0x50 Nov 15 19:20:21 localhost kernel: [31237.808410] __down+0x1d/0x30 Nov 15 19:20:21 localhost kernel: [31237.808411] down+0x51/0x70 Nov 15 19:20:21 localhost kernel: [31237.808414] nvkms_kthread_q_callback+0xa9/0x180 [nvidia_modeset] Nov 15 19:20:21 localhost kernel: [31237.808438] _main_loop+0x8c/0x140 [nvidia_modeset] Nov 15 19:20:21 localhost kernel: [31237.808455] ? nvkms_get_per_open_data+0x60/0x60 [nvidia_modeset] Nov 15 19:20:21 localhost kernel: [31237.808472] kthread+0xfd/0x130 Nov 15 19:20:21 localhost kernel: [31237.808475] ? kthread_complete_and_exit+0x20/0x20 Nov 15 19:20:21 localhost kernel: [31237.808478] ret_from_fork+0x22/0x30 Nov 15 19:20:21 localhost kernel: [31237.808481] </TASK> Nov 15 19:30:01 localhost kernel: [31818.196724] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 19:40:01 localhost kernel: [32418.205506] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 19:50:01 localhost kernel: [33018.214102] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 19:50:56 localhost kernel: [33072.795804] INFO: task nvidia-modeset/:551 blocked for more than 5505 seconds. Nov 15 19:50:56 localhost kernel: [33072.803046] Tainted: G U W OE 5.19.17+ #1 Nov 15 19:50:56 localhost kernel: [33072.808470] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Nov 15 19:50:56 localhost kernel: [33072.816313] task:nvidia-modeset/ state:D stack: 0 pid: 551 ppid: 2 flags:0x00004000 Nov 15 19:50:56 localhost kernel: [33072.816317] Call Trace: Nov 15 19:50:56 localhost kernel: [33072.816318] <TASK> Nov 15 19:50:56 localhost kernel: [33072.816320] __schedule+0x2f8/0x930 Nov 15 19:50:56 localhost kernel: [33072.816326] schedule+0x6b/0xf0 Nov 15 19:50:56 localhost kernel: [33072.816328] schedule_timeout+0x11c/0x160 Nov 15 19:50:56 localhost kernel: [33072.816330] ? schedule_timeout+0x11c/0x160 Nov 15 19:50:56 localhost kernel: [33072.816332] __down_common+0xfd/0x1e0 Nov 15 19:50:56 localhost kernel: [33072.816334] ? _raw_spin_lock_irq+0x18/0x50 Nov 15 19:50:56 localhost kernel: [33072.816337] __down+0x1d/0x30 Nov 15 19:50:56 localhost kernel: [33072.816338] down+0x51/0x70 Nov 15 19:50:56 localhost kernel: [33072.816341] nvkms_kthread_q_callback+0xa9/0x180 [nvidia_modeset] Nov 15 19:50:56 localhost kernel: [33072.816365] _main_loop+0x8c/0x140 [nvidia_modeset] Nov 15 19:50:56 localhost kernel: [33072.816383] ? nvkms_get_per_open_data+0x60/0x60 [nvidia_modeset] Nov 15 19:50:56 localhost kernel: [33072.816400] kthread+0xfd/0x130 Nov 15 19:50:56 localhost kernel: [33072.816403] ? kthread_complete_and_exit+0x20/0x20 Nov 15 19:50:56 localhost kernel: [33072.816406] ret_from_fork+0x22/0x30 Nov 15 19:50:56 localhost kernel: [33072.816409] </TASK> Nov 15 20:00:01 localhost kernel: [33618.222883] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 20:10:01 localhost kernel: [34218.231530] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 20:20:01 localhost kernel: [34818.243894] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 20:21:31 localhost kernel: [34907.803726] INFO: task nvidia-modeset/:551 blocked for more than 7340 seconds. Nov 15 20:21:31 localhost kernel: [34907.810969] Tainted: G U W OE 5.19.17+ #1 Nov 15 20:21:31 localhost kernel: [34907.816397] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Nov 15 20:21:31 localhost kernel: [34907.824253] task:nvidia-modeset/ state:D stack: 0 pid: 551 ppid: 2 flags:0x00004000 Nov 15 20:21:31 localhost kernel: [34907.824257] Call Trace: Nov 15 20:21:31 localhost kernel: [34907.824258] <TASK> Nov 15 20:21:31 localhost kernel: [34907.824260] __schedule+0x2f8/0x930 Nov 15 20:21:31 localhost kernel: [34907.824266] schedule+0x6b/0xf0 Nov 15 20:21:31 localhost kernel: [34907.824268] schedule_timeout+0x11c/0x160 Nov 15 20:21:31 localhost kernel: [34907.824270] ? schedule_timeout+0x11c/0x160 Nov 15 20:21:31 localhost kernel: [34907.824272] __down_common+0xfd/0x1e0 Nov 15 20:21:31 localhost kernel: [34907.824273] ? _raw_spin_lock_irq+0x18/0x50 Nov 15 20:21:31 localhost kernel: [34907.824276] __down+0x1d/0x30 Nov 15 20:21:31 localhost kernel: [34907.824278] down+0x51/0x70 Nov 15 20:21:31 localhost kernel: [34907.824280] nvkms_kthread_q_callback+0xa9/0x180 [nvidia_modeset] Nov 15 20:21:31 localhost kernel: [34907.824304] _main_loop+0x8c/0x140 [nvidia_modeset] Nov 15 20:21:31 localhost kernel: [34907.824321] ? nvkms_get_per_open_data+0x60/0x60 [nvidia_modeset] Nov 15 20:21:31 localhost kernel: [34907.824338] kthread+0xfd/0x130 Nov 15 20:21:31 localhost kernel: [34907.824342] ? kthread_complete_and_exit+0x20/0x20 Nov 15 20:21:31 localhost kernel: [34907.824344] ret_from_fork+0x22/0x30 Nov 15 20:21:31 localhost kernel: [34907.824348] </TASK> Nov 15 20:30:01 localhost kernel: [35418.252724] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 20:40:01 localhost kernel: [36018.261566] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 20:50:01 localhost kernel: [36618.270189] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 20:52:06 localhost kernel: [36742.811730] INFO: task nvidia-modeset/:551 blocked for more than 9175 seconds. Nov 15 20:52:06 localhost kernel: [36742.818968] Tainted: G U W OE 5.19.17+ #1 Nov 15 20:52:06 localhost kernel: [36742.824384] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Nov 15 20:52:06 localhost kernel: [36742.832228] task:nvidia-modeset/ state:D stack: 0 pid: 551 ppid: 2 flags:0x00004000 Nov 15 20:52:06 localhost kernel: [36742.832231] Call Trace: Nov 15 20:52:06 localhost kernel: [36742.832233] <TASK> Nov 15 20:52:06 localhost kernel: [36742.832235] __schedule+0x2f8/0x930 Nov 15 20:52:06 localhost kernel: [36742.832240] schedule+0x6b/0xf0 Nov 15 20:52:06 localhost kernel: [36742.832242] schedule_timeout+0x11c/0x160 Nov 15 20:52:06 localhost kernel: [36742.832244] ? schedule_timeout+0x11c/0x160 Nov 15 20:52:06 localhost kernel: [36742.832246] __down_common+0xfd/0x1e0 Nov 15 20:52:06 localhost kernel: [36742.832248] ? _raw_spin_lock_irq+0x18/0x50 Nov 15 20:52:06 localhost kernel: [36742.832251] __down+0x1d/0x30 Nov 15 20:52:06 localhost kernel: [36742.832252] down+0x51/0x70 Nov 15 20:52:06 localhost kernel: [36742.832255] nvkms_kthread_q_callback+0xa9/0x180 [nvidia_modeset] Nov 15 20:52:06 localhost kernel: [36742.832278] _main_loop+0x8c/0x140 [nvidia_modeset] Nov 15 20:52:06 localhost kernel: [36742.832296] ? nvkms_get_per_open_data+0x60/0x60 [nvidia_modeset] Nov 15 20:52:06 localhost kernel: [36742.832313] kthread+0xfd/0x130 Nov 15 20:52:06 localhost kernel: [36742.832316] ? kthread_complete_and_exit+0x20/0x20 Nov 15 20:52:06 localhost kernel: [36742.832319] ret_from_fork+0x22/0x30 Nov 15 20:52:06 localhost kernel: [36742.832322] </TASK> Nov 15 21:00:01 localhost kernel: [37218.278861] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 21:10:01 localhost kernel: [37818.287213] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 21:20:01 localhost kernel: [38418.299852] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 21:22:41 localhost kernel: [38577.819684] INFO: task nvidia-modeset/:551 blocked for more than 11010 seconds. Nov 15 21:22:41 localhost kernel: [38577.827011] Tainted: G U W OE 5.19.17+ #1 Nov 15 21:22:41 localhost kernel: [38577.832436] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Nov 15 21:22:41 localhost kernel: [38577.840285] task:nvidia-modeset/ state:D stack: 0 pid: 551 ppid: 2 flags:0x00004000 Nov 15 21:22:41 localhost kernel: [38577.840290] Call Trace: Nov 15 21:22:41 localhost kernel: [38577.840291] <TASK> Nov 15 21:22:41 localhost kernel: [38577.840293] __schedule+0x2f8/0x930 Nov 15 21:22:41 localhost kernel: [38577.840301] schedule+0x6b/0xf0 Nov 15 21:22:41 localhost kernel: [38577.840303] schedule_timeout+0x11c/0x160 Nov 15 21:22:41 localhost kernel: [38577.840307] ? schedule_timeout+0x11c/0x160 Nov 15 21:22:41 localhost kernel: [38577.840309] __down_common+0xfd/0x1e0 Nov 15 21:22:41 localhost kernel: [38577.840311] ? _raw_spin_lock_irq+0x18/0x50 Nov 15 21:22:41 localhost kernel: [38577.840314] __down+0x1d/0x30 Nov 15 21:22:41 localhost kernel: [38577.840316] down+0x51/0x70 Nov 15 21:22:41 localhost kernel: [38577.840318] nvkms_kthread_q_callback+0xa9/0x180 [nvidia_modeset] Nov 15 21:22:41 localhost kernel: [38577.840342] _main_loop+0x8c/0x140 [nvidia_modeset] Nov 15 21:22:41 localhost kernel: [38577.840361] ? nvkms_get_per_open_data+0x60/0x60 [nvidia_modeset] Nov 15 21:22:41 localhost kernel: [38577.840378] kthread+0xfd/0x130 Nov 15 21:22:41 localhost kernel: [38577.840381] ? kthread_complete_and_exit+0x20/0x20 Nov 15 21:22:41 localhost kernel: [38577.840383] ret_from_fork+0x22/0x30 Nov 15 21:22:41 localhost kernel: [38577.840387] </TASK> Nov 15 21:30:01 localhost kernel: [39018.308659] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 21:35:23 localhost kernel: [39340.352268] perf: interrupt took too long (2503 > 2500), lowering kernel.perf_event_max_sample_rate to 79750 Nov 15 21:40:01 localhost kernel: [39618.317581] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 21:50:01 localhost kernel: [40218.326292] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 21:53:16 localhost kernel: [40412.827630] INFO: task nvidia-modeset/:551 blocked for more than 12845 seconds. Nov 15 21:53:16 localhost kernel: [40412.834963] Tainted: G U W OE 5.19.17+ #1 Nov 15 21:53:16 localhost kernel: [40412.840392] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Nov 15 21:53:16 localhost kernel: [40412.848245] task:nvidia-modeset/ state:D stack: 0 pid: 551 ppid: 2 flags:0x00004000 Nov 15 21:53:16 localhost kernel: [40412.848250] Call Trace: Nov 15 21:53:16 localhost kernel: [40412.848251] <TASK> Nov 15 21:53:16 localhost kernel: [40412.848253] __schedule+0x2f8/0x930 Nov 15 21:53:16 localhost kernel: [40412.848259] schedule+0x6b/0xf0 Nov 15 21:53:16 localhost kernel: [40412.848261] schedule_timeout+0x11c/0x160 Nov 15 21:53:16 localhost kernel: [40412.848263] ? schedule_timeout+0x11c/0x160 Nov 15 21:53:16 localhost kernel: [40412.848265] __down_common+0xfd/0x1e0 Nov 15 21:53:16 localhost kernel: [40412.848267] ? _raw_spin_lock_irq+0x18/0x50 Nov 15 21:53:16 localhost kernel: [40412.848270] __down+0x1d/0x30 Nov 15 21:53:16 localhost kernel: [40412.848271] down+0x51/0x70 Nov 15 21:53:16 localhost kernel: [40412.848273] nvkms_kthread_q_callback+0xa9/0x180 [nvidia_modeset] Nov 15 21:53:16 localhost kernel: [40412.848297] _main_loop+0x8c/0x140 [nvidia_modeset] Nov 15 21:53:16 localhost kernel: [40412.848315] ? nvkms_get_per_open_data+0x60/0x60 [nvidia_modeset] Nov 15 21:53:16 localhost kernel: [40412.848335] kthread+0xfd/0x130 Nov 15 21:53:16 localhost kernel: [40412.848338] ? kthread_complete_and_exit+0x20/0x20 Nov 15 21:53:16 localhost kernel: [40412.848341] ret_from_fork+0x22/0x30 Nov 15 21:53:16 localhost kernel: [40412.848344] </TASK> Nov 15 22:00:01 localhost kernel: [40818.335014] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 22:10:01 localhost kernel: [41418.343723] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 22:20:01 localhost kernel: [42018.356361] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 22:23:51 localhost kernel: [42247.835554] INFO: task nvidia-modeset/:551 blocked for more than 14680 seconds. Nov 15 22:23:51 localhost kernel: [42247.842880] Tainted: G U W OE 5.19.17+ #1 Nov 15 22:23:51 localhost kernel: [42247.848298] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Nov 15 22:23:51 localhost kernel: [42247.856143] task:nvidia-modeset/ state:D stack: 0 pid: 551 ppid: 2 flags:0x00004000 Nov 15 22:23:51 localhost kernel: [42247.856147] Call Trace: Nov 15 22:23:51 localhost kernel: [42247.856149] <TASK> Nov 15 22:23:51 localhost kernel: [42247.856151] __schedule+0x2f8/0x930 Nov 15 22:23:51 localhost kernel: [42247.856158] schedule+0x6b/0xf0 Nov 15 22:23:51 localhost kernel: [42247.856160] schedule_timeout+0x11c/0x160 Nov 15 22:23:51 localhost kernel: [42247.856162] ? schedule_timeout+0x11c/0x160 Nov 15 22:23:51 localhost kernel: [42247.856164] __down_common+0xfd/0x1e0 Nov 15 22:23:51 localhost kernel: [42247.856166] ? _raw_spin_lock_irq+0x18/0x50 Nov 15 22:23:51 localhost kernel: [42247.856170] __down+0x1d/0x30 Nov 15 22:23:51 localhost kernel: [42247.856171] down+0x51/0x70 Nov 15 22:23:51 localhost kernel: [42247.856174] nvkms_kthread_q_callback+0xa9/0x180 [nvidia_modeset] Nov 15 22:23:51 localhost kernel: [42247.856206] _main_loop+0x8c/0x140 [nvidia_modeset] Nov 15 22:23:51 localhost kernel: [42247.856234] ? nvkms_get_per_open_data+0x60/0x60 [nvidia_modeset] Nov 15 22:23:51 localhost kernel: [42247.856260] kthread+0xfd/0x130 Nov 15 22:23:51 localhost kernel: [42247.856264] ? kthread_complete_and_exit+0x20/0x20 Nov 15 22:23:51 localhost kernel: [42247.856267] ret_from_fork+0x22/0x30 Nov 15 22:23:51 localhost kernel: [42247.856271] </TASK> Nov 15 22:30:01 localhost kernel: [42618.364997] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 22:40:01 localhost kernel: [43218.373508] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 22:50:01 localhost kernel: [43818.382199] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 22:54:26 localhost kernel: [44082.843553] INFO: task nvidia-modeset/:551 blocked for more than 16515 seconds. Nov 15 22:54:26 localhost kernel: [44082.850883] Tainted: G U W OE 5.19.17+ #1 Nov 15 22:54:26 localhost kernel: [44082.856316] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Nov 15 22:54:26 localhost kernel: [44082.864156] task:nvidia-modeset/ state:D stack: 0 pid: 551 ppid: 2 flags:0x00004000 Nov 15 22:54:26 localhost kernel: [44082.864160] Call Trace: Nov 15 22:54:26 localhost kernel: [44082.864161] <TASK> Nov 15 22:54:26 localhost kernel: [44082.864163] __schedule+0x2f8/0x930 Nov 15 22:54:26 localhost kernel: [44082.864169] schedule+0x6b/0xf0 Nov 15 22:54:26 localhost kernel: [44082.864171] schedule_timeout+0x11c/0x160 Nov 15 22:54:26 localhost kernel: [44082.864173] ? schedule_timeout+0x11c/0x160 Nov 15 22:54:26 localhost kernel: [44082.864175] __down_common+0xfd/0x1e0 Nov 15 22:54:26 localhost kernel: [44082.864177] ? _raw_spin_lock_irq+0x18/0x50 Nov 15 22:54:26 localhost kernel: [44082.864180] __down+0x1d/0x30 Nov 15 22:54:26 localhost kernel: [44082.864181] down+0x51/0x70 Nov 15 22:54:26 localhost kernel: [44082.864184] nvkms_kthread_q_callback+0xa9/0x180 [nvidia_modeset] Nov 15 22:54:26 localhost kernel: [44082.864209] _main_loop+0x8c/0x140 [nvidia_modeset] Nov 15 22:54:26 localhost kernel: [44082.864227] ? nvkms_get_per_open_data+0x60/0x60 [nvidia_modeset] Nov 15 22:54:26 localhost kernel: [44082.864244] kthread+0xfd/0x130 Nov 15 22:54:26 localhost kernel: [44082.864247] ? kthread_complete_and_exit+0x20/0x20 Nov 15 22:54:26 localhost kernel: [44082.864250] ret_from_fork+0x22/0x30 Nov 15 22:54:26 localhost kernel: [44082.864253] </TASK> Nov 15 23:00:01 localhost kernel: [44418.390781] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 23:10:01 localhost kernel: [45018.399356] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 23:20:01 localhost kernel: [45618.411830] pcie_rcdrv 0000:03:00.0: get_pcie_log device_open(pcie_rcdrv_kmsg) Nov 15 23:25:01 localhost kernel: [45917.851463] INFO: task nvidia-modeset/:551 blocked for more than 18350 seconds. Nov 15 23:25:01 localhost kernel: [45917.858797] Tainted: G U W OE 5.19.17+ #1 Nov 15 23:25:01 localhost kernel: [45917.864230] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Nov 15 23:25:01 localhost kernel: [45917.872088] task:nvidia-modeset/ state:D stack: 0 pid: 551 ppid: 2 flags:0x00004000 Nov 15 23:25:01 localhost kernel: [45917.872093] Call Trace: Nov 15 23:25:01 localhost kernel: [45917.872094] <TASK> Nov 15 23:25:01 localhost kernel: [45917.872096] __schedule+0x2f8/0x930 Nov 15 23:25:01 localhost kernel: [45917.872102] schedule+0x6b/0xf0 Nov 15 23:25:01 localhost kernel: [45917.872104] schedule_timeout+0x11c/0x160 Nov 15 23:25:01 localhost kernel: [45917.872106] ? schedule_timeout+0x11c/0x160 Nov 15 23:25:01 localhost kernel: [45917.872108] __down_common+0xfd/0x1e0 Nov 15 23:25:01 localhost kernel: [45917.872110] ? _raw_spin_lock_irq+0x18/0x50 Nov 15 23:25:01 localhost kernel: [45917.872113] __down+0x1d/0x30 Nov 15 23:25:01 localhost kernel: [45917.872114] down+0x51/0x70 Nov 15 23:25:01 localhost kernel: [45917.872116] nvkms_kthread_q_callback+0xa9/0x180 [nvidia_modeset] Nov 15 23:25:01 localhost kernel: [45917.872140] _main_loop+0x8c/0x140 [nvidia_modeset] Nov 15 23:25:01 localhost kernel: [45917.872158] ? nvkms_get_per_open_data+0x60/0x60 [nvidia_modeset] Nov 15 23:25:01 localhost kernel: [45917.872175] kthread+0xfd/0x130 Nov 15 23:25:01 localhost kernel: [45917.872178] ? kthread_complete_and_exit+0x20/0x20 Nov 15 23:25:01 localhost kernel: [45917.872181] ret_from_fork+0x22/0x30 Nov 15 23:25:01 localhost kernel: [45917.872185] </TASK>