最近的修改导致解析DNS耗尽CPU #1196

darren · 2021-08-11T17:41:44Z

darren
Aug 11, 2021

Tasks: 179 total,   1 running, 178 sleeping,   0 stopped,   0 zombie
%Cpu0  :  92.1/7.9   100[                                                                                               ]     %Cpu1  :  96.1/2.9    99[                                                                                               ]
%Cpu2  :  97.1/2.0    99[                                                                                               ]     %Cpu3  :  96.0/4.0   100[                                                                                               ]
MiB Mem : 14.4/15931.7  [||||||||||||||                                                                                 ]
MiB Swap:  0.0/0.0      [                                                                                               ]

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                                                                           
 328020 root      20   0 5800.2m  74.9m  17.0m S 364.4   0.5 675:58.01 /usr/bin/v2ray -config /etc/v2/default.json

Samples: 284K of event 'cycles', 4000 Hz, Event count (approx.): 27101695767 lost: 0/0 drop: 0/0                                                                                                                                                         
Overhead  Shared Object       Symbol                                                                                                                                                                                                                     
  17.27%  v2                  [.] runtime.selectgo                                                                                                                                                                                                       
  15.02%  v2                  [.] github.com/v2fly/v2ray-core/v4/app/dns.(*DoHNameServer).findIPsForDomain                                                                                                                                               
   7.22%  v2                  [.] runtime.unlock2                                                                                                                                                                                                        
   6.92%  v2                  [.] context.(*cancelCtx).Done                                                                                                                                                                                              
   6.81%  v2                  [.] runtime.lock2                                                                                                                                                                                                          
   6.81%  [vdso]              [.] 0x0000000000000695                                                                                                                                                                                                     
   4.63%  v2                  [.] runtime.mapaccess2_faststr                                                                                                                                                                                             
   4.34%  [vdso]              [.] __vdso_clock_gettime                                                                                                                                                                                                   
   3.11%  v2                  [.] runtime.sellock                                                                                                                                                                                                        
   3.00%  v2                  [.] runtime.selunlock                                                                                                                                                                                                      
   2.70%  v2                  [.] runtime.walltime1                                                                                                                                                                                                      
   2.70%  v2                  [.] github.com/v2fly/v2ray-core/v4/app/dns.(*DoHNameServer).QueryIP                                                                                                                                                        
   2.53%  v2                  [.] runtime.nanotime1                                                                                                                                                                                                      
   2.45%  v2                  [.] aeshashbody                                                                                                                                                                                                            
   2.36%  v2                  [.] github.com/v2fly/v2ray-core/v4/app/dns.(*IPRecord).getIPs                                                                                                                                                              
   1.52%  v2                  [.] time.now                                                                                                                                                                                                               
   1.42%  v2                  [.] memeqbody                                                                                                                                                                                                              
   1.21%  v2                  [.] time.Now                                                                                                                                                                                                               
   0.75%  v2                  [.] runtime.typedslicecopy                                                                                                                                                                                                 
   0.70%  v2                  [.] context.(*timerCtx).Done                                                                                                                                                                                               
   0.61%  v2                  [.] time.Time.Before                                                                                                                                                                                                       
   0.32%  v2                  [.] runtime.strhash                                                                                                                                                                                                        
   0.27%  [vdso]              [.] 0x0000000000000698

版本： 639148d

Loyalsoldier · 2021-08-11T20:08:39Z

Loyalsoldier
Aug 11, 2021
Maintainer

分别试一试这三个版本：
a783cd5
73470e8
ecaf597

0 replies

darren · 2021-08-28T20:59:02Z

darren
Aug 28, 2021
Author

测试结果：

a783cd5 bad
73470e8 bad
ecaf597 good

50bcb68 版本的profile结果

发现出现CPU被全部占用的时候，可能是遇到了不存在AAAA记录的域名，反复执行以下代码

v2ray-core/app/dns/nameserver_doh.go

Lines 373 to 384 in 50bcb68

    
           for { 
        
           	ips, err := s.findIPsForDomain(fqdn, option) 
        
           	if err != errRecordNotFound { 
        
           		return ips, err 
        
           	} 
        
           	select { 
        
           	case <-ctx.Done(): 
        
           		return nil, ctx.Err() 
        
           	case <-done: 
        
           	} 
        
           }

记录日志跟踪了一下:

        i := 0
	for {
                i++
		ips, err := s.findIPsForDomain(fqdn, option)
	        log.Printf("RESOLVE: %s %d", fqdn, i)
	        if err != errRecordNotFound {
			return ips, err
		}

		select {
		case <-ctx.Done():
			return nil, ctx.Err()
		case <-done:
		}
	}

记录到以下日志:


 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19698
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19699
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 17996
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19700
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 17997
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19701
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 17998
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19702
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 17999
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 18000
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 18001
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19703
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19704
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 18002
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19705
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 18003
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 18004
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19706
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 18005
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19707
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19708
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 18006
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19709
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 18007
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19710
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19711
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 18008
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 18009
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19712
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19713
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19714
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19715
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69275
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19716
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19717
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69276
 2021/08/29 04:54:20 RESOLVE: vortex.data.microsoft.com. 19718
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69277
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69278
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69279
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69280
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69281
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69282
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69283
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69284
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69285
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69286
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69287
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69288
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69289
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69290
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69291
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69292
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69293
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69294
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69295
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69296
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69297
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69298
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69299
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69300
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69301
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 4259
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69302
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 4260
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69303
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 4261
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69304
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 4262
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69305
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69306
 2021/08/29 04:54:20 RESOLVE: alive.github.com. 69307

解析 alive.github.com 在一秒内这段代码被执行了 2,026,539次

0 replies

Loyalsoldier · 2021-08-29T08:26:30Z

Loyalsoldier
Aug 29, 2021
Maintainer

I think 46fa8f7 has fixed the problem.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

最近的修改导致解析DNS耗尽CPU #1196

Uh oh!

{{title}}

Uh oh!

Replies: 3 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

最近的修改导致解析DNS耗尽CPU #1196

Uh oh!

darren Aug 11, 2021

Replies: 3 comments

Uh oh!

Loyalsoldier Aug 11, 2021 Maintainer

Uh oh!

Uh oh!

darren Aug 28, 2021 Author

Uh oh!

Loyalsoldier Aug 29, 2021 Maintainer

darren
Aug 11, 2021

Loyalsoldier
Aug 11, 2021
Maintainer

darren
Aug 28, 2021
Author

Loyalsoldier
Aug 29, 2021
Maintainer