03.srt

1
00:00:02,300 --> 00:00:04,300
字幕生成：慎独     字幕校对：lim

2
00:00:04,775 --> 00:00:08,023
大家好,我是经常拖更的ZOMI

3
00:00:08,023 --> 00:00:10,280
最近更新的频率确实低了好多

4
00:00:10,280 --> 00:00:14,760
现在来到了AI芯片里面的AI计算体系

5
00:00:14,760 --> 00:00:17,640
还是在深度学习计算模式

6
00:00:17,640 --> 00:00:21,960
去了解深度学习的计算模式有哪些不一样的点

7
00:00:21,960 --> 00:00:24,120
在这里面上一节课还记得吗

8
00:00:24,120 --> 00:00:26,320
讲了三个主要的内容

9
00:00:26,320 --> 00:00:30,200
第一个就是AI的发展和AI的范式

10
00:00:30,200 --> 00:00:33,120
第二个就是一些经典的网络模型

11
00:00:33,120 --> 00:00:34,080
ResNet那些

12
00:00:34,080 --> 00:00:36,640
第三个就是网络模型和剪枝

13
00:00:36,640 --> 00:00:40,400
这三个技术对深度学习计算的模式

14
00:00:40,400 --> 00:00:41,520
带来哪些冲击

15
00:00:41,520 --> 00:00:43,280
带来哪些不一样的点

16
00:00:43,280 --> 00:00:45,200
接下来在这一节里面

17
00:00:45,200 --> 00:00:47,040
将会给大家去汇报和分享

18
00:00:47,040 --> 00:00:49,120
轻量化的网络模型

19
00:00:49,120 --> 00:00:50,640
例如之前谈到的

20
00:00:50,640 --> 00:00:53,280
MobileNet, EfficientNet这种轻量化的

21
00:00:53,280 --> 00:00:54,680
看看大模型

22
00:00:54,680 --> 00:00:56,680
特别是分布式并行里面

23
00:00:56,680 --> 00:00:59,200
对整个深度学习的计算体系

24
00:00:59,240 --> 00:01:03,120
又有哪些带来新的冲击和新的idea

25
00:01:04,240 --> 00:01:06,880
第一个内容要给大家汇报和分享的

26
00:01:06,880 --> 00:01:09,920
就是轻量化的网络模型的设计

27
00:01:09,920 --> 00:01:12,280
其实在之前

28
00:01:12,280 --> 00:01:13,852
推理引擎系列里面的

29
00:01:13,852 --> 00:01:16,440
模型小型化和轻量模型里面

30
00:01:16,440 --> 00:01:19,160
就讲了两个主要的系列

31
00:01:19,160 --> 00:01:22,480
和跟大家一起去分享了很多相关的内容

32
00:01:22,480 --> 00:01:25,640
第一个就是左边的CNN的系列

33
00:01:25,640 --> 00:01:28,120
第二个就是Transformer的系列

34
00:01:28,160 --> 00:01:30,160
从现在可以看得出来

35
00:01:30,160 --> 00:01:31,560
从一开始的SqueezeNet

36
00:01:31,560 --> 00:01:33,720
到后来的GhostNet, EfficientNet

37
00:01:33,720 --> 00:01:36,480
基本上是从2016年到2019年

38
00:01:36,480 --> 00:01:38,520
那这个时间段随着非常多的

39
00:01:38,520 --> 00:01:41,040
CNN的轻量化网络模型的提出

40
00:01:41,040 --> 00:01:43,920
CNN也越来越多的运用在端侧

41
00:01:43,920 --> 00:01:46,840
而Transformer确实在后面才起来的

42
00:01:46,840 --> 00:01:49,440
能够做好非常多的attention的机制

43
00:01:49,440 --> 00:01:51,560
然后让网络模型越来越大

44
00:01:51,560 --> 00:01:53,520
模型的精度也越来越好

45
00:01:53,520 --> 00:01:55,160
但是怎么轻量化的部署

46
00:01:55,440 --> 00:01:57,280
于是从2021年开始

47
00:01:57,560 --> 00:01:59,840
到去年年底到今年年初

48
00:02:00,120 --> 00:02:02,640
现在有非常多的去把MobileNet

49
00:02:02,640 --> 00:02:06,120
还有VisionTransform这个结构相结合

50
00:02:06,120 --> 00:02:08,000
然后又衍生了一系列的

51
00:02:08,000 --> 00:02:10,520
Transformer的轻量级的模型

52
00:02:10,825 --> 00:02:13,760
在这里面我给大家回顾一下

53
00:02:13,760 --> 00:02:16,240
其实在之前做了很多的分享

54
00:02:16,240 --> 00:02:19,160
就是在推理系统里面的模型小型化

55
00:02:19,160 --> 00:02:22,120
感兴趣的朋友们也可以去了解一下

56
00:02:22,120 --> 00:02:23,800
之前分享的课程

57
00:02:23,800 --> 00:02:26,320
里面确实讲了很多相关的论文

58
00:02:26,320 --> 00:02:29,880
而这里面简单再串一串相关的知识

59
00:02:29,880 --> 00:02:32,920
最重要的是引起对于深度学习

60
00:02:33,120 --> 00:02:35,080
计算模式的思考

61
00:02:35,706 --> 00:02:38,135
轻量化网络模型的主要的设计方式

62
00:02:38,255 --> 00:02:39,735
其实有两个

63
00:02:39,895 --> 00:02:42,655
第一个就是改变网络模型的不同的层数

64
00:02:42,655 --> 00:02:44,135
包括layer的shape

65
00:02:44,135 --> 00:02:45,775
包括channel的计算方式

66
00:02:46,015 --> 00:02:47,695
还有卷积的计算方式

67
00:02:47,695 --> 00:02:50,775
从而衍生出一种新的网络模型结构

68
00:02:50,775 --> 00:02:51,975
这是第一种

69
00:02:52,135 --> 00:02:54,935
第二种就是通过NAS来搜索

70
00:02:54,935 --> 00:02:57,135
一些更轻量化的网络模型

71
00:02:57,135 --> 00:02:58,455
而通过NAS来搜索

72
00:02:58,455 --> 00:03:01,815
很多时候会根据硬件来做定制化的

73
00:03:01,815 --> 00:03:04,855
下面两个图就是后面会展开来讲

74
00:03:05,095 --> 00:03:07,335
第一个就是把传统的一些卷积

75
00:03:07,655 --> 00:03:10,375
变成两个不同的卷积去做

76
00:03:10,575 --> 00:03:13,535
这种就是MobileNet里面的DepthMask卷积

77
00:03:13,775 --> 00:03:16,575
右边这种就是去减少卷积里面的

78
00:03:16,575 --> 00:03:18,215
那个channel的层数

79
00:03:18,215 --> 00:03:20,375
然后让网络模型更浅

80
00:03:20,655 --> 00:03:22,415
下面看几个具体的例子

81
00:03:22,615 --> 00:03:24,135
第一个例子比较明显的

82
00:03:24,135 --> 00:03:26,535
就减少整个空间的大小

83
00:03:26,535 --> 00:03:27,935
Spatial Sized

84
00:03:28,255 --> 00:03:29,975
里面就像GoogleNet

85
00:03:29,975 --> 00:03:32,015
还有ExceptionNet V3

86
00:03:32,015 --> 00:03:34,815
就会把5×5的一种卷积

87
00:03:34,815 --> 00:03:37,455
decompose成为一种5×1的卷积

88
00:03:37,455 --> 00:03:39,255
加一种1×5的卷积

89
00:03:39,495 --> 00:03:41,215
最后从最右边的图

90
00:03:41,295 --> 00:03:42,175
可以看到

91
00:03:42,175 --> 00:03:45,175
基本上可以减少大量的计算的空间

92
00:03:45,455 --> 00:03:47,895
像VGG其实一开始VGG之前

93
00:03:48,255 --> 00:03:49,975
有很多5×5的卷积

94
00:03:50,095 --> 00:03:52,335
VGG把它给decompose成为

95
00:03:52,335 --> 00:03:53,535
3×3的卷积

96
00:03:53,535 --> 00:03:54,895
加3×3的卷积

97
00:03:54,895 --> 00:03:56,895
后面再加一个残差层

98
00:03:57,135 --> 00:03:58,415
从这里面可以看出

99
00:03:58,415 --> 00:04:01,255
也是减少了大量的内存空间

100
00:04:01,895 --> 00:04:02,975
再往下看

101
00:04:02,975 --> 00:04:05,135
第二种可能也可以做

102
00:04:05,135 --> 00:04:07,575
减少一些channels的计算

103
00:04:07,775 --> 00:04:09,695
这里面channels非常多

104
00:04:09,695 --> 00:04:12,175
像MobileNet就改成了

105
00:04:12,175 --> 00:04:13,455
Pointwise的卷积

106
00:04:13,455 --> 00:04:14,175
1×1

107
00:04:14,175 --> 00:04:16,975
还有一些直接使用1×1的卷积

108
00:04:16,975 --> 00:04:19,265
去代替掉原来的卷积层

109
00:04:20,175 --> 00:04:23,175
到了后面还有减少filter的数量

110
00:04:23,175 --> 00:04:24,335
像DenseNet这种

111
00:04:24,335 --> 00:04:26,935
就是重复的去使用feature map

112
00:04:26,935 --> 00:04:29,335
而GhostNet华为诺亚去发明的

113
00:04:29,335 --> 00:04:32,375
就是直接基于得到的feature map

114
00:04:32,375 --> 00:04:35,255
再去提取新的feature map

115
00:04:35,255 --> 00:04:35,935
很有意思

116
00:04:35,935 --> 00:04:37,415
一开始就是直接减少到

117
00:04:37,415 --> 00:04:39,135
feature的数量

118
00:04:40,035 --> 00:04:41,735
现在回过头来思考

119
00:04:41,735 --> 00:04:42,775
轻量化网络模型

120
00:04:42,895 --> 00:04:44,855
对AI的计算模式的

121
00:04:44,855 --> 00:04:47,255
一种不一样的启示

122
00:04:47,495 --> 00:04:49,255
第一个我分开三个方面

123
00:04:49,415 --> 00:04:50,015
这三个方面

124
00:04:50,015 --> 00:04:52,055
其实我之前在推理引擎里面

125
00:04:52,055 --> 00:04:52,895
已经总结过了

126
00:04:52,895 --> 00:04:54,735
简单的把它串起来看一看

127
00:04:54,735 --> 00:04:56,775
首先是卷积核方面

128
00:04:57,015 --> 00:04:58,535
会使用大的卷积核

129
00:04:58,735 --> 00:05:01,615
把变成很多小的卷积核来代替

130
00:05:01,615 --> 00:05:03,295
然后单一尺寸的卷积核

131
00:05:03,415 --> 00:05:06,415
会用很多不同的卷积核来代替

132
00:05:06,415 --> 00:05:08,175
就是一个变成多个

133
00:05:08,175 --> 00:05:09,335
还有一些固定的

134
00:05:09,335 --> 00:05:11,735
就使用大量的使用1×1的卷积

135
00:05:11,735 --> 00:05:13,415
作为bottleneck

136
00:05:13,855 --> 00:05:17,215
第二种就是卷积层通道方面

137
00:05:17,215 --> 00:05:19,215
根据channel的数

138
00:05:19,255 --> 00:05:20,375
去修改

139
00:05:20,375 --> 00:05:21,615
例如刚才讲到的

140
00:05:21,615 --> 00:05:22,655
把一些标准的卷积

141
00:05:22,775 --> 00:05:24,495
变成一个depthwise的卷积

142
00:05:24,495 --> 00:05:26,415
然后做一个group的卷积

143
00:05:26,415 --> 00:05:27,655
就分组的卷积

144
00:05:27,815 --> 00:05:28,855
分组卷积之前

145
00:05:28,975 --> 00:05:31,295
可能还会做一个shuffle的操作

146
00:05:31,295 --> 00:05:32,695
那可以看到这种

147
00:05:32,855 --> 00:05:34,455
确实就是shufflenet的

148
00:05:34,455 --> 00:05:36,415
一些新结构的提出

149
00:05:37,987 --> 00:05:38,575
接下来

150
00:05:38,575 --> 00:05:41,215
看一下大模型分布式并行

151
00:05:41,215 --> 00:05:42,055
第二个内容

152
00:05:42,055 --> 00:05:44,375
对AI计算模式的一个重要的改变

153
00:05:44,375 --> 00:05:46,295
大模型现在非常的火了

154
00:05:46,295 --> 00:05:48,295
现在像ChatGPT已经火的不行了

155
00:05:49,145 --> 00:05:51,185
同样ZOMI在很早之前

156
00:05:51,185 --> 00:05:53,665
讲AI框架的核心技术的时候

157
00:05:53,665 --> 00:05:55,785
就分开了三个主要的内容

158
00:05:55,785 --> 00:05:57,465
因为AI框架的最核心的技术

159
00:05:57,465 --> 00:05:58,945
我觉得分布式

160
00:05:58,985 --> 00:06:00,705
绝对占了其中的一个大头

161
00:06:00,705 --> 00:06:01,945
所以我分开了

162
00:06:02,185 --> 00:06:03,145
三个主要大的内容

163
00:06:03,145 --> 00:06:04,705
给大家去汇报过的

164
00:06:04,705 --> 00:06:07,305
第一个就是分布式的系统

165
00:06:07,305 --> 00:06:09,065
是讲系统的AI框架

166
00:06:09,065 --> 00:06:10,505
通讯集群的管理

167
00:06:10,505 --> 00:06:12,105
这个是跟系统相关的

168
00:06:12,145 --> 00:06:14,585
接着我又给大家去汇报了一个

169
00:06:14,585 --> 00:06:16,225
分布式的算法

170
00:06:16,225 --> 00:06:18,065
就是从10亿规模的模型

171
00:06:18,065 --> 00:06:19,305
到万亿规模

172
00:06:19,305 --> 00:06:20,465
从Transformer

173
00:06:20,465 --> 00:06:21,305
bird出现

174
00:06:21,305 --> 00:06:23,545
最后到moet5

175
00:06:23,545 --> 00:06:25,305
这种大规模的网络模型

176
00:06:25,305 --> 00:06:26,745
就讲分布式的算法

177
00:06:26,785 --> 00:06:29,265
接着有了分布式的系统和算法之外

178
00:06:29,545 --> 00:06:30,853
确实基于系统之上

179
00:06:30,853 --> 00:06:32,874
要做很多并行的工作

180
00:06:32,874 --> 00:06:33,834
例如模型并行

181
00:06:33,834 --> 00:06:34,594
数据并行

182
00:06:34,594 --> 00:06:35,505
张量并行

183
00:06:35,505 --> 00:06:36,387
pipeline并行等

184
00:06:36,387 --> 00:06:38,062
很多并行的操作

185
00:06:38,062 --> 00:06:39,180
不同的并行的操作

186
00:06:39,180 --> 00:06:40,380
决定这些模型

187
00:06:40,380 --> 00:06:41,220
这些算法

188
00:06:41,260 --> 00:06:45,180
怎么跑在AI系统里面

189
00:06:45,420 --> 00:06:47,620
如果大家不了解也可以回头去看一看

190
00:06:47,660 --> 00:06:50,660
接下来重新的回到这一个内容里面

191
00:06:50,660 --> 00:06:51,300
可以看到

192
00:06:51,300 --> 00:06:52,940
在最新的2020年里面

193
00:06:53,100 --> 00:06:53,940
foundation model

194
00:06:53,940 --> 00:06:54,980
就大模型

195
00:06:55,220 --> 00:06:57,020
已经成为一种新的范式

196
00:06:57,020 --> 00:06:58,900
新的一种AI的牵引的方向

197
00:06:58,900 --> 00:07:00,660
特别是ChatGPT

198
00:07:00,900 --> 00:07:02,620
这个时候看到并不是

199
00:07:02,620 --> 00:07:04,060
万丈高楼从头建的

200
00:07:04,060 --> 00:07:05,860
而是它有一个时间的序列

201
00:07:05,860 --> 00:07:07,380
或者有一些演进的

202
00:07:07,380 --> 00:07:09,540
从一开始的GPT123

203
00:07:09,540 --> 00:07:10,740
到现在ChatGPT

204
00:07:10,740 --> 00:07:12,260
也是有一个改变的

205
00:07:12,260 --> 00:07:13,780
这里面最明显的一个趋势

206
00:07:14,020 --> 00:07:16,540
就是训练的flop数

207
00:07:16,540 --> 00:07:17,900
计算量

208
00:07:17,940 --> 00:07:20,380
极大的提升和膨胀

209
00:07:20,380 --> 00:07:21,620
包括现在的ChatGPT

210
00:07:21,780 --> 00:07:23,500
用到的网络模型的规模

211
00:07:23,500 --> 00:07:25,620
已经超过千亿了

212
00:07:25,780 --> 00:07:27,980
下面一起简单的回顾一下

213
00:07:27,980 --> 00:07:29,820
之前给大家分享过的内容

214
00:07:30,020 --> 00:07:31,620
第一个就是数据并行

215
00:07:31,620 --> 00:07:34,140
数据并行里面其实分为DP

216
00:07:34,380 --> 00:07:36,420
这个时候最简单的数据并行

217
00:07:36,420 --> 00:07:37,780
还有分布式数据并行

218
00:07:37,780 --> 00:07:38,500
DDP

219
00:07:38,500 --> 00:07:40,740
另外还有FSDP

220
00:07:40,900 --> 00:07:43,020
这三种都可以在PyTorch里面

221
00:07:43,020 --> 00:07:44,140
去看得到的

222
00:07:44,140 --> 00:07:46,220
而这里面DP是最简单的

223
00:07:46,220 --> 00:07:47,620
就是把训练的数据

224
00:07:47,980 --> 00:07:49,580
分布在不同的机器

225
00:07:49,780 --> 00:07:52,540
这也是最简单的其中一种

226
00:07:52,540 --> 00:07:55,020
而下面看看这个图

227
00:07:55,020 --> 00:07:56,300
训练的数据

228
00:07:56,300 --> 00:07:57,660
分布在不同的机器

229
00:07:57,820 --> 00:07:59,180
上面是一台机器

230
00:07:59,180 --> 00:08:01,580
下面也是另外一台机器

231
00:08:01,580 --> 00:08:03,460
分在不同的机器上面去执行

232
00:08:03,460 --> 00:08:05,300
不过值得注意的就是DP

233
00:08:05,300 --> 00:08:07,300
所谓的data parallelism

234
00:08:07,300 --> 00:08:09,260
不仅仅是指训练的数据

235
00:08:09,260 --> 00:08:10,780
还有权重的数据

236
00:08:10,780 --> 00:08:12,220
同步的数据

237
00:08:12,220 --> 00:08:14,260
训练的去做并行的

238
00:08:14,460 --> 00:08:15,540
这个时候可以看到

239
00:08:15,540 --> 00:08:17,260
这里面就有一条直线

240
00:08:17,260 --> 00:08:19,220
可以做一个all-reduce的操作

241
00:08:19,220 --> 00:08:21,020
把数据进行同步

242
00:08:21,180 --> 00:08:24,140
另外的话来到了FSDP

243
00:08:24,140 --> 00:08:25,860
FSDP的并行的模式

244
00:08:26,020 --> 00:08:27,380
就更加复杂了

245
00:08:27,380 --> 00:08:30,180
它做了一个全切分的操作

246
00:08:30,576 --> 00:08:31,980
当网络模型

247
00:08:31,980 --> 00:08:33,380
大到一定程度的时候

248
00:08:33,540 --> 00:08:36,340
就没办法去把一个网络模型

249
00:08:36,340 --> 00:08:37,820
直接塞到一张卡里面

250
00:08:38,060 --> 00:08:40,060
这个时候可能把N层layer

251
00:08:40,100 --> 00:08:41,380
放在一台机器

252
00:08:41,380 --> 00:08:42,300
放在一张卡

253
00:08:42,300 --> 00:08:44,460
再把N层layer放在一张卡

254
00:08:44,660 --> 00:08:46,700
这个时候会去做很多

255
00:08:46,700 --> 00:08:48,540
大量的并行的操作

256
00:08:48,540 --> 00:08:49,580
并行的模式

257
00:08:49,580 --> 00:08:50,780
一旦并行

258
00:08:50,780 --> 00:08:52,020
就有很多数据

259
00:08:52,020 --> 00:08:53,580
进行交互和同步

260
00:08:53,580 --> 00:08:54,660
有很多数据

261
00:08:54,820 --> 00:08:56,100
要做并行的操作

262
00:08:56,100 --> 00:08:58,460
所以这里面就会对权重的数据

263
00:08:58,460 --> 00:08:59,580
进行一个同步

264
00:08:59,580 --> 00:09:01,140
对权重的数据进行同步

265
00:09:01,140 --> 00:09:03,220
对优化器的数据进行同步

266
00:09:03,220 --> 00:09:05,260
所以说有了FSDP

267
00:09:05,460 --> 00:09:06,700
FSDP具体的原理

268
00:09:06,940 --> 00:09:09,700
在前面其实也是跟大家分享过的

269
00:09:09,900 --> 00:09:12,620
这个就是FSDP的更加复杂的操作

270
00:09:12,620 --> 00:09:15,220
就不在这里面去详细的打开

271
00:09:15,220 --> 00:09:16,220
有兴趣的同学

272
00:09:16,340 --> 00:09:18,700
可以回看之前的一些分享

273
00:09:19,620 --> 00:09:23,940
下面以Megatron-LM大模型作为例子

274
00:09:23,940 --> 00:09:25,940
去看看整体的解决方案

275
00:09:25,940 --> 00:09:27,860
因为其实有很多并行的模式

276
00:09:28,060 --> 00:09:29,700
这里面就融合了

277
00:09:29,700 --> 00:09:30,660
Pipeline的并行

278
00:09:30,660 --> 00:09:33,260
还有张量的并行两种方式

279
00:09:33,260 --> 00:09:34,620
通过不同的并行方式

280
00:09:34,620 --> 00:09:35,060
当然了

281
00:09:35,060 --> 00:09:37,660
DP就数据并行是默认使用的

282
00:09:37,700 --> 00:09:39,980
模型并行两个融合起来

283
00:09:39,980 --> 00:09:41,660
就变成下面的图

284
00:09:41,660 --> 00:09:43,980
具体的就是Pipeline的时序图

285
00:09:43,980 --> 00:09:46,300
中间的空的就是一些bubble

286
00:09:46,300 --> 00:09:48,660
这里面可以看到Megatron-LM

287
00:09:48,820 --> 00:09:51,020
就提出了更多新的算法

288
00:09:51,020 --> 00:09:53,980
把很多的batch变成一个microbatch

289
00:09:53,980 --> 00:09:56,820
然后同步的方式也进行了一个改进

290
00:09:57,140 --> 00:09:58,380
具体简单的来说

291
00:09:58,380 --> 00:09:59,500
就是在devices

292
00:09:59,500 --> 00:10:02,580
就硬件卡的数量不变之下

293
00:10:02,940 --> 00:10:05,540
分出更多的Pipeline的stage

294
00:10:05,540 --> 00:10:08,700
就假设现在所有的一些卡

295
00:10:08,900 --> 00:10:10,100
是放在GPU1的

296
00:10:10,100 --> 00:10:13,500
于是可以分开更多不同的stage

297
00:10:13,500 --> 00:10:15,660
然后以更多的通讯量

298
00:10:15,660 --> 00:10:18,060
换取更少的空泡率

299
00:10:18,260 --> 00:10:20,060
这个就是具体的Megatron-LM

300
00:10:20,060 --> 00:10:21,340
所做的一些工作

301
00:10:21,340 --> 00:10:24,180
它的效果也是非常的好

302
00:10:24,460 --> 00:10:26,340
最后还是一样的

303
00:10:26,340 --> 00:10:28,180
回到AI的计算模式

304
00:10:28,180 --> 00:10:28,980
对大模型

305
00:10:28,980 --> 00:10:30,220
对分布式并行

306
00:10:30,220 --> 00:10:31,620
提出的一些思考

307
00:10:31,620 --> 00:10:33,820
这里面我总结了两点

308
00:10:33,820 --> 00:10:34,620
不一定对

309
00:10:34,620 --> 00:10:35,740
大家可以听一听

310
00:10:35,740 --> 00:10:36,980
也可以吐槽吐槽

311
00:10:36,980 --> 00:10:38,940
那下面就是芯片间

312
00:10:38,940 --> 00:10:40,900
必须要互联技术的支持

313
00:10:40,900 --> 00:10:43,180
提供XXGB的带宽

314
00:10:43,180 --> 00:10:44,860
也就是芯片间

315
00:10:44,860 --> 00:10:46,620
NPU跟NPU的卡之间

316
00:10:46,620 --> 00:10:48,660
需要进行一个互联互通

317
00:10:48,660 --> 00:10:50,340
而且需要有一个大的带宽

318
00:10:50,340 --> 00:10:53,500
就是因为能够适配到大模型

319
00:10:53,500 --> 00:10:54,460
大模型刚才讲了

320
00:10:54,460 --> 00:10:56,380
有很多的并行的操作

321
00:10:56,380 --> 00:10:57,420
一旦涉及到并行

322
00:10:57,420 --> 00:10:58,260
数据

323
00:10:58,260 --> 00:10:59,740
就需要不断的去传输

324
00:10:59,740 --> 00:11:00,540
模型

325
00:11:00,540 --> 00:11:02,020
就要进行一个切分

326
00:11:02,020 --> 00:11:04,420
这个就是同构的芯片类

327
00:11:04,420 --> 00:11:07,580
希望能够有更好的互联的技术

328
00:11:07,580 --> 00:11:08,980
那英伟达就推出了

329
00:11:08,980 --> 00:11:11,140
NVLink和NVSwitch

330
00:11:11,140 --> 00:11:13,500
而另外一种就是需要支持

331
00:11:13,500 --> 00:11:16,020
CPU加GPU的双架构

332
00:11:16,020 --> 00:11:17,580
为整个大规模的AI

333
00:11:17,580 --> 00:11:18,660
就大模型

334
00:11:18,860 --> 00:11:20,220
还有HPC的异构平台

335
00:11:20,580 --> 00:11:22,220
提供高带宽

336
00:11:22,220 --> 00:11:25,380
这里面从英伟达的H100

337
00:11:25,380 --> 00:11:27,140
虽然现在没有太多的供货

338
00:11:27,140 --> 00:11:29,900
（可以看到）CPU加GPU的这种双架构

339
00:11:29,900 --> 00:11:33,580
确实能给性能带来极大的提升

340
00:11:33,700 --> 00:11:36,340
那第二种就是整体的高速的

341
00:11:36,340 --> 00:11:37,460
Transformer引擎

342
00:11:37,460 --> 00:11:38,940
因为大模型主要是以

343
00:11:38,940 --> 00:11:40,100
Transformer为结构

344
00:11:40,100 --> 00:11:41,260
进行一个堆叠的

345
00:11:41,260 --> 00:11:43,340
所以这个时候提供高速的

346
00:11:43,340 --> 00:11:44,740
Transformer引擎来说

347
00:11:44,740 --> 00:11:47,140
对整个大模型的速度提升

348
00:11:47,140 --> 00:11:48,940
是非常有帮助的

349
00:11:49,180 --> 00:11:51,620
第二个就是大模型的时候

350
00:11:51,780 --> 00:11:53,660
其实并不是完全都要使用

351
00:11:53,660 --> 00:11:55,100
FP32去训练的

352
00:11:55,100 --> 00:11:56,780
还可以用FP16

353
00:11:56,780 --> 00:11:57,380
另外的话

354
00:11:57,380 --> 00:11:58,900
可能大模型还可以支持

355
00:11:58,900 --> 00:12:00,540
MoE的去构建

356
00:12:00,540 --> 00:12:02,220
万亿规模的大模型的结构

357
00:12:02,380 --> 00:12:04,820
这个就是对AI计算模式的思考

358
00:12:05,260 --> 00:12:07,500
讲完所有之后

359
00:12:07,940 --> 00:12:10,740
总体的去回顾一下

360
00:12:10,740 --> 00:12:12,300
做一个summary

361
00:12:12,300 --> 00:12:14,180
整个AI的计算模式

362
00:12:14,180 --> 00:12:15,700
其实把之前的工作

363
00:12:15,820 --> 00:12:17,220
或者之前的汇报

364
00:12:17,260 --> 00:12:18,580
简单地串一串

365
00:12:18,580 --> 00:12:20,300
这里面没有新的知识

366
00:12:20,660 --> 00:12:22,100
可以回顾一下

367
00:12:22,100 --> 00:12:23,420
第一个就是讲了

368
00:12:23,420 --> 00:12:25,100
一些经典的网络模型的

369
00:12:25,100 --> 00:12:26,020
结构的知识

370
00:12:26,220 --> 00:12:27,140
经典的网络模型

371
00:12:27,260 --> 00:12:28,580
需要支持高维的

372
00:12:28,580 --> 00:12:29,460
张量的存储

373
00:12:29,460 --> 00:12:31,220
支持一些非常复杂的

374
00:12:31,220 --> 00:12:32,580
神经网络的计算

375
00:12:32,580 --> 00:12:35,060
因为很多sota的网络模型

376
00:12:35,060 --> 00:12:36,620
很多sota的算法

377
00:12:36,820 --> 00:12:38,020
AI的芯片

378
00:12:38,020 --> 00:12:39,260
AI的计算模式

379
00:12:39,260 --> 00:12:40,340
必须要考虑到的

380
00:12:40,340 --> 00:12:43,220
第二点就是模型的压缩和量化

381
00:12:43,460 --> 00:12:44,540
提到压缩量化

382
00:12:44,780 --> 00:12:45,980
基本上很明显

383
00:12:45,980 --> 00:12:48,740
量化就需要提供低比特的位数

384
00:12:48,740 --> 00:12:52,100
而剪枝就需要硬件提供稀疏的计算

385
00:12:52,100 --> 00:12:54,700
另外来到今天给大家去分享的内容

386
00:12:54,700 --> 00:12:55,580
就是轻量化

387
00:12:55,580 --> 00:12:56,500
轻量化的时候

388
00:12:56,500 --> 00:12:57,860
希望能够硬件

389
00:12:57,860 --> 00:12:59,060
或者计算模式

390
00:12:59,060 --> 00:13:01,740
会需要有支持很多复杂的计算

391
00:13:01,740 --> 00:13:03,140
例如小卷积核的计算

392
00:13:03,260 --> 00:13:04,420
1×1的卷积

393
00:13:04,420 --> 00:13:06,260
另外还复用卷积核的

394
00:13:06,260 --> 00:13:08,100
很多内存的信息

395
00:13:08,100 --> 00:13:09,700
就是convolution的weight

396
00:13:09,700 --> 00:13:12,180
或者filter进行一个reused

397
00:13:12,380 --> 00:13:15,124
最后一个就是大模型分布式并行

398
00:13:15,124 --> 00:13:15,940
这个内容

399
00:13:15,940 --> 00:13:18,700
是希望能够有大的

400
00:13:18,700 --> 00:13:20,820
内存容量和高速带宽

401
00:13:20,820 --> 00:13:22,900
另外还希望有一个大模型的

402
00:13:22,900 --> 00:13:23,980
DSA模块

403
00:13:23,980 --> 00:13:26,380
特别是针对Transform的结构

404
00:13:26,420 --> 00:13:28,860
另外还可以提供一些低比特的

405
00:13:28,860 --> 00:13:29,580
bit wave

406
00:13:29,580 --> 00:13:31,660
然后进行一个高速的运算

407
00:13:31,660 --> 00:13:33,740
这个就是对整个

408
00:13:33,740 --> 00:13:35,380
AI计算模式的总结和思考

409
00:13:35,380 --> 00:13:35,820
好了

410
00:13:35,820 --> 00:13:37,769
今天的内容就到这里为止

411
00:13:37,975 --> 00:13:39,558
卷的不行了

412
00:13:39,558 --> 00:13:40,958
记得一键三连加关注

413
00:13:41,358 --> 00:13:42,718
所有的内容都会开源

414
00:13:42,718 --> 00:13:44,478
在下面这条链接里面

415
00:13:44,918 --> 00:13:45,798
摆了个拜