Skip to content

Commit 668ae90

Browse files
jacquesqiaojacquesqiao
committed
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into add-ctr-reader
2 parents aff54ef + 3ae6692 commit 668ae90

File tree

1,010 files changed

+45097
-14915
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,010 files changed

+45097
-14915
lines changed

‎.github/ISSUE_TEMPLATE/---feature-request-.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
---
2+
name: 建议(Feature request)
3+
about: 您可以提出您的建议。 You could use this template for reporting a suggestion  issue.
4+
5+
---
6+
7+
欢迎您对PaddlePaddle提出建议，非常感谢您对PaddlePaddle的贡献！
8+
在留下您的建议时，辛苦您同步提供如下信息：
9+
- 版本、环境信息
10+
1）PaddlePaddle版本：请提供您的PaddlePaddle版本号，例如1.1
11+
2）CPU/GPU：您是否使用GPU进行训练，如是，请提供您的CUDA和cuDNN版本号
12+
3）系统环境：请您描述系统类型、版本，例如Mac OS 10.14
13+
- 复现信息：如为报错，请给出复现环境、复现步骤
14+
- 建议描述：请您详细描述，您认为需优化的功能
15+
16+
Thank you for contributing to PaddlePaddle.
17+
Before submitting the issue, you could search issue in the github in case that there was a similar issue submitted or resolved before.
18+
Please make sure that this is a feature request.
19+
**System information**
20+
-PaddlePaddle version （eg.1.1）or CommitID
21+
-CPU: including CPUMKL/OpenBlas/MKLDNN version
22+
-GPU: including CUDA/CUDNN version
23+
-OS Platform (eg.Mac OS 10.14)
24+
**To Reproduce**
25+
Steps to reproduce the behavior
26+
**Describe the feature and the current behavior/state.**
27+
**Any Other info.**

‎.github/ISSUE_TEMPLATE/---inference-issue-.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
name: 预测（Inference Issue）
3+
about: 您可以提问预测中报错、应用等问题。 You could use this template for reporting an inference issue.
4+
5+
---
6+
7+
为使您的问题得到快速解决，在建立Issue前，请您先通过如下方式搜索是否有相似问题:【搜索issue关键字】【使用labels筛选】【官方文档】
8+
9+
如果您没有查询到相似问题，为快速解决您的提问，建立issue时请提供如下细节信息：
10+
- 标题：简洁、精准描述您的问题，例如“最新预测库的API文档在哪儿 ”
11+
- 版本、环境信息：
12+
   1）PaddlePaddle版本：请提供您的PaddlePaddle版本号（如1.1）或CommitID
13+
   2）CPU：预测若用CPU，请提供CPU型号，MKL/OpenBlas/MKLDNN/等数学库使用情况
14+
   3）GPU：预测若用GPU，请提供GPU型号、CUDA和CUDNN版本号
15+
   4）系统环境：请您描述系统类型、版本（如Mac OS 10.14），Python版本
16+
-预测信息
17+
   1）C++预测：请您提供预测库安装包的版本信息，及其中的version.txt文件
18+
   2）CMake包含路径的完整命令
19+
   3）API信息（如调用请提供）
20+
   4）预测库来源：官网下载/特殊环境（如BCLOUD编译）
21+
- 复现信息：如为报错，请给出复现环境、复现步骤
22+
- 问题描述：请详细描述您的问题，同步贴出报错信息、日志/代码关键片段
23+
24+
Thank you for contributing to PaddlePaddle.
25+
Before submitting the issue, you could search issue in the github in case that th
26+
If there is no solution,please make sure that this is an inference issue including the following details :
27+
**System information**
28+
-PaddlePaddle version （eg.1.1）or CommitID
29+
-CPU: including CPUMKL/OpenBlas/MKLDNN version
30+
-GPU: including CUDA/CUDNN version
31+
-OS Platform (eg.Mac OS 10.14)
32+
-Python version
33+
-Cmake orders
34+
-C++version.txt
35+
-API information
36+
**To Reproduce**
37+
Steps to reproduce the behavior
38+
**Describe your current behavior**
39+
**Code to reproduce the issue**
40+
**Other info / logs**

‎.github/ISSUE_TEMPLATE/---installation-issue-.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
name: 安装（Installation Issue）
3+
about: 您可以提问安装、编译出现报错等问题。 You could use this template for reporting an installation
4+
 issue.
5+
6+
---
7+
8+
为使您的问题得到快速解决，在建立Issue前，请您先通过如下方式搜索是否有相似问题:【搜索issue关键字】【使用labels筛选】【官方文档】
9+
10+
建立issue时，为快速解决问题，请您根据使用情况给出如下信息：
11+
- 标题：请包含关键词“安装错误”/“编译错误”，例如“Mac编译错误”
12+
- 版本、环境信息：
13+
   1）PaddlePaddle版本：请提供您的PaddlePaddle版本号（如1.1）或CommitID
14+
   2）CPU：请提供CPU型号，MKL/OpenBlas/MKLDNN/等数学库的使用情况
15+
   3）GPU：请提供GPU型号，CUDA和CUDNN版本号
16+
   4）系统环境：请说明系统类型、版本（如Mac OS 10.14）、Python版本
17+
- 安装方式信息：
18+
1）pip安装/docker安装
19+
2）本地编译：请提供cmake命令，编译命令
20+
3）docker编译：请提供docker镜像，编译命令           
21+
 特殊环境请注明：如离线安装等
22+
- 复现信息：如为报错，请给出复现环境、复现步骤
23+
- 问题描述：请详细描述您的问题，同步贴出报错信息、日志/代码关键片段
24+
25+
Thank you for contributing to PaddlePaddle.
26+
Before submitting the issue, you could search issue in Github in case that there was a similar issue submitted or resolved before.
27+
If there is no solution,please make sure that this is an installation issue including the following details:
28+
**System information**
29+
-PaddlePaddle version （eg.1.1）or CommitID
30+
-CPU: including CPUMKL/OpenBlas/MKLDNN version
31+
-GPU: including CUDA/CUDNN version
32+
-OS Platform (eg. Mac OS 10.14)
33+
-Python version
34+
- Install method: pip install/install with docker/build from source(without docker)/build within docker
35+
- Other special cases that you think may be related to this problem, eg. offline install, special internet condition  
36+
**To Reproduce**
37+
Steps to reproduce the behavior
38+
**Describe your current behavior**
39+
**Code to reproduce the issue**
40+
**Other info / logs**

‎.github/ISSUE_TEMPLATE/---model-issue-.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
---
2+
name: 模型（Model Issue）
3+
about: 您可以提问模型、算法、数据集方向的使用报错等问题。You could use this template for reporting a model/
4+
algorithm/dataset  issue.
5+
6+
---
7+
8+
为使您的问题得到快速解决，在建立Issue前，请您先通过如下方式搜索是否有相似问题:【搜索issue关键字】【使用labels筛选】【官方文档】
9+
10+
建立issue时，为快速解决问题，请您根据使用情况给出如下信息：
11+
- 标题：简洁、精准描述您的问题，例如“ssd 模型前置lstm报错  ”
12+
- 版本、环境信息：
13+
   1）PaddlePaddle版本：请提供PaddlePaddle版本号，例如1.1或CommitID
14+
   2）CPU：请提供CPU型号，MKL/OpenBlas/MKLDNN/等数学库的使用情况
15+
   3）GPU：请提供GPU型号，CUDA和CUDNN版本号
16+
   4）系统环境：请说明系统类型、版本（例如Mac OS 10.14），Python版本
17+
- 模型信息
18+
   1）模型名称 2）使用数据集名称 3）使用算法名称 4）模型链接
19+
- 复现信息：如为报错，请给出复现环境、复现步骤
20+
- 问题描述：请详细描述您的问题，同步贴出报错信息、日志/代码关键片段
21+
22+
Thank you for contributing to PaddlePaddle.
23+
Before submitting the issue, you could search issue in the github.Probably there was a similar issue submitted or resolved before.
24+
If there is no solution,please make sure that this is a issue of models including the following details:
25+
**System information**
26+
-PaddlePaddle version （eg.1.1）or CommitID
27+
-CPU: including CPUMKL/OpenBlas/MKLDNN version
28+
-GPU: including CUDA/CUDNN version
29+
-OS Platform (eg.Mac OS 10.14)
30+
-Python version
31+
-Name of Models&Dataset/details of operator
32+
**To Reproduce**
33+
Steps to reproduce the behavior
34+
**Describe your current behavior**
35+
**Code to reproduce the issue**
36+
**Other info / logs**

‎.github/ISSUE_TEMPLATE/---others-.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
name: 其他（Others）
3+
about: 如上述分类未包含您的问题，可在此提出。 You could use this template for reporting other issues
4+
5+
---
6+
7+
为使您的问题得到快速解决，在建立Issues前，请您先通过如下方式搜索是否有相似问题:【搜索issue关键字】【使用labels筛选】【官方文档】
8+
9+
如果您没有查询到相似问题，为快速解决您的提问，建立issue时请提供如下细节信息：
10+
- 标题：简洁、精准概括您的问题
11+
- 版本、环境信息：
12+
   1）PaddlePaddle版本：请提供您的PaddlePaddle版本号，例如1.1或CommitID
13+
   2）CPU/GPU：如果您使用GPU训练，请提供GPU驱动版本、CUDA和cuDNN版本号
14+
   3）系统环境：请您描述系统类型、版本，例如Mac OS 10.14
15+
   4）Python版本号
16+
   5）显存信息
17+
- 复现信息：如为报错，请给出复现环境、复现步骤
18+
- 问题描述：请详细描述您的问题，同步贴出报错信息、日志/代码关键片段
19+
20+
Thank you for contributing to PaddlePaddle.
21+
Before submitting the issue, you could search issue in the github in case that there was a similar issue submitted or resolved before.
22+
If there is no solution,please provide us with the following details :
23+
**System information**
24+
-PaddlePaddle version （eg.1.1）or CommitID
25+
-CPU: including CPUMKL/OpenBlas/MKLDNN version
26+
-GPU: including CUDA/cuDNN version
27+
-OS Platform and Distribution(eg.Mac OS 10.14)
28+
-Python version
29+
**To Reproduce**
30+
Steps to reproduce the behavior
31+
**Describe your current behavior**
32+
**Code to reproduce the issue**
33+
**Other info / logs**

‎.github/ISSUE_TEMPLATE/---training-issue-.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
name: 训练（Training issue）
3+
about: 您可以提问训练中报错、应用、出core等问题。 You could use this template for reporting an training
4+
 issue.
5+
6+
---
7+
8+
为使您的问题得到快速解决，在建立Issues前，请您先通过如下方式搜索是否有相似问题:【搜索issue关键字】【使用labels筛选】【官方文档】
9+
10+
如果您没有查询到相似问题，为快速解决您的提问，建立issue时请提供如下细节信息：
11+
- 标题：简洁、精准概括您的问题，例如“Insufficient Memory xxx" ”
12+
- 版本、环境信息：
13+
   1）PaddlePaddle版本：请提供您的PaddlePaddle版本号，例如1.1或CommitID
14+
   2）CPU：预测若用CPU，请提供CPU型号，MKL/OpenBlas/MKLDNN/等数学库使用情况
15+
   3）GPU：预测若用GPU，请提供GPU型号、CUDA和CUDNN版本号
16+
   4）系统环境：请您描述系统类型、版本，例如Mac OS 10.14，Python版本
17+
- 训练信息
18+
   1）单机/多机，单卡/多卡
19+
   2）显存信息
20+
   3）Operator信息
21+
- 复现信息：如为报错，请给出复现环境、复现步骤
22+
- 问题描述：请详细描述您的问题，同步贴出报错信息、日志、可复现的代码片段
23+
24+
Thank you for contributing to PaddlePaddle.
25+
Before submitting the issue, you could search issue in the github in case that there was a similar issue submitted or resolved before.
26+
If there is no solution,please make sure that this is a training issue including the following details:
27+
**System information**
28+
-PaddlePaddle version （eg.1.1）or CommitID
29+
-CPU: including CPUMKL/OpenBlas/MKLDNN version
30+
-GPU: including CUDA/CUDNN version
31+
-OS Platform (eg.Mac OS 10.14)
32+
-Other imformation: Distriuted training/informantion of operator/
33+
Graphics card storage
34+
**To Reproduce**
35+
Steps to reproduce the behavior
36+
**Describe your current behavior**
37+
**Code to reproduce the issue**
38+
**Other info / logs**

‎.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
1+
python/paddle/fluid/tests/unittests/reader_reset_test.recordio
12
paddle/operators/check_t.save
23
paddle/operators/check_tensor.ls
34
paddle/operators/tensor.save
45
python/paddle/v2/fluid/tests/book/image_classification_resnet.inference.model/
56
python/paddle/v2/fluid/tests/book/image_classification_vgg.inference.model/
67
python/paddle/v2/fluid/tests/book/label_semantic_roles.inference.model/
8+
paddle/fluid/operators/distributed/send_recv.proto
79
*.DS_Store
810
*.vs
911
build/
@@ -28,3 +30,5 @@ third_party/
2830
build_*
2931
# clion workspace.
3032
cmake-build-*
33+
paddle/fluid/operators/distributed/send_recv.proto
34+
model_test

‎AUTHORS.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
| kexinzhao | Ke-Xin Zhao |
2626
| kuke | Yi-Bing Liu |
2727
| lcy-seso | Ying Cao |
28+
| cjld | Dun Liang |
2829
| lipeng-unisound | Peng Li |
2930
| liuyuan | Yuan Liu |
3031
| livc | Zhao Li |
@@ -42,7 +43,9 @@
4243
| QiJune | Jun Qi |
4344
| qingqing01 | Qing-Qing Dang |
4445
| reyoung | Yang Yu |
46+
| Sand3r- | Michal Gallus |
4547
| Superjom | Chun-Wei Yan |
48+
| tensor-tang | Jian Tang |
4649
| tianbingsz | Tian-Bing Xu |
4750
| tpatejko | Tomasz Patejko |
4851
| typhoonzero | Yi Wu |

‎CMakeLists.txt

Lines changed: 44 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,11 @@ message(STATUS "C compiler: ${CMAKE_C_COMPILER}, version: "
2626
"${CMAKE_C_COMPILER_ID} ${CMAKE_C_COMPILER_VERSION}")
2727
if(WIN32)
2828
set(CMAKE_STATIC_LIBRARY_PREFIX lib)
29+
add_definitions("/DGOOGLE_GLOG_DLL_DECL=")
30+
set(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /bigobj /MTd")
31+
set(CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} /bigobj /MT")
32+
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /bigobj /MTd")
33+
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /bigobj /MT")
2934
endif(WIN32)
3035

3136
if(NOT CMAKE_CROSSCOMPILING)
@@ -41,6 +46,7 @@ option(WITH_GPU "Compile PaddlePaddle with NVIDIA GPU" ${CUDA_F
4146
option(WITH_AMD_GPU "Compile PaddlePaddle with AMD GPU" OFF)
4247
option(WITH_AVX "Compile PaddlePaddle with AVX intrinsics" ${AVX_FOUND})
4348
option(WITH_MKL "Compile PaddlePaddle with MKL support." ${AVX_FOUND})
49+
option(WITH_NGRAPH "Compile PaddlePaddle with nGraph support." OFF)
4450
option(WITH_DSO "Compile PaddlePaddle with dynamic linked CUDA" ON)
4551
option(WITH_TESTING "Compile PaddlePaddle with unit testing" OFF)
4652
option(WITH_SWIG_PY "Compile PaddlePaddle with inference api" ON)
@@ -62,13 +68,14 @@ option(WITH_DISTRIBUTE "Compile with distributed support" OFF)
6268
option(USE_EIGEN_FOR_BLAS "Use matrix multiplication in Eigen" OFF)
6369
option(EIGEN_USE_THREADS "Compile with multi-threaded Eigen" OFF)
6470
option(WITH_ARM_FP16 "Use half precision support on armv8.2-a cpu" OFF)
65-
option(WITH_FAST_BUNDLE_TEST "Bundle tests that can be run in a single process together to reduce launch overhead" OFF)
6671
option(WITH_CONTRIB "Compile the third-party contributation" OFF)
6772
option(REPLACE_ENFORCE_GLOG "Replace PADDLE_ENFORCE with glog/CHECK for better debug." OFF)
6873
option(WITH_ANAKIN "Compile with Anakin library" OFF)
74+
option(ANAKIN_BUILD_FAT_BIN "Build anakin cuda fat-bin lib for all device plantform, ignored when WITH_ANAKIN=OFF" OFF)
75+
option(ANAKIN_BUILD_CROSS_PLANTFORM "Build anakin lib for any nvidia device plantform. ignored when WITH_ANAKIN=OFF" ON)
6976
option(WITH_GRPC "Use grpc as the default rpc framework" ${WITH_DISTRIBUTE})
7077
option(WITH_BRPC_RDMA "Use brpc rdma as the rpc protocal" OFF)
71-
option(WITH_INFERENCE "Compile fluid inference library" ON)
78+
option(ON_INFER "Turn on inference optimization." OFF)
7279
option(WITH_INFERENCE_API_TEST "Test fluid inference high-level api interface" OFF)
7380
option(WITH_SYSTEM_BLAS "Use system blas library" OFF)
7481
option(PY_VERSION "Compile PaddlePaddle with python3 support" ${PY_VERSION})
@@ -104,6 +111,8 @@ if(ANDROID OR IOS)
104111
"Disable RDMA when cross-compiling for Android and iOS" FORCE)
105112
set(WITH_MKL OFF CACHE STRING
106113
"Disable MKL when cross-compiling for Android and iOS" FORCE)
114+
set(WITH_NGRAPH OFF CACHE STRING
115+
"Disable nGraph when cross-compiling for Android and iOS" FORCE)
107116
set(WITH_GOLANG OFF CACHE STRING
108117
"Disable golang when cross-compiling for Android and iOS" FORCE)
109118

@@ -121,12 +130,30 @@ if (APPLE OR WIN32)
121130
"Disable MKL for building on mac and windows" FORCE)
122131
endif()
123132

133+
if (WIN32)
134+
set(WITH_AVX OFF CACHE STRING
135+
"Disable AVX when compiling for Windows" FORCE)
136+
set(WITH_DSO OFF CACHE STRING
137+
"Disable DSO when compiling for Windows" FORCE)
138+
set(WITH_MKL OFF CACHE STRING
139+
"Disable MKL when compiling for Windows" FORCE)
140+
set(WITH_DISTRIBUTE OFF CACHE STRING
141+
"Disable DISTRIBUTE when compiling for Windows" FORCE)
142+
set(WITH_C_API OFF CACHE STRING
143+
"Disable C_API when compiling for Windows" FORCE)
144+
set(WITH_FLUID_ONLY ON CACHE STRING
145+
"Enable FLUID_ONLY when compiling for Windows" FORCE)
146+
endif()
147+
124148
set(THIRD_PARTY_PATH "${CMAKE_BINARY_DIR}/third_party" CACHE STRING
125149
"A path setting third party libraries download & build directories.")
126150

127151
set(FLUID_INSTALL_DIR "${CMAKE_BINARY_DIR}/fluid_install_dir" CACHE STRING
128152
"A path setting fluid shared and static libraries")
129153

154+
set(FLUID_INFERENCE_INSTALL_DIR "${CMAKE_BINARY_DIR}/fluid_inference_install_dir" CACHE STRING
155+
"A path setting fluid inference shared and static libraries")
156+
130157
if (WITH_C_API AND WITH_PYTHON)
131158
message(WARNING "It is suggest not embedded a python interpreter in Paddle "
132159
"when using C-API. It will give an unpredictable behavior when using a "
@@ -169,6 +196,7 @@ include(external/protobuf) # download, build, install protobuf
169196
include(external/python) # download, build, install python
170197
include(external/openblas) # download, build, install openblas
171198
include(external/mkldnn) # download, build, install mkldnn
199+
include(external/ngraph) # download, build, install nGraph
172200
include(external/swig) # download, build, install swig
173201
include(external/boost) # download boost
174202
include(external/any) # download libn::any
@@ -177,11 +205,14 @@ include(external/pybind11) # download pybind11
177205
include(external/cares)
178206
include(external/cub)
179207
include(external/gzstream)
180-
181-
if (NOT WIN32)
182-
# there is no official support of snappystream, warpctc, nccl, cupti in windows
208+
include(external/rocprim)
209+
include(external/xxhash) # download xxhash
210+
include(external/dlpack)
183211
include(external/snappy) # download snappy
184212
include(external/snappystream) # download snappystream
213+
214+
if (NOT WIN32)
215+
# there is no official support of warpctc, nccl, cupti in windows
185216
include(external/warpctc) # download, build, install warpctc
186217
include(cupti)
187218
endif (NOT WIN32)
@@ -289,6 +320,14 @@ set(PADDLE_PYTHON_BUILD_DIR "${CMAKE_CURRENT_BINARY_DIR}/python/build")
289320
set(CMAKE_CXX_FLAGS_RELWITHDEBINFO "-O3 -g -DNDEBUG")
290321
set(CMAKE_C_FLAGS_RELWITHDEBINFO "-O3 -g -DNDEBUG")
291322

323+
if (ON_INFER)
324+
message(STATUS "On inference mode, will take place some specific optimization.")
325+
add_definitions(-DPADDLE_ON_INFERENCE)
326+
else()
327+
#TODO(luotao), combine this warning with `make inference_lib_dist` command.
328+
message(WARNING "On inference mode, will take place some specific optimization. Turn on the ON_INFER flag when building inference_lib only.")
329+
endif()
330+
292331
add_subdirectory(paddle)
293332
if(WITH_PYTHON)
294333
add_subdirectory(python)

0 commit comments

Comments
 (0)