Bazel Build

[reyoung]

Related Issue #681, #819, #818

The issue used for tracking Paddle feature/bazel_build develop status.

Problems:

• How bazel read command line flags or environment variable? It is very useful for Paddle to make various binary file. Such as WITH_GPU, WITH_AVX, WITH_DOUBLE, etc. See here.

• Currently, reference Python library just use a precompiled static library. It may be a critical defect. See here. Is there a better way to reference a third party library, without manually writing a bazel build for it?

TODOs:

• incorporate gtest and protobuf Incorporate gtest and protobuf #816
• incorporate glog and gflags Incorporate glog and gflags into Bazel #822
• paddle python package
• paddle_proto module
  • Remove m4 macro language when generating protobuf. Done with Remove m4 when generate protobuf #846
• logging using glog Logging using glog #869
• paddle_util module
  • Related ISSUE Make paddle_util not to reference cuda #793
  • Under developing. Maybe some new issues will be added.
• paddle_math module.
• gserver module.
• pserver module
• trainer module
• api module

[wangkuiyi]

@reyoung About the Python library: it that because Tensorflow only builds itself as a library callable from Python, but not embedding Python interpreter into it, so we cannot learn from Tensorflow's BUILD files?

I noticed that you packed Python as a tarball. Can you share how did you do that?

Also that tarball's full path includes substring "macos". Is that tarball only for Mac OS X?

[wangkuiyi]

One thing to remind is that we should keep all third-party dependencies in a subdirectory, usually named third_party. A good example is:

https://github.com/tensorflow/tensorflow/tree/master/third_party

This would allow us to inspect and manage all our third-party dependencies because any introduction of a new third-party dependency would have to go through a PR that adds a sub-directory and/or a .build/.bzl file into thrid-party.

[reyoung]

it that because Tensorflow only builds itself as a library callable from Python, but not embedding Python interpreter into it, so we cannot learn from Tensorflow's BUILD files?

You're right. We cannot learn from Tensorflow's BUILD. To manually write a bazel build file for Python interpreter is a complicated job. So currently, we just give a prebuild binary for Python dependency.

I noticed that you packed Python as a tarball. Can you share how did you do that?

I just copy my laptop system's library and header now. There is a personal repository for building static Python Interpreter. I'm trying to refine this scripts, and then it can be used to make a full static tar.gz package for both Linux and MacOS.

[wangkuiyi]

To demonstrate how to use protobuf wth Bazel-managed projects, I wrote this example project https://github.com/wangkuiyi/learn-bazel. It also shows how to develop using Docker and Bazel.

[reyoung]

Closed because we choose cmake to manage paddle build