Inference with C++ interface under OpenMP

[KawaSakiHakuButsu]

When I make inference with C++ interface under OpenMP in my code, there are different errors about memory for repeated tests as following.
e. g. 1

e. g. 2

e. g. 3

Interestingly, if I "cout" something around the inference function "compute", all errors disappear. Meanwhile, as long as the line with "dp.compute" is deleted together with some necessary changes, thers's also no errors.
Specifically, I use inference under OpenMP in the work-sharing construct "for" like #pragma omp parallel for.

To clarify the connection between inference with C++ and above errors, I also test via the example of H2O. The code is put in the below

#include "deepmd/DeepPot.h"
#include <iostream>
#include <omp.h>

typedef double ValueType;

int main(){
  deepmd::DeepPot dp ("graph.pb");
  #pragma omp parallel for num_threads(16) schedule(static, 2)
  for (int i = 0 ; i < 36 ; ++i){
    std::vector<ValueType > coord = {1., 0., 0., 0., 0., 1.5, 1. ,0. ,3.};
    std::vector<ValueType > cell = {10., 0., 0., 0., 10., 0., 0., 0., 10.};
    std::vector<int > atype = {1, 0, 1};
    double e;
    std::vector<ValueType > f, v;
    dp.compute(e, f, v, coord, atype, cell);
    std::cout << e << " " << omp_get_thread_num() << " " << omp_get_num_threads() << std::endl;
  }

However, sometimes there is error about (array) index out of bounds, like Segmentation fault (core dumped).

So how can I use inference with C++ interface under OpenMP correctly?
Thank you!

[njzjz]

I don't think it's thread-safe. Also note that TensorFlow itself utilizes multiple threads.

[KawaSakiHakuButsu]

Could you provide more detailed explanation for the lack of thread-safety?
Thank you!

[njzjz]

I am unsure whether TensorFlow is thread-safe, but at least it never claims or is designed to be.

[KawaSakiHakuButsu]

I get your point now.
However, when I want to do inference on different structures simultaneously, how can I improve calculation efficiency? If it's convenient for you, could you provide me with some ideas?
Thank you!

[njzjz]

I don't know that you program is but you can try MPI. If they are not required to infer in the same time, you do not need to run only a single program.

[KawaSakiHakuButsu]

Thank you for your suggestions, I'll have a try.