Using neighbor-stat

[nisargjoshi10]

Hey all,

I am trying to using the neighbor-stat command for determining the sel value for my system. I am not sure what data to use here, I have tried using directories with raw files and npy files and for both I get,

RuntimeError: Did not find valid system

error. Not sure which path is supposed to be added for the -s flag. Any help would be appreciated.

Thanks!

[njzjz]

Examples are provided here.

[nisargjoshi10]

@njzjz I've tried those examples for the command too but I get the same error

(base) [nisargj@n3353 data]$ ls
data_0  data_1  data_2  data_3
(base) [nisargj@n3353 data]$ apptainer exec ../deepmd-kit_2.1.1_cuda11.6_gpu.sif dp neighbor-stat -s . -r 6 -t H O
WARNING:tensorflow:From /opt/deepmd-kit/lib/python3.10/site-packages/tensorflow/python/compat/v2_compat.py:107: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
WARNING:root:To get the best performance, it is recommended to adjust the number of threads by setting the environment variables OMP_NUM_THREADS, TF_INTRA_OP_PARALLELISM_THREADS, and TF_INTER_OP_PARALLELISM_THREADS.
WARNING:root:Environment variable KMP_BLOCKTIME is empty. Use the default value 0
WARNING:root:Environment variable KMP_AFFINITY is empty. Use the default value granularity=fine,verbose,compact,1,0
/opt/deepmd-kit/lib/python3.10/importlib/__init__.py:169: UserWarning: The NumPy module was reloaded (imported a second time). This can in some cases result in small but subtle issues and is discouraged.
  _bootstrap._exec(spec, module)
Traceback (most recent call last):
  File "/opt/deepmd-kit/bin/dp", line 10, in <module>
    sys.exit(main())
  File "/opt/deepmd-kit/lib/python3.10/site-packages/deepmd/entrypoints/main.py", line 491, in main
    neighbor_stat(**dict_args)
  File "/opt/deepmd-kit/lib/python3.10/site-packages/deepmd/entrypoints/neighbor_stat.py", line 36, in neighbor_stat
    raise RuntimeError("Did not find valid system")
RuntimeError: Did not find valid system

I even tried changing the path of the data by running the command with -s data but that doesn't work either

[njzjz]

It looks you load an image but didn't mount the path?

[nisargjoshi10]

yeah I think that was the issue and not the data files. Thanks for pointing to the direction! It was my first time working with docker and didn't realize I had to mount the path.

[lnnbig]

While I explicitly set “sel” to [200, 400] in the input file for training for H2O systems, there is information “training data with max nbor size: [112, 209]” in train.log.
It seems that the deepmd code [version 2.0.1] will automatically select the values. Am I correct to think that the "sel" setting by users in the input is not important?

[njzjz]

It will be automatically set only if you set it to auto.

[lnnbig]

I checked the manual which says that "--skip-neighbor-stat Skip calculating neighbor statistics. Sel checking, automatic sel, and model compression will be disabled. (default: False)."

If the command --skip-neighbor-stat is not used, automatic sel will be invoked. This explains why I get the information “training data with max nbor size: [112, 209]” despite that I set sel to [200, 400] in input.

[njzjz]

This is sel checking but not automatic sel.

[asedova]

Can the above answer be elaborated a bit? I am not sure how to understand the explanation.

[njzjz]

Neighbor stating has three uses: (1) compute min_nbor_dist for model compression; (2) if one sets a custom sel, check if it is enough; (3) if one sets sel to auto, fill it with the recommended sel.

This means, to use the automatic sel, one has to both enable neighbor stating and set sel to auto.

Currently, the GPU support for neighbor stating is lacking.