Enable Tensorboard Debugger

[AnuragKr]

Hello,

I am getting a run time error regarding shape after making some changes in the code. As I am not getting errors during compile time, it is becoming challenging to solve. After going through the documentation I found a way to enable the Tensorboard debugger --

tf.debugging.experimental.enable_dump_debug_info('debug_log', tensor_debug_mode="FULL_HEALTH", circular_buffer_size=-1)

I added the above line in the env.py file but I am getting an error.

##################################################
Traceback (most recent call last):
File "/home/anurag1/venv/deepmd_env/bin/dp", line 8, in
sys.exit(main())
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/deepmd/entrypoints/main.py", line 580, in main
train_dp(**dict_args)
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/deepmd/entrypoints/train.py", line 121, in train
_do_work(jdata, run_opt, is_compress)
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/deepmd/entrypoints/train.py", line 198, in _do_work
model.build(train_data, stop_batch, origin_type_map=origin_type_map)
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/deepmd/train/trainer.py", line 445, in build
self._build_network(data, suffix)
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/deepmd/train/trainer.py", line 472, in build_network
= self.model.build (self.place_holders['coord'],
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/deepmd/model/ener.py", line 178, in build
= self.descrpt.build(coord,
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/deepmd/descriptor/se_a.py", line 472, in build
self.t_avg = tf.get_variable('t_avg',
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 1616, in get_variable
return get_variable_scope().get_variable(
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 1326, in get_variable
return var_store.get_variable(
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 582, in get_variable
return _true_getter(
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 535, in _true_getter
return self._get_single_variable(
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 956, in _get_single_variable
v = variables.VariableV1(
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/anurag1/venv/deepmd_env/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 444, in getattr
self.getattribute(name)
AttributeError: 'Tensor' object has no attribute 'assign'
INFO:tensorflow:Disabled dumping callback in thread MainThread (dump root: debug_log)
DEEPMD INFO Disabled dumping callback in thread MainThread (dump root: debug_log)

######################################################################

Please suggest How to fix this issue or could you please suggest some other ways of debugging computational graph errors?

Thanks