Get errors when running at 26 #3
Description
Hello Shashank Kotyan,
Nice work and helpful code for me!
It seems that some error happens when run run_evolution.py
I got AttributeError: 'dict' object has no attribute 'graph', when the algorithm goes at GPU 2 running 26
Thanks if you can give some advices.
there is the log:
(ras) root@iZuf63vsuvldhpds99q9crZ:~/project/RobustArchitectureSearch# python -u run_evolution.py --use_adaptive_mutations --use_augmentation --use_limited_data
Namespace(dataset=2, epochs=50, log_dir='logs', num_mutated_child=2, num_mutations=5, population_dir='population', population_size=25, test=True, use_adaptive_mutations=True, use_augmentation=True, use_cycles=True, use_limited_data=True, use_non_squares=False, use_random_params=False)
Searching for previous geneartions
Did Not Found Any Last Generation
GPU 2 running 1
GPU 2 running 2
GPU 2 running 3
GPU 2 running 4
GPU 2 running 5
GPU 2 running 6
GPU 2 running 7
GPU 2 running 8
GPU 2 running 9
GPU 2 running 10
GPU 2 running 11
GPU 2 running 12
GPU 2 running 13
GPU 2 running 14
GPU 2 running 15
GPU 2 running 16
GPU 2 running 17
GPU 2 running 18
GPU 2 running 19
GPU 2 running 20
GPU 2 running 21
GPU 2 running 22
GPU 2 running 23
GPU 2 running 24
GPU 2 running 25
GPU 2 running 26
Traceback (most recent call last):
File "run_evolution.py", line 315, in <module>
NeuroEvolution(args).run()
File "run_evolution.py", line 127, in run
self.evolve()
File "run_evolution.py", line 251, in evolve
self.evolve_ras()
File "run_evolution.py", line 199, in evolve_ras
self.parallel_run(1, [0, parent_dna, self.num_mutations, child_individual, self.population, self.store_individual, self.gen])
File "run_evolution.py", line 62, in parallel_run
if self.jobs[deviceID].exception is not None: raise Exception(f"{self.jobs[deviceID].exception[0]}, {self.jobs[deviceID].exception[1]}")
Exception: 'dict' object has no attribute 'graph', Traceback (most recent call last):
File "/root/project/RobustArchitectureSearch/process.py", line 26, in run
mp.Process.run(self)
File "/root/anaconda3/envs/ras/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/root/project/RobustArchitectureSearch/worker.py", line 156, in create_child
while test_dna is None: mutation, test_dna = population.mutate(dna)
File "<string>", line 2, in mutate
File "/root/anaconda3/envs/ras/lib/python3.7/multiprocessing/managers.py", line 834, in _callmethod
raise convert_to_error(kind, result)
AttributeError: 'dict' object has no attribute 'graph'