Conversation
|
Hello @Nunkyl! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:
Comment last updated at 2024-03-22 11:33:01 UTC |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #268 +/- ##
==========================================
- Coverage 72.88% 72.76% -0.12%
==========================================
Files 140 140
Lines 8338 8409 +71
==========================================
+ Hits 6077 6119 +42
- Misses 2261 2290 +29 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
насчет сохранения каждые 60 секунд -- будто бы время это не оптимальный критерий для определения интервала сохранения, лучше делать это в поколениях. Например, каждое или каждое пятое. Можно делать выбор исходя из времени, затраченного на первое поколение, -- если оно большое, то сохранять каждое последующее, если нет -- каждое N |
| graph_generation_params: GraphGenerationParams, | ||
| graph_optimizer_params: GPAlgorithmParameters, | ||
| use_saved_state: bool = False, | ||
| saved_state_path: str = 'saved_optimisation_state/main/evo_graph_optimiser', |
There was a problem hiding this comment.
'saved_optimisation_state/main' повторяется, можно вынести в отдельную константу. и вообще все строки
| if os.path.isfile(saved_state_file): | ||
| current_saved_state_path = saved_state_file | ||
| else: | ||
| raise SystemExit('ERROR: Could not restore saved optimisation state: ' |
There was a problem hiding this comment.
можно наверно просто писать лог, мол начать с сохраненного состояния не удалось, оптимизация начинается с нуля
| saved_state_path, **custom_optimizer_params) | ||
|
|
||
| # Restore state from previous run | ||
| if use_saved_state: |
There was a problem hiding this comment.
лучше вынести всю restore optimisation related логику в отдельный приватный метод
| break | ||
| # Adding of new population to history | ||
| self._update_population(new_population) | ||
| delta = datetime.now() - last_write_time |
There was a problem hiding this comment.
отдельный метод, код будет более читаемым
| if self.use_saved_state: | ||
| bar = tqdm(total=self.requirements.num_of_generations, desc='Generations', unit='gen', | ||
| initial=self.current_generation_num - 2) | ||
| else: | ||
| bar = tqdm(total=self.requirements.num_of_generations, desc='Generations', unit='gen', initial=0) |
There was a problem hiding this comment.
У этого класса нет атрибута self.current_generation_num и self.use_saved_state. Можно не менять код этой функции, а в PopulationalOptimizer в optimize сетить нужное значение. Что-то типа:
pbar.n = self.current_generation_num
pbar.refresh()
| assert time1 > 2 | ||
| assert time2 < 1 |
There was a problem hiding this comment.
Эта проверка не совсем ясна - почему именно 1 и 2?
| initial_population = [generate_labeled_graph('tree', 5, node_types) for _ in range(10)] | ||
|
|
||
| # Setup optimization parameters | ||
| requirements_run_1 = GraphRequirements(timeout=timedelta(minutes=timeout), |
There was a problem hiding this comment.
Действительно ли нужно создавать два отдельных объекта?
В идеале, наверное, сохранять фиксированную небольшую долю времени, затрачиваемого на сохранение. |
6 participants
Implements the optimisation restart feature. During optimisation the state of the optimiser object will be continuously saved to disk. If optimisation fails at some point it will be possible to continue the optimisation process starting with the last saved configuration, instead of starting over. Right now the function was implemented for
PopulationalOptimizerandEvoGraphOptimizerclassesImplementation details
The optimiser object's state will automatically always get saved to a .pkl file
The path is
/default_data_dir/saved_state_path/run_id/timestamp.pkl, wheredefault_data_dir- default_data_dir() from golem.core.pathssaved_state_path- folder path insidedefault_data_dir(its default value can be changed by the user)run_id- a unique id generated for every runtimestamp- the timestamp of when the file is written to diskBy default the state will be saved every 60 seconds. This can be changed in the
save_state_deltaparameter inoptimise()To restore saved state -> while creating an optimiser object (i.e. EvoGraphOptimizer) set class parameter
use_saved_stateto True and specify the path to the file if necessary (otherwise the last available file will be used)New params in the optimiser class:
use_saved_state[optional] true or falsesaved_state_path[optional] path to location of files (string)saved_state_file[optional] path to a specific file that will be used for restoration (string)New parameter in the optimise() function:
save_state_delta[optional] the amount of seconds to wait before saving the next file with the state (int), default = 60Examples:
Important:
While the saved state is restored all settings for the optimisation will be taken from the saved state file except for two:
It is possible to change their values for the second run of the optimiser
A new run using saved state will write data to the same folder the saved state file is in