-
Notifications
You must be signed in to change notification settings - Fork 79
[RedSuns Review] Add progress visualization feature #1663
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
kkawa14
merged 15 commits into
SonySemiconductorSolutions:feature_progress_visualization
from
kkawa14:progress_visual_dev
Mar 4, 2026
+956
−36
Merged
Changes from 11 commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
39d57b2
add progress_bar files and unit test files
kkawa14 fc19cd0
modify progres_info_controller.py and unit test code
kkawa14 34ce950
add blank
kkawa14 30d8527
remove method info
kkawa14 9cabd01
small change
kkawa14 ad03f10
add progress controll instance and description
kkawa14 e807154
adding initialize type check
kkawa14 1b2a26b
Merge branch 'main' into progress_visual_dev
kkawa14 094d974
add blank
kkawa14 84cee43
fixed sensitivity_evaluator unit test
kkawa14 47288aa
add e2e test
kkawa14 84b2f0b
modify test id name
kkawa14 b823629
fixes based on review comments
kkawa14 543aa6d
remove unusing modules
kkawa14 e12008b
remove unusing modules
kkawa14 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
14 changes: 14 additions & 0 deletions
14
model_compression_toolkit/core/common/progress_config/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| # Copyright 2026 Sony Semiconductor Solutions, Inc. All rights reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| # ============================================================================== |
24 changes: 24 additions & 0 deletions
24
model_compression_toolkit/core/common/progress_config/constants.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # Copyright 2026 Sony Semiconductor Solutions, Inc. All rights reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| # ============================================================================== | ||
|
|
||
| COMPLETED_COMPONENTS = 'completedComponents' | ||
| TOTAL_COMPONENTS = 'totalComponents' | ||
| CURRENT_COMPONENT = 'currentComponent' | ||
|
|
||
| PROGRESS_INFO_CALLBACK = 'progress_info_callback' | ||
| TOTAL_STEP = 'total_step' | ||
|
|
||
| PROGRESS_BAR_POSITION = 2 | ||
| DEFAULT_TOTAL_STEP = 4 |
161 changes: 161 additions & 0 deletions
161
model_compression_toolkit/core/common/progress_config/progress_info_controller.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,161 @@ | ||
|
|
||
| # Copyright 2026 Sony Semiconductor Solutions, Inc. All rights reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| # ============================================================================== | ||
|
|
||
| from typing import Optional, Callable, TYPE_CHECKING | ||
| from dataclasses import dataclass, field | ||
| from tqdm import tqdm | ||
|
|
||
| from model_compression_toolkit.core.common.progress_config.constants import ( | ||
| COMPLETED_COMPONENTS, TOTAL_COMPONENTS, CURRENT_COMPONENT, | ||
| PROGRESS_BAR_POSITION, PROGRESS_INFO_CALLBACK, TOTAL_STEP, DEFAULT_TOTAL_STEP | ||
| ) | ||
|
|
||
| if TYPE_CHECKING: # pragma: no cover | ||
| from model_compression_toolkit.core import CoreConfig | ||
| from model_compression_toolkit.gptq.common.gptq_config import GradientPTQConfig | ||
| from model_compression_toolkit.core.common.mixed_precision.resource_utilization_tools.resource_utilization import ResourceUtilization | ||
|
|
||
|
|
||
| @dataclass | ||
| class ProgressInfoController: | ||
| """ | ||
| A unified progress bar controller class. | ||
| Support single progress bar. | ||
|
|
||
| Attributes: | ||
| total_step: Total number of processing steps. | ||
| description: Description for the progress bar. | ||
| current_step: Current step number (starts from 0, incremented by set_description()). | ||
| callback: User-defined callback function. | ||
| """ | ||
| total_step: int = field(default=0) | ||
| current_step: int = field(default=0) | ||
| description: str = field(default="Model Compression Toolkit Progress Infomation") | ||
| progress_info_callback: Optional[Callable] = field(default=None) | ||
|
|
||
| def __new__(cls, *args, **kwargs): | ||
| """ | ||
| Create or skip instantiation based on the enable flag. | ||
| Returns None when progress display should be disabled. | ||
| """ | ||
| progress_info_callback = kwargs.get(PROGRESS_INFO_CALLBACK) | ||
| total_step = kwargs.get(TOTAL_STEP) | ||
|
|
||
| if progress_info_callback is None or total_step <= 0: | ||
| return None | ||
|
|
||
| if not callable(progress_info_callback): | ||
| raise TypeError(f"{PROGRESS_INFO_CALLBACK} must be a callable (function or callable instance).") | ||
|
|
||
| return super().__new__(cls) | ||
|
|
||
| def __post_init__(self): | ||
| """Create progress bar after initialization.""" | ||
| # Initial single bar mode (position=0, top of screen) | ||
| self.pbar = tqdm( | ||
| total=self.total_step, | ||
| desc=self.description, | ||
| position=PROGRESS_BAR_POSITION, | ||
| leave=False, | ||
| unit='step', | ||
| dynamic_ncols=True, | ||
| bar_format='{l_bar}{bar:}|' | ||
| ) | ||
|
|
||
| def set_description(self, description: str): | ||
| """ | ||
| Update progress bar description. | ||
| Automatically increments step number each time set_description is called, | ||
| displaying in "Step X/Y: ..." format. | ||
|
|
||
| Args: | ||
| description: New description text ("Step X/Y: " is automatically added). | ||
| """ | ||
| self.description = description | ||
| self.current_step += 1 | ||
| formatted_description = f"Step {self.current_step}/{self.total_step}: {description}" | ||
|
|
||
| try: | ||
| assert self.current_step <= self.total_step, \ | ||
| f"current_step: {self.current_step}, exceeded total_step: {self.total_step}." | ||
| except AssertionError: | ||
| self.close() | ||
| raise | ||
|
|
||
| self.pbar.set_description(formatted_description, refresh=False) | ||
| self.pbar.update() | ||
|
|
||
| progress_info = { | ||
| COMPLETED_COMPONENTS: description, | ||
| TOTAL_COMPONENTS: self.total_step, | ||
| CURRENT_COMPONENT: self.current_step | ||
| } | ||
| self.progress_info_callback(progress_info) | ||
|
|
||
| def close(self): | ||
| """Close progress bar.""" | ||
| if self.pbar is not None: | ||
| self.pbar.close() | ||
| self.pbar = None | ||
|
|
||
|
|
||
| def research_progress_total( | ||
| core_config: 'CoreConfig', | ||
| target_resource_utilization: 'ResourceUtilization' = None, | ||
| gptq_config: 'GradientPTQConfig' = None, | ||
yt0705 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ) -> int: | ||
yt0705 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| """ | ||
| Check whether specific processing will be executed based on input arguments | ||
| and calculate the total number of processing steps. | ||
|
|
||
| Processing step breakdown: | ||
| 1. Preprocessing (required) | ||
| 2. Statistics calculation (required) | ||
| 3. Weight parameter calculation (required) | ||
| 4. Hessian calculation (when GPTQ or specific settings enabled) | ||
| 5. MP calculation (when Mixed Precision enabled) | ||
| 6. Post-processing ~ conversion to exportable model (required) | ||
|
|
||
| Args: | ||
| core_config: CoreConfig object. | ||
| target_resource_utilization: ResourceUtilization object (used for Mixed Precision determination). | ||
| gptq_config: GPTQ configuration object. | ||
|
|
||
| Returns: | ||
| Total number of processing steps. | ||
| """ | ||
| # Base required steps: preprocessing, statistics, weight params, post-processing | ||
| total_steps = DEFAULT_TOTAL_STEP | ||
|
|
||
| # Add MP calculation step (when Mixed Precision enabled) | ||
| if target_resource_utilization is not None and \ | ||
| target_resource_utilization.is_any_restricted(): | ||
| total_steps += 1 | ||
|
|
||
| # Add Hessian step (when Mixed Precision with Hessian enabled) | ||
| if core_config.mixed_precision_config is not None and \ | ||
| core_config.mixed_precision_config.use_hessian_based_scores: | ||
| total_steps += 1 | ||
|
|
||
| # Add GPTQ training step (when GPTQ is enabled) | ||
| if gptq_config is not None: | ||
| total_steps += 1 | ||
|
|
||
| # Add Hessian step (when GPTQ with Hessian enabled) | ||
| if gptq_config.hessian_weights_config is not None: | ||
| total_steps += 1 | ||
|
|
||
| return total_steps | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.