Skip to content

[Cosmetic] Rename data_args to dataset_args#1206

Merged
dsikka merged 10 commits intomainfrom
datasetargs
Mar 5, 2025
Merged

[Cosmetic] Rename data_args to dataset_args#1206
dsikka merged 10 commits intomainfrom
datasetargs

Conversation

@horheynm
Copy link
Copy Markdown

@horheynm horheynm commented Feb 27, 2025

Order of reviews:
#1206 <-- Here
#1207
#1209
#1212
#1214

SUMMARY:
Rename data_args to dataset_args

TEST PLAN:
Pass tests
FInd data_args using grep

@github-actions
Copy link
Copy Markdown

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

George Ohashi added 2 commits February 28, 2025 10:06
Signed-off-by: George Ohashi <george@neuralmagic.com>
George Ohashi added 3 commits February 28, 2025 17:40
Signed-off-by: George Ohashi <george@neuralmagic.com>
Copy link
Copy Markdown
Collaborator

@brian-dellabetta brian-dellabetta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we do a linter check here or will the tests capture full coverage of the changes?

Copy link
Copy Markdown
Collaborator

@dsikka dsikka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just changing variable names, not the actual class name?

@horheynm
Copy link
Copy Markdown
Author

horheynm commented Mar 3, 2025

Yes

This is just changing variable names, not the actual class name?

dsikka pushed a commit that referenced this pull request Mar 3, 2025
Order of reviews:
#1206
#1207 <-- Here
#1209 
#1212
#1214 

SUMMARY:
* Decouple arg parser to be used for both oneshot and train

TEST PLAN:
* Pass tests
Signed-off-by: George Ohashi <george@neuralmagic.com>
Copy link
Copy Markdown
Collaborator

@brian-dellabetta brian-dellabetta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like some things were changed in a find+replace that shouldn't have been

Signed-off-by: George Ohashi <george@neuralmagic.com>
@dsikka dsikka enabled auto-merge (squash) March 5, 2025 14:55
@dsikka dsikka merged commit 391b202 into main Mar 5, 2025
8 checks passed
@dsikka dsikka deleted the datasetargs branch March 5, 2025 15:27
dsikka pushed a commit that referenced this pull request Mar 5, 2025
Order of reviews:
#1206
#1207
#1209 <-- Here
#1212
#1214 

SUMMARY:
* Move dataset logic out of transformers module
`src/llmcompressor/transformers/finetune/data/data_helpers.py`, add it
to `src/llmcompressor/datasets/utils.py`


TEST PLAN:
Pass tests
dsikka pushed a commit that referenced this pull request Mar 6, 2025
…ot (#1212)

Order of reviews:
#1206
#1207
#1209
#1212  <-- Here
#1214

SUMMARY:
* Move the preprocessing and postprocessing logic out of
`src/llmcompressor/transformers/finetune/text_generation.py` and into
`src/llmcompressor/entrypoints/utils.py`

TEST PLAN:
Pass tests
brian-dellabetta pushed a commit that referenced this pull request Mar 10, 2025
…ot (#1212)

Order of reviews:
#1206
#1207
#1209
#1212  <-- Here
#1214

SUMMARY:
* Move the preprocessing and postprocessing logic out of
`src/llmcompressor/transformers/finetune/text_generation.py` and into
`src/llmcompressor/entrypoints/utils.py`

TEST PLAN:
Pass tests

Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
kylesayrs pushed a commit that referenced this pull request Mar 13, 2025
Order of reviews:
#1206
#1207
#1209
#1212
#1214 <-- Here

SUMMARY:
* Refactor Training pipeline
* Remove initialize, finalize from the session functions
* Add training information on entrypoints/readme.md on the different
types of training that can be carried out on llm-compressor
* Decouple training from text_generation.py::main. The new logic loves
in llmcompressor/entrypoints/train.py that takes the flow of
pre-process, carry out training logic and then post-process
* Delete outdated info on transformers/finetune/readme.md
* Update session_mixin.py to use session().initialize or
session().finalize.
* Deprecate train.py in text_generation.py, raising deprecation message
if used.
* Update tests to use llmcompressor's train, not
llmcompressor.transformers' train

TEST PLAN:
* Pass tests

---------

Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready When a PR is ready for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants