Skip to content

Conversation

@Samoed
Copy link
Member

@Samoed Samoed commented Mar 30, 2025

No description provided.

@Samoed Samoed requested a review from Copilot March 30, 2025 17:51
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces node validators and search space configurations for various module types while updating the node optimizer to use these new validators.

  • Added new schema for node validation, dynamically generating Pydantic models for module initialization parameters.
  • Updated the node optimizer to validate search spaces using the new SearchSpaceConfig.
  • Adjusted module exports in the modules package to include additional node classes.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
autointent/schemas/node_validation.py Added new validators, model generation and search space configurations.
autointent/nodes/_node_optimizer.py Modified search space validation to use the new SearchSpaceConfig.
autointent/modules/init.py Updated all to export additional module classes.
Comments suppressed due to low confidence (2)

autointent/schemas/node_validation.py:288

  • [nitpick] Consider collecting all errors rather than breaking the loop immediately by using 'continue' instead of 'break' to provide more comprehensive feedback on misconfigurations.
                error_message += f"Unknown node type '{item['node_type']}' at index {i}.\n"

autointent/schemas/node_validation.py:139

  • Wrapping the default value in a list may unintentionally nest list values if the default is already a list. Verify that the intended behavior is preserved when assigning defaults.
            field = Field(default=[param.default]) if param.default is not inspect.Parameter.empty else Field(...)

@Samoed Samoed requested a review from voorhs March 31, 2025 12:25
@Samoed Samoed mentioned this pull request Mar 31, 2025
Copy link
Collaborator

@voorhs voorhs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

покажешь на созвоне пример логов с ошибками?

@Samoed
Copy link
Member Author

Samoed commented Apr 2, 2025

Пример ошибки если ловить ошибку в test_invalid_optimizer_config_wrong_type

E           TypeError: Search space configuration at index 0 dnnc is invalid: 4 validation errors for DNNCScorerInitModel
E           k.list[constrained-int]
E             Input should be a valid list [type=list_type, input_value='wrong_type', input_type=str]
E               For further information visit https://errors.pydantic.dev/2.11/v/list_type
E           k.ParamSpaceInt
E             Input should be a valid dictionary or instance of ParamSpaceInt [type=model_type, input_value='wrong_type', input_type=str]
E               For further information visit https://errors.pydantic.dev/2.11/v/model_type
E           cross_encoder_name
E             Extra inputs are not permitted [type=extra_forbidden, input_value='cross-encoder/ms-marco-MiniLM-L-6-v2', input_type=str]
E               For further information visit https://errors.pydantic.dev/2.11/v/extra_forbidden
E           train_head
E             Extra inputs are not permitted [type=extra_forbidden, input_value='true', input_type=str]
E               For further information visit https://errors.pydantic.dev/2.11/v/extra_forbidden

@voorhs voorhs merged commit 86384cf into dev Apr 4, 2025
25 of 26 checks passed
@voorhs voorhs deleted the get_validation branch April 4, 2025 11:56
Darinochka pushed a commit that referenced this pull request Apr 8, 2025
* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>
voorhs added a commit that referenced this pull request Apr 9, 2025
* Feat/code carbon each node (#175)

* feat: update codecarbon

* feat: update codecarbon

* feat: added codecarbon

* Update optimizer_config.schema.json

* fix: fixed import mypy

* fix: codecarbon package

* fix: only float\integer log

* fix: codecarbon package

* fix: mypy

* fix: test

* fix: delete emissions

* fix: test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* standartize pyproject & speedup tests (#176)

* speedup tests

* fix pyproject

* Update optimizer_config.schema.json

* move optional dependencies

* fixes

* add xdist

* fix ci

* download data from hub in doc

* add caching

* add doc cache

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* add proper `omit` definition for tests coverage report (#179)

* add proper `omit` definition

* Update optimizer_config.schema.json

* exclude tmp from coverage report

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* update makefile

* update bert test

* mypy workaround

* attempt to fix windows permission error

* workaround

---------

Co-authored-by: Darinochka <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>
voorhs added a commit that referenced this pull request Apr 9, 2025
* added main code for saving models

* Update optimizer_config.schema.json

* checker fixes

* Revert "checker fixes"

This reverts commit 6e32eb9.

* Revert "added main code for saving models"

This reverts commit 5637fb8.

* drat main code for new dumper

* ruf fix

* comments

* added code for test dumper

* Check dumper (#182)

* Feat/code carbon each node (#175)

* feat: update codecarbon

* feat: update codecarbon

* feat: added codecarbon

* Update optimizer_config.schema.json

* fix: fixed import mypy

* fix: codecarbon package

* fix: only float\integer log

* fix: codecarbon package

* fix: mypy

* fix: test

* fix: delete emissions

* fix: test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* standartize pyproject & speedup tests (#176)

* speedup tests

* fix pyproject

* Update optimizer_config.schema.json

* move optional dependencies

* fixes

* add xdist

* fix ci

* download data from hub in doc

* add caching

* add doc cache

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* add proper `omit` definition for tests coverage report (#179)

* add proper `omit` definition

* Update optimizer_config.schema.json

* exclude tmp from coverage report

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* update makefile

* update bert test

* mypy workaround

* attempt to fix windows permission error

* workaround

---------

Co-authored-by: Darinochka <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Алексеев Илья <[email protected]>
Co-authored-by: Darinochka <[email protected]>
Co-authored-by: Roman Solomatin <[email protected]>
voorhs added a commit that referenced this pull request Apr 18, 2025
* feat: added crossencoder

* refactor

* feat: added arg similarity

* Update optimizer_config.schema.json

* feat: added tests

* feat: added errors

* fix: scoring test

* fix: description vectors error

* fix: description vectors error

* fix: lint

* fix: test

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* fix: unit tests

* feat: added test for description

* feat: delete encoder_type from the class args

* feat: update assets

* feat: update assets

* fix: fixed test

* Update optimizer_config.schema.json

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>
Co-authored-by: voorhs <[email protected]>
nikiduki pushed a commit that referenced this pull request Apr 21, 2025
* feat: added crossencoder

* refactor

* feat: added arg similarity

* Update optimizer_config.schema.json

* feat: added tests

* feat: added errors

* fix: scoring test

* fix: description vectors error

* fix: description vectors error

* fix: lint

* fix: test

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* fix: unit tests

* feat: added test for description

* feat: delete encoder_type from the class args

* feat: update assets

* feat: update assets

* fix: fixed test

* Update optimizer_config.schema.json

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>
Co-authored-by: voorhs <[email protected]>
Samoed added a commit that referenced this pull request Apr 21, 2025
* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>
voorhs added a commit that referenced this pull request Apr 25, 2025
* Initial commit of  PTuningScorer module

* Added peft (>=0.10.0, <0.15.0) in dependencies

* Implement fit/predict PTuningScorer

* Added PTuningScorer in __init__ file

* Update optimizer_config.schema.json

* Minor fixs

* PGH00

* Refactor clear_cache in fit method

* Refactor typing ignore + remove unnecessary

* Fix fit method status check

* Added test for PTuningScorer

* Fix mypy typing

* Update and fix peft version dependencies

* Fix mypy typing

* Added test in multiclass.yaml, multilabel.yaml

* Update docs strings

* Fix mypy typing

* Added trust_remote_code

* make proper rst reference

* Added test for dump lod

* feat: added crossencoder (#181)

* feat: added crossencoder

* refactor

* feat: added arg similarity

* Update optimizer_config.schema.json

* feat: added tests

* feat: added errors

* fix: scoring test

* fix: description vectors error

* fix: description vectors error

* fix: lint

* fix: test

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* fix: unit tests

* feat: added test for description

* feat: delete encoder_type from the class args

* feat: update assets

* feat: update assets

* fix: fixed test

* Update optimizer_config.schema.json

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>
Co-authored-by: voorhs <[email protected]>

* Added fixed seed to test reproduction

* Pull LoraScorer and Bert Refactor

* Refactor PTuningScorer

* Refactor test for ptuning

* Fix typing

* Fix multilabel multiclass tests

* Fix typing

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>
Co-authored-by: Darinochka <[email protected]>
Co-authored-by: Roman Solomatin <[email protected]>
voorhs added a commit that referenced this pull request May 3, 2025
* full tuning (#165)

* Added code for full tuning

* work on review

* renaming

* fix ruff

* mypy test

* ignote mypy

* Feat/bert scorer config refactoring (#168)

* refactor configs

* add proper configs to BERTScorer

* fix typing

* fix tokenizer's parameters

* fix transformers and accelerate issue

* Update optimizer_config.schema.json

* bug fix

* update callback test

* fix tests

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* delete validate_task

* report_to

* batches

* Fix/docs building for bert scorer (#171)

* fix

* fix codestyle

---------

Co-authored-by: Алексеев Илья <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* bert-scorer ending (#172)

* batches

* tests check

* fix

* return to torch

* fix for tests

* Fix/bert scorer (#174)

* fix str and float issue and shrinken search space

* update `inference node config` overriding logic

* fix typing

* fix codestyle

* fix multilabel issue

* attempt to fix `inference node config` bugs

* another attempt

---------

Co-authored-by: Алексеев Илья <[email protected]>

* Feat/code carbon each node (#175)

* feat: update codecarbon

* feat: update codecarbon

* feat: added codecarbon

* Update optimizer_config.schema.json

* fix: fixed import mypy

* fix: codecarbon package

* fix: only float\integer log

* fix: codecarbon package

* fix: mypy

* fix: test

* fix: delete emissions

* fix: test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* standartize pyproject & speedup tests (#176)

* speedup tests

* fix pyproject

* Update optimizer_config.schema.json

* move optional dependencies

* fixes

* add xdist

* fix ci

* download data from hub in doc

* add caching

* add doc cache

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* add proper `omit` definition for tests coverage report (#179)

* add proper `omit` definition

* Update optimizer_config.schema.json

* exclude tmp from coverage report

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* dumper saving (#180)

* added main code for saving models

* Update optimizer_config.schema.json

* checker fixes

* Revert "checker fixes"

This reverts commit 6e32eb9.

* Revert "added main code for saving models"

This reverts commit 5637fb8.

* drat main code for new dumper

* ruf fix

* comments

* added code for test dumper

* Check dumper (#182)

* Feat/code carbon each node (#175)

* feat: update codecarbon

* feat: update codecarbon

* feat: added codecarbon

* Update optimizer_config.schema.json

* fix: fixed import mypy

* fix: codecarbon package

* fix: only float\integer log

* fix: codecarbon package

* fix: mypy

* fix: test

* fix: delete emissions

* fix: test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* standartize pyproject & speedup tests (#176)

* speedup tests

* fix pyproject

* Update optimizer_config.schema.json

* move optional dependencies

* fixes

* add xdist

* fix ci

* download data from hub in doc

* add caching

* add doc cache

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* add proper `omit` definition for tests coverage report (#179)

* add proper `omit` definition

* Update optimizer_config.schema.json

* exclude tmp from coverage report

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* update makefile

* update bert test

* mypy workaround

* attempt to fix windows permission error

* workaround

---------

Co-authored-by: Darinochka <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Алексеев Илья <[email protected]>
Co-authored-by: Darinochka <[email protected]>
Co-authored-by: Roman Solomatin <[email protected]>

* Update embedder prompt (#183)

* Add trust remote code (#185)

* lint

* fix trust remote code

* Update optimizer_config.schema.json

* update fix trust remote code

* fix test cllback

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Remove autointent org from docs (#186)

* lint

* update paths

* feat: added crossencoder (#181)

* feat: added crossencoder

* refactor

* feat: added arg similarity

* Update optimizer_config.schema.json

* feat: added tests

* feat: added errors

* fix: scoring test

* fix: description vectors error

* fix: description vectors error

* fix: lint

* fix: test

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* fix: unit tests

* feat: added test for description

* feat: delete encoder_type from the class args

* feat: update assets

* feat: update assets

* fix: fixed test

* Update optimizer_config.schema.json

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>
Co-authored-by: voorhs <[email protected]>

* Add few shot (#187)

* init few shot

* Update optimizer_config.schema.json

* apply few shot to all

* Update optimizer_config.schema.json

* fix test

* lint

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* update numpy typing (#188)

* Lora scorer (#170)

* added lora scorer

* fix ruff

* Update __init__.py

* updated after mr #165

* Update pyproject.toml

* fixed requested changes

* fixed ruff failing

* fixed remarks

* Update optimizer_config.schema.json

* added test

* ruff fix

* convert labels to float

* Update autointent/modules/scoring/_lora/lora.py

Co-authored-by: Roman Solomatin <[email protected]>

* Update autointent/modules/scoring/_lora/lora.py

Co-authored-by: Roman Solomatin <[email protected]>

* change model_config name, added trust_remote_code

* Update lora.py

* inherited lora from bert

* fix ruff

* fix search space

* Update lora.py

* Update lora.py

* added dump check

* Update test_lora.py

* Update test_lora.py

* added docstring

* fix ruff

* Update test_lora.py

* Update test_lora.py

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>

* PTuningScorer (#178)

* Initial commit of  PTuningScorer module

* Added peft (>=0.10.0, <0.15.0) in dependencies

* Implement fit/predict PTuningScorer

* Added PTuningScorer in __init__ file

* Update optimizer_config.schema.json

* Minor fixs

* PGH00

* Refactor clear_cache in fit method

* Refactor typing ignore + remove unnecessary

* Fix fit method status check

* Added test for PTuningScorer

* Fix mypy typing

* Update and fix peft version dependencies

* Fix mypy typing

* Added test in multiclass.yaml, multilabel.yaml

* Update docs strings

* Fix mypy typing

* Added trust_remote_code

* make proper rst reference

* Added test for dump lod

* feat: added crossencoder (#181)

* feat: added crossencoder

* refactor

* feat: added arg similarity

* Update optimizer_config.schema.json

* feat: added tests

* feat: added errors

* fix: scoring test

* fix: description vectors error

* fix: description vectors error

* fix: lint

* fix: test

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* fix: unit tests

* feat: added test for description

* feat: delete encoder_type from the class args

* feat: update assets

* feat: update assets

* fix: fixed test

* Update optimizer_config.schema.json

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>
Co-authored-by: voorhs <[email protected]>

* Added fixed seed to test reproduction

* Pull LoraScorer and Bert Refactor

* Refactor PTuningScorer

* Refactor test for ptuning

* Fix typing

* Fix multilabel multiclass tests

* Fix typing

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>
Co-authored-by: Darinochka <[email protected]>
Co-authored-by: Roman Solomatin <[email protected]>

* Rerank scorer: опция для выбора источника для расчета вектора вероятностей (#115)

* Enable rerank scorer to use crossencoder scores for the probability vector

* add cross encoder scores range options

* upd test

---------

Co-authored-by: voorhs <[email protected]>

* feat: add DISABLE_EMISSIONS_TRACKING (#191)

* feat: add DISABLE_EMISSIONS_TRACKING

* try to fix docs error

* Update optimizer_config.schema.json

* another attempt

* Update optimizer_config.schema.json

* i give up for now

* Update optimizer_config.schema.json

---------

Co-authored-by: voorhs <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* fix issue (#194)

* Refactor/embedding caching (#195)

* implement new hashing strategy

* fix codestyle

* Update optimizer_config.schema.json

* minor bug fix

* fix typing error

* refactor similarity calculation

* Update optimizer_config.schema.json

* upd callback test

* solve 429 error

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* forgot something

---------

Co-authored-by: Сергей Малышев <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Darinochka <[email protected]>
Co-authored-by: Roman Solomatin <[email protected]>
Co-authored-by: VALERIA RUBANOVA <[email protected]>
Co-authored-by: nikiduki <[email protected]>
Co-authored-by: Dmitryv-2024 <[email protected]>
voorhs added a commit that referenced this pull request May 3, 2025
* add interruption handling

* fix test

* fix test

* update

* fix test

* lint

* remove step

* use patch instead of monkeypatch

* add n_jobs as param

* change n_jobs to -1

* try fix

* remove old study

* add logging warning

* Update optimizer_config.schema.json

* lint

* try dumping

* lint

* np encoder

* update warning trigger

* Fix/n trials issue (#196)

* try to fix

* fix typing errors

* bug fix

* Update autointent/nodes/_node_optimizer.py

Co-authored-by: Roman Solomatin <[email protected]>

---------

Co-authored-by: Roman Solomatin <[email protected]>

* Fix/context not dumped error (#197)

* try to fix

* dump context constantly and fix serialization issues

* add exclude option to dumper

* fix codestyle and typing errors

* try to fix file exists error

* fix no fixture found error

* Update interruption handling (#198)

* full tuning (#165)

* Added code for full tuning

* work on review

* renaming

* fix ruff

* mypy test

* ignote mypy

* Feat/bert scorer config refactoring (#168)

* refactor configs

* add proper configs to BERTScorer

* fix typing

* fix tokenizer's parameters

* fix transformers and accelerate issue

* Update optimizer_config.schema.json

* bug fix

* update callback test

* fix tests

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* delete validate_task

* report_to

* batches

* Fix/docs building for bert scorer (#171)

* fix

* fix codestyle

---------

Co-authored-by: Алексеев Илья <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* bert-scorer ending (#172)

* batches

* tests check

* fix

* return to torch

* fix for tests

* Fix/bert scorer (#174)

* fix str and float issue and shrinken search space

* update `inference node config` overriding logic

* fix typing

* fix codestyle

* fix multilabel issue

* attempt to fix `inference node config` bugs

* another attempt

---------

Co-authored-by: Алексеев Илья <[email protected]>

* Feat/code carbon each node (#175)

* feat: update codecarbon

* feat: update codecarbon

* feat: added codecarbon

* Update optimizer_config.schema.json

* fix: fixed import mypy

* fix: codecarbon package

* fix: only float\integer log

* fix: codecarbon package

* fix: mypy

* fix: test

* fix: delete emissions

* fix: test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* standartize pyproject & speedup tests (#176)

* speedup tests

* fix pyproject

* Update optimizer_config.schema.json

* move optional dependencies

* fixes

* add xdist

* fix ci

* download data from hub in doc

* add caching

* add doc cache

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* add proper `omit` definition for tests coverage report (#179)

* add proper `omit` definition

* Update optimizer_config.schema.json

* exclude tmp from coverage report

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* dumper saving (#180)

* added main code for saving models

* Update optimizer_config.schema.json

* checker fixes

* Revert "checker fixes"

This reverts commit 6e32eb9.

* Revert "added main code for saving models"

This reverts commit 5637fb8.

* drat main code for new dumper

* ruf fix

* comments

* added code for test dumper

* Check dumper (#182)

* Feat/code carbon each node (#175)

* feat: update codecarbon

* feat: update codecarbon

* feat: added codecarbon

* Update optimizer_config.schema.json

* fix: fixed import mypy

* fix: codecarbon package

* fix: only float\integer log

* fix: codecarbon package

* fix: mypy

* fix: test

* fix: delete emissions

* fix: test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* standartize pyproject & speedup tests (#176)

* speedup tests

* fix pyproject

* Update optimizer_config.schema.json

* move optional dependencies

* fixes

* add xdist

* fix ci

* download data from hub in doc

* add caching

* add doc cache

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* add proper `omit` definition for tests coverage report (#179)

* add proper `omit` definition

* Update optimizer_config.schema.json

* exclude tmp from coverage report

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* update makefile

* update bert test

* mypy workaround

* attempt to fix windows permission error

* workaround

---------

Co-authored-by: Darinochka <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Алексеев Илья <[email protected]>
Co-authored-by: Darinochka <[email protected]>
Co-authored-by: Roman Solomatin <[email protected]>

* Update embedder prompt (#183)

* Add trust remote code (#185)

* lint

* fix trust remote code

* Update optimizer_config.schema.json

* update fix trust remote code

* fix test cllback

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Remove autointent org from docs (#186)

* lint

* update paths

* feat: added crossencoder (#181)

* feat: added crossencoder

* refactor

* feat: added arg similarity

* Update optimizer_config.schema.json

* feat: added tests

* feat: added errors

* fix: scoring test

* fix: description vectors error

* fix: description vectors error

* fix: lint

* fix: test

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* fix: unit tests

* feat: added test for description

* feat: delete encoder_type from the class args

* feat: update assets

* feat: update assets

* fix: fixed test

* Update optimizer_config.schema.json

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>
Co-authored-by: voorhs <[email protected]>

* Add few shot (#187)

* init few shot

* Update optimizer_config.schema.json

* apply few shot to all

* Update optimizer_config.schema.json

* fix test

* lint

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* update numpy typing (#188)

* Lora scorer (#170)

* added lora scorer

* fix ruff

* Update __init__.py

* updated after mr #165

* Update pyproject.toml

* fixed requested changes

* fixed ruff failing

* fixed remarks

* Update optimizer_config.schema.json

* added test

* ruff fix

* convert labels to float

* Update autointent/modules/scoring/_lora/lora.py

Co-authored-by: Roman Solomatin <[email protected]>

* Update autointent/modules/scoring/_lora/lora.py

Co-authored-by: Roman Solomatin <[email protected]>

* change model_config name, added trust_remote_code

* Update lora.py

* inherited lora from bert

* fix ruff

* fix search space

* Update lora.py

* Update lora.py

* added dump check

* Update test_lora.py

* Update test_lora.py

* added docstring

* fix ruff

* Update test_lora.py

* Update test_lora.py

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>

* PTuningScorer (#178)

* Initial commit of  PTuningScorer module

* Added peft (>=0.10.0, <0.15.0) in dependencies

* Implement fit/predict PTuningScorer

* Added PTuningScorer in __init__ file

* Update optimizer_config.schema.json

* Minor fixs

* PGH00

* Refactor clear_cache in fit method

* Refactor typing ignore + remove unnecessary

* Fix fit method status check

* Added test for PTuningScorer

* Fix mypy typing

* Update and fix peft version dependencies

* Fix mypy typing

* Added test in multiclass.yaml, multilabel.yaml

* Update docs strings

* Fix mypy typing

* Added trust_remote_code

* make proper rst reference

* Added test for dump lod

* feat: added crossencoder (#181)

* feat: added crossencoder

* refactor

* feat: added arg similarity

* Update optimizer_config.schema.json

* feat: added tests

* feat: added errors

* fix: scoring test

* fix: description vectors error

* fix: description vectors error

* fix: lint

* fix: test

* add node validators (#177)

* add node validators

* add comments

* Update optimizer_config.schema.json

* rename bert model

* lint

* fixes

* fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>

* fix: unit tests

* feat: added test for description

* feat: delete encoder_type from the class args

* feat: update assets

* feat: update assets

* fix: fixed test

* Update optimizer_config.schema.json

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Roman Solomatin <[email protected]>
Co-authored-by: voorhs <[email protected]>

* Added fixed seed to test reproduction

* Pull LoraScorer and Bert Refactor

* Refactor PTuningScorer

* Refactor test for ptuning

* Fix typing

* Fix multilabel multiclass tests

* Fix typing

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: voorhs <[email protected]>
Co-authored-by: Darinochka <[email protected]>
Co-authored-by: Roman Solomatin <[email protected]>

* Rerank scorer: опция для выбора источника для расчета вектора вероятностей (#115)

* Enable rerank scorer to use crossencoder scores for the probability vector

* add cross encoder scores range options

* upd test

---------

Co-authored-by: voorhs <[email protected]>

* feat: add DISABLE_EMISSIONS_TRACKING (#191)

* feat: add DISABLE_EMISSIONS_TRACKING

* try to fix docs error

* Update optimizer_config.schema.json

* another attempt

* Update optimizer_config.schema.json

* i give up for now

* Update optimizer_config.schema.json

---------

Co-authored-by: voorhs <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* fix issue (#194)

* Refactor/embedding caching (#195)

* implement new hashing strategy

* fix codestyle

* Update optimizer_config.schema.json

* minor bug fix

* fix typing error

* refactor similarity calculation

* Update optimizer_config.schema.json

* upd callback test

* solve 429 error

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* forgot something

---------

Co-authored-by: Сергей Малышев <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Darinochka <[email protected]>
Co-authored-by: Roman Solomatin <[email protected]>
Co-authored-by: VALERIA RUBANOVA <[email protected]>
Co-authored-by: nikiduki <[email protected]>
Co-authored-by: Dmitryv-2024 <[email protected]>

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Алексеев Илья <[email protected]>
Co-authored-by: Сергей Малышев <[email protected]>
Co-authored-by: Darinochka <[email protected]>
Co-authored-by: VALERIA RUBANOVA <[email protected]>
Co-authored-by: nikiduki <[email protected]>
Co-authored-by: Dmitryv-2024 <[email protected]>
Co-authored-by: voorhs <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants