v0.5.0 by XkunW · Pull Request #78 · VectorInstitute/vector-inference

XkunW · 2025-03-20T20:03:39Z

PR Type

[Release]

Short Description

Decouple model config from repo, the config priority is the following descending order: user-defined config, cached config, default config
Code base refactor, command logic moved to helper classes
Retire launch server bash script, moved server launching logic to python
Update metrics command to use metrics API endpoint
Exposed additional launch parameters for CLI
Automate docker image build process and pushing to dockerhub
Improve the dev build tooling (migrate to uv from poetry)
Reduce maintenance overhead by having dependabot submit security updates and dependency updates as long as they conform to the requirements of the library
Added unit tests for _cli.py, _utils.py, and imports
Add server launch info json to logging
Misc house-keeping fixes
Onboarded various new models (VLM, Reward Modelling models)

Tests Added

test_cli.py:
- test_launch_command_success
- test_launch_command_with_json_output
- test_launch_command_model_not_in_config_with_weights
- test_launch_command_model_not_found
- test_list_all_models
- test_list_single_model
- test_metrics_command_pending_server
- test_metrics_command_server_not_ready
- test_metrics_command_server_ready
- test_metrics_command_request_failed
test_utils.py:
- test_run_bash_command_success
- test_run_bash_command_error
- test_read_slurm_log_found
- test_read_slurm_log_not_found
- test_is_server_running_statuses
- test_get_base_url_found
- test_get_base_url_not_found
- test_model_health_check
- test_model_health_check_request_exception
- test_create_table_with_header
- test_create_table_without_header
- test_load_config_default_only
- test_load_config_with_user_override
- test_load_config_invalid_user_model
test_imports.py:
- test_import_cli_modules

Release v0.4.1

Bumps [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) from 1.4.2 to 1.12.4. - [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases) - [Commits](pypa/gh-action-pypi-publish@27b3170...76f52bc) --- updated-dependencies: - dependency-name: pypa/gh-action-pypi-publish dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.0.0 to 5.4.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5...v5.4.0) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>

Bumps [pypa/gh-action-pip-audit](https://github.com/pypa/gh-action-pip-audit) from 1.0.8 to 1.1.0. - [Release notes](https://github.com/pypa/gh-action-pip-audit/releases) - [Commits](pypa/gh-action-pip-audit@v1.0.8...v1.1.0) --- updated-dependencies: - dependency-name: pypa/gh-action-pip-audit dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>

Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.1 to 4.2.2. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4.1.1...v4.2.2) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>

…pypi-publish-1.12.4

…pip-audit-1.1.0

…ython-5.4.0

…/pypa/gh-action-pypi-publish-1.12.4 Bump pypa/gh-action-pypi-publish from 1.4.2 to 1.12.4

…t-4.2.2

…/pypa/gh-action-pip-audit-1.1.0 Bump pypa/gh-action-pip-audit from 1.0.8 to 1.1.0

…/actions/setup-python-5.4.0 Bump actions/setup-python from 5.0.0 to 5.4.0

…/actions/checkout-4.2.2 Bump actions/checkout from 4.1.1 to 4.2.2

…ctor-inference into decouple_model_config

for more information, see https://pre-commit.ci

… models, remove default fields in model config, fixed a wrong model family name

for more information, see https://pre-commit.ci

…ector-inference into nmb_vlm_models

…t ready

for more information, see https://pre-commit.ci

…ector-inference into nmb_vlm_models

* Onboard new VLM models * Add prefix cache hit rate metric to metrics command

…ute/vector-inference into feature/update-docs

for more information, see https://pre-commit.ci

…ute/vector-inference into feature/update-docs

Update docs

jwilles

LGTM.

amrit110 and others added 30 commits February 14, 2025 15:07

Merge pull request #36 from VectorInstitute/main

8d80a1a

Release v0.4.1

Merge branch 'develop' into dependabot/github_actions/pypa/gh-action-…

9b00440

…pypi-publish-1.12.4

Merge branch 'develop' into dependabot/github_actions/pypa/gh-action-…

4e9f2e7

…pip-audit-1.1.0

Merge branch 'develop' into dependabot/github_actions/actions/setup-p…

fcede66

…ython-5.4.0

Merge pull request #38 from VectorInstitute/dependabot/github_actions…

4c82c7a

…/pypa/gh-action-pypi-publish-1.12.4 Bump pypa/gh-action-pypi-publish from 1.4.2 to 1.12.4

Merge branch 'develop' into dependabot/github_actions/actions/checkou…

86beee3

…t-4.2.2

Merge pull request #40 from VectorInstitute/dependabot/github_actions…

f9d87d2

…/pypa/gh-action-pip-audit-1.1.0 Bump pypa/gh-action-pip-audit from 1.0.8 to 1.1.0

Merge pull request #39 from VectorInstitute/dependabot/github_actions…

63c06e8

…/actions/setup-python-5.4.0 Bump actions/setup-python from 5.0.0 to 5.4.0

Merge pull request #41 from VectorInstitute/dependabot/github_actions…

75f93dc

…/actions/checkout-4.2.2 Bump actions/checkout from 4.1.1 to 4.2.2

Decouple model config, allow user config

9572499

Remove unused pytest fixtures

11238ca

Fix issues in dev env

e012d21

Fix indentation in pyproject.toml

fc213e8

Fix documentation

00da3ce

Fix bugs for coloring, remove deepseek r1 plain

4d2d5d8

Revert the click options with help

d120c61

Small fix with underscore

068eb21

Fix bugs

a9c52ba

Merge branch 'decouple_model_config' of github.com:VectorInstitute/ve…

88661b7

…ctor-inference into decouple_model_config

[pre-commit.ci] Add auto fixes from pre-commit.com hooks

f0f1c7c

for more information, see https://pre-commit.ci

Lint fixes

3fdf472

Fix small bug

6e45a88

Remove defaults from click option

e0e7330

Increase limit on context

26a4dde

Update readme and user guide in docs

8419698

[pre-commit.ci] Add auto fixes from pre-commit.com hooks

baa3142

for more information, see https://pre-commit.ci

XkunW and others added 25 commits March 19, 2025 15:41

Update new VLM model config to the latest format, add in deepseek vl2…

b6e2b4b

… models, remove default fields in model config, fixed a wrong model family name

Remove 1, 2 for compilation config as they are for internal testing only

bd58a23

Update models README

34ef192

[pre-commit.ci] Add auto fixes from pre-commit.com hooks

91b4328

for more information, see https://pre-commit.ci

Add prefix cache hit rate metrics if prefix caching is enabled

8b642ae

Merge branch 'nmb_vlm_models' of https://github.com/VectorInstitute/v…

06c3dd0

…ector-inference into nmb_vlm_models

Add error handling for checking prefix caching in case job json is no…

d6f1431

…t ready

Add additional check for default log dir when reading slurm logs

d51af26

Fix test mocking

dae749f

[pre-commit.ci] Add auto fixes from pre-commit.com hooks

03d4cf0

for more information, see https://pre-commit.ci

Fix too many return args issue

182b0cc

[pre-commit.ci] Add auto fixes from pre-commit.com hooks

c331351

for more information, see https://pre-commit.ci

Fix mypy errors

b081625

Merge branch 'nmb_vlm_models' of https://github.com/VectorInstitute/v…

784b6cb

…ector-inference into nmb_vlm_models

Merge pull request #76 from VectorInstitute/nmb_vlm_models

a4aad27

* Onboard new VLM models * Add prefix cache hit rate metric to metrics command

Bump version to 0.5.0

a339b9c

Update chat completions example to print dict object

fb6e205

Update docs

759aa41

Update images in README.md

a2a765a

Update images in user guide

f5c9d91

Merge branch 'feature/update-docs' of https://github.com/VectorInstit…

57a248e

…ute/vector-inference into feature/update-docs

[pre-commit.ci] Add auto fixes from pre-commit.com hooks

f4f2f48

for more information, see https://pre-commit.ci

Update job ID

f963e5a

Merge branch 'feature/update-docs' of https://github.com/VectorInstit…

6c08b33

…ute/vector-inference into feature/update-docs

Merge pull request #77 from VectorInstitute/feature/update-docs

4933fa6

Update docs

XkunW requested review from amrit110 and jwilles March 20, 2025 20:04

amrit110 approved these changes Mar 20, 2025

View reviewed changes

jwilles approved these changes Mar 20, 2025

View reviewed changes

XkunW merged commit 45ef380 into main Mar 20, 2025
12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

v0.5.0#78

v0.5.0#78
XkunW merged 193 commits intomainfrom
develop

XkunW commented Mar 20, 2025

Uh oh!

jwilles left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Comments

Conversation

XkunW commented Mar 20, 2025

PR Type

Short Description

Tests Added

Uh oh!

jwilles left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants