Skip to content

Releases: NVIDIA-NeMo/Evaluator

NVIDIA NeMo Evaluator Launcher 0.1.71

22 Jan 01:36
eb3fc5a

Choose a tag to compare

ci: Add `release-docs` workflow (#645)

This workflow will be launched manually via the `Actions` tab - it is
called `Release docs`.

When launching it, we can select the branch of the docs we want to
deploy via the dropdown menu. When deploying `main`, we will upload the
docs to the `latest` directory of the webserver. If it's a versioned
tag, it will strip off the version prefix `v` and upload it into the
version folder of the webserver (e.g. `0.2.0`).

---------

Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Marta Stepniewska-Dziubinska <marta-sd@users.noreply.github.com>

NVIDIA NeMo Evaluator 0.1.68

21 Jan 01:36
30692aa

Choose a tag to compare

nemo-evaluator-v0.1.68

docs(readme): add known issues section to readme and fill with keycha…

NVIDIA NeMo Evaluator Launcher 0.1.70

21 Jan 01:36
30692aa

Choose a tag to compare

nemo-evaluator-launcher-v0.1.70

docs(readme): add known issues section to readme and fill with keycha…

NVIDIA NeMo Evaluator 0.1.67

20 Jan 01:35
3eecd59

Choose a tag to compare

nemo-evaluator-v0.1.67

feat: save original user-provided config in experiment's artifacts (#…

NVIDIA NeMo Evaluator Launcher 0.1.69

20 Jan 01:35
3eecd59

Choose a tag to compare

nemo-evaluator-launcher-v0.1.69

feat: save original user-provided config in experiment's artifacts (#…

NVIDIA NeMo Evaluator 0.1.66

19 Jan 01:36
a8ef50a

Choose a tag to compare

nemo-evaluator-v0.1.66

docs: [nemo fw] Docs for TRTLLM checkpoint deployment and evaluation …

NVIDIA NeMo Evaluator Launcher 0.1.68

19 Jan 01:36
a8ef50a

Choose a tag to compare

nemo-evaluator-launcher-v0.1.68

docs: [nemo fw] Docs for TRTLLM checkpoint deployment and evaluation …

NVIDIA NeMo Evaluator 0.1.65

15 Jan 01:35
ecc07bd

Choose a tag to compare

feat: add sandbox logic (#624)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added sandbox backend supporting AWS ECS Fargate infrastructure with
session management.
* Enabled interactive command execution and file upload capabilities
within sandboxes.
* Integrated AWS credential validation and comprehensive error handling.

* **Documentation**
* Added API documentation for the sandbox module and ECS Fargate usage.

* **Tests**
  * Added unit tests for sandbox module behavior and edge cases.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Michal Bien <mbien@nvidia.com>

NVIDIA NeMo Evaluator Launcher 0.1.67

15 Jan 01:35
ecc07bd

Choose a tag to compare

feat: add sandbox logic (#624)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added sandbox backend supporting AWS ECS Fargate infrastructure with
session management.
* Enabled interactive command execution and file upload capabilities
within sandboxes.
* Integrated AWS credential validation and comprehensive error handling.

* **Documentation**
* Added API documentation for the sandbox module and ECS Fargate usage.

* **Tests**
  * Added unit tests for sandbox module behavior and edge cases.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Michal Bien <mbien@nvidia.com>

NVIDIA NeMo Evaluator 0.1.64

14 Jan 01:36
ed2b4b0

Choose a tag to compare

nemo-evaluator-v0.1.64

fix(docs): don't refer to meta/llama-3.1-8b-instruct from build.nvidi…