Skip to content

Commit 9909d17

Browse files
yunchuyunchu
authored
Merge pull request #2639 from openvinotoolkit/mergeback/1.4.4rc2
Mergeback 1.4.4rc2 to 1.5.0
2 parents 9f591b6 + 8107372 commit 9909d17

File tree

6 files changed

+156
-48
lines changed

6 files changed

+156
-48
lines changed

‎.github/workflows/publish.yml‎

Lines changed: 15 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: Build and upload to internal PyPI
1+
name: Build and upload to PyPI
22

33
on:
44
workflow_dispatch: # run on request (no need for PR)
@@ -40,22 +40,17 @@ jobs:
4040
name: Publish package
4141
needs: [build_wheels, build_sdist]
4242
environment: pypi
43-
runs-on: [self-hosted, linux, x64, dev]
43+
runs-on: ubuntu-latest
4444
permissions: write-all
4545
steps:
46-
- name: Set up Python
47-
uses: actions/setup-python@v4
48-
with:
49-
python-version: "3.10"
50-
- name: Install dependencies
51-
run: python -m pip install twine
5246
- name: Download artifacts
5347
uses: actions/download-artifact@v3
5448
with:
5549
# unpacks default artifact into dist/
5650
# if `name: artifact` is omitted, the action will create extra parent dir
5751
name: artifact
5852
path: dist
53+
# to determine where to publish the source distribution to PyPI or TestPyPI
5954
- name: Check tag
6055
id: check-tag
6156
uses: actions-ecosystem/action-regex-match@v2
@@ -71,18 +66,15 @@ jobs:
7166
tag: ${{ github.ref }}
7267
overwrite: true
7368
file_glob: true
74-
- name: Check dist contents
75-
run: twine check dist/*
76-
- name: Publish package dist to internal PyPI
77-
run: |
78-
export no_proxy=${{ secrets.PYPI_HOST }}
79-
export REPOSITORY_URL=http://${{ secrets.PYPI_HOST }}:${{ secrets.PYPI_PORT }}
80-
twine upload --verbose --repository-url $REPOSITORY_URL dist/* -u ${{ secrets.PYPI_USER }} -p ${{ secrets.PYPI_PASSWORD }}
81-
- name: Clean up dist
82-
if: ${{ always() }}
83-
run: |
84-
if OUTPUT=$(ls | grep -c dist)
85-
then
86-
echo "Cleaning up dist directory"
87-
rm -r dist
88-
fi
69+
- name: Publish package distributions to PyPI
70+
if: ${{ steps.check-tag.outputs.match != '' }}
71+
uses: pypa/[email protected]
72+
with:
73+
password: ${{ secrets.PYPI_API_TOKEN }}
74+
- name: Publish package distributions to TestPyPI
75+
if: ${{ steps.check-tag.outputs.match == '' }}
76+
uses: pypa/[email protected]
77+
with:
78+
password: ${{ secrets.TESTPYPI_API_TOKEN }}
79+
repository-url: https://test.pypi.org/legacy/
80+
verbose: true

‎.github/workflows/publish_internal.yml‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
name: Build and upload to internal PyPI
2+
3+
on:
4+
workflow_dispatch: # run on request (no need for PR)
5+
6+
jobs:
7+
build_wheels:
8+
name: Build wheels
9+
runs-on: ubuntu-latest
10+
steps:
11+
- name: Checkout
12+
uses: actions/checkout@v3
13+
- name: Build wheels
14+
uses: pypa/[email protected]
15+
- uses: actions/upload-artifact@v3
16+
with:
17+
path: ./wheelhouse/*.whl
18+
19+
build_sdist:
20+
name: Build source distribution
21+
runs-on: ubuntu-latest
22+
steps:
23+
- name: Checkout
24+
uses: actions/checkout@v3
25+
- name: Set up Python 3.10
26+
uses: actions/setup-python@v3
27+
with:
28+
python-version: "3.10"
29+
- name: Install pypa/build
30+
run: python -m pip install build
31+
- name: Build sdist
32+
run: python -m build --sdist
33+
- uses: actions/upload-artifact@v3
34+
with:
35+
path: dist/*.tar.gz
36+
37+
publish_package:
38+
name: Publish package
39+
needs: [build_wheels, build_sdist]
40+
environment: pypi
41+
runs-on: [self-hosted, linux, x64, dev]
42+
permissions: write-all
43+
steps:
44+
- name: Set up Python
45+
uses: actions/setup-python@v4
46+
with:
47+
python-version: "3.10"
48+
- name: Install dependencies
49+
run: python -m pip install twine
50+
- name: Download artifacts
51+
uses: actions/download-artifact@v3
52+
with:
53+
# unpacks default artifact into dist/
54+
# if `name: artifact` is omitted, the action will create extra parent dir
55+
name: artifact
56+
path: dist
57+
- name: Check tag
58+
id: check-tag
59+
uses: actions-ecosystem/action-regex-match@v2
60+
with:
61+
text: ${{ github.ref }}
62+
regex: '^refs/heads/releases/[0-9]+\.[0-9]+\.[0-9]+(\.[0-9]+rc[0-9]+|rc[0-9]+)?$'
63+
- name: Check dist contents
64+
run: twine check dist/*
65+
- name: Publish package dist to internal PyPI
66+
if: ${{ steps.check-tag.outputs.match != '' }}
67+
run: |
68+
export no_proxy=${{ secrets.PYPI_HOST }}
69+
export REPOSITORY_URL=http://${{ secrets.PYPI_HOST }}:${{ secrets.PYPI_PORT }}
70+
twine upload --verbose --repository-url $REPOSITORY_URL dist/* -u ${{ secrets.PYPI_USER }} -p ${{ secrets.PYPI_PASSWORD }}
71+
- name: Publish package distributions to TestPyPI
72+
if: ${{ steps.check-tag.outputs.match == '' }}
73+
run: |
74+
export REPOSITORY_URL=https://test.pypi.org/legacy/
75+
twine upload --verbose --repository-url $REPOSITORY_URL dist/* -u __token__ -p ${{ secrets.TESTPYPI_API_TOKEN }}
76+
- name: Clean up dist
77+
if: ${{ always() }}
78+
run: |
79+
if OUTPUT=$(ls | grep -c dist)
80+
then
81+
echo "Cleaning up dist directory"
82+
rm -r dist
83+
fi

‎CHANGELOG.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,18 +2,18 @@
22

33
All notable changes to this project will be documented in this file.
44

5-
## \[v1.5.0 - unreleased\]
5+
## \[v1.5.0\]
66

77
### New features
88

9-
- Enable configurable confidence threshold for otx eval and export(<https://github.com/openvinotoolkit/training_extensions/pull/2388>)
9+
- Enable configurable confidence threshold for otx eval and export (<https://github.com/openvinotoolkit/training_extensions/pull/2388>)
1010
- Add YOLOX variants as new object detector models (<https://github.com/openvinotoolkit/training_extensions/pull/2402>)
11-
- Enable FeatureVectorHook to support action tasks(<https://github.com/openvinotoolkit/training_extensions/pull/2408>)
11+
- Enable FeatureVectorHook to support action tasks (<https://github.com/openvinotoolkit/training_extensions/pull/2408>)
1212
- Add ONNX metadata to detection, instance segmantation, and segmentation models (<https://github.com/openvinotoolkit/training_extensions/pull/2418>)
13-
- Add a new feature to configure input size(<https://github.com/openvinotoolkit/training_extensions/pull/2420>)
13+
- Add a new feature to configure input size (<https://github.com/openvinotoolkit/training_extensions/pull/2420>)
1414
- Introduce the OTXSampler and AdaptiveRepeatDataHook to achieve faster training at the small data regime (<https://github.com/openvinotoolkit/training_extensions/pull/2428>)
15-
- Add a new object detector Lite-DINO(<https://github.com/openvinotoolkit/training_extensions/pull/2457>)
16-
- Add Semi-SL Mean Teacher algorithm for Instance Segmentation task(<https://github.com/openvinotoolkit/training_extensions/pull/2444>)
15+
- Add a new object detector Lite-DINO (<https://github.com/openvinotoolkit/training_extensions/pull/2457>)
16+
- Add Semi-SL Mean Teacher algorithm for Instance Segmentation task (<https://github.com/openvinotoolkit/training_extensions/pull/2444>)
1717
- Official supports for YOLOX-X, YOLOX-L, YOLOX-S, ResNeXt101-ATSS (<https://github.com/openvinotoolkit/training_extensions/pull/2485>)
1818
- Add new argument to track resource usage in train command (<https://github.com/openvinotoolkit/training_extensions/pull/2500>)
1919
- Add Self-SL for semantic segmentation of SegNext families (<https://github.com/openvinotoolkit/training_extensions/pull/2215>)

‎README.md‎

Lines changed: 21 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,8 @@
55
---
66

77
[Key Features](#key-features) •
8-
[Installation](https://openvinotoolkit.github.io/training_extensions/stable/guide/get_started/installation.html) •
9-
[Documentation](https://openvinotoolkit.github.io/training_extensions/stable/index.html) •
8+
[Installation](https://openvinotoolkit.github.io/training_extensions/1.5.0/guide/get_started/installation.html) •
9+
[Documentation](https://openvinotoolkit.github.io/training_extensions/1.5.0/index.html) •
1010
[License](#license)
1111

1212
[![PyPI](https://img.shields.io/pypi/v/otx)](https://pypi.org/project/otx)
@@ -42,7 +42,7 @@ If you are an experienced user, you can configure your own model based on [torch
4242

4343
Furthermore, OpenVINO™ Training Extensions provides automatic configuration for ease of use.
4444
The framework will analyze your dataset and identify the most suitable model and figure out the best input size setting and other hyper-parameters.
45-
The development team is continuously extending this [Auto-configuration](https://openvinotoolkit.github.io/training_extensions/latest/guide/explanation/additional_features/auto_configuration.html) functionalities to make training as simple as possible so that single CLI command can obtain accurate, efficient and robust models ready to be integrated into your project.
45+
The development team is continuously extending this [Auto-configuration](https://openvinotoolkit.github.io/training_extensions/stable/guide/explanation/additional_features/auto_configuration.html) functionalities to make training as simple as possible so that single CLI command can obtain accurate, efficient and robust models ready to be integrated into your project.
4646

4747
### Key Features
4848

@@ -63,19 +63,19 @@ OpenVINO™ Training Extensions supports the [following learning methods](https:
6363

6464
OpenVINO™ Training Extensions provides the following usability features:
6565

66-
- [Auto-configuration](https://openvinotoolkit.github.io/training_extensions/latest/guide/explanation/additional_features/auto_configuration.html). OpenVINO™ Training Extensions analyzes provided dataset and selects the proper task and model with appropriate input size to provide the best accuracy/speed trade-off. It will also make a random auto-split of your dataset if there is no validation set provided.
66+
- [Auto-configuration](https://openvinotoolkit.github.io/training_extensions/stable/guide/explanation/additional_features/auto_configuration.html). OpenVINO™ Training Extensions analyzes provided dataset and selects the proper task and model with appropriate input size to provide the best accuracy/speed trade-off. It will also make a random auto-split of your dataset if there is no validation set provided.
6767
- [Datumaro](https://openvinotoolkit.github.io/datumaro/stable/index.html) data frontend: OpenVINO™ Training Extensions supports the most common academic field dataset formats for each task. We are constantly working to extend supported formats to give more freedom of datasets format choice.
6868
- **Distributed training** to accelerate the training process when you have multiple GPUs
6969
- **Mixed-precision training** to save GPUs memory and use larger batch sizes
70-
- Integrated, efficient [hyper-parameter optimization module (HPO)](https://openvinotoolkit.github.io/training_extensions/latest/guide/explanation/additional_features/hpo.html). Through dataset proxy and built-in hyper-parameter optimizer, you can get much faster hyper-parameter optimization compared to other off-the-shelf tools. The hyperparameter optimization is dynamically scheduled based on your resource budget.
70+
- Integrated, efficient [hyper-parameter optimization module (HPO)](https://openvinotoolkit.github.io/training_extensions/stable/guide/explanation/additional_features/hpo.html). Through dataset proxy and built-in hyper-parameter optimizer, you can get much faster hyper-parameter optimization compared to other off-the-shelf tools. The hyperparameter optimization is dynamically scheduled based on your resource budget.
7171

7272
---
7373

7474
## Getting Started
7575

7676
### Installation
7777

78-
Please refer to the [installation guide](https://openvinotoolkit.github.io/training_extensions/stable/guide/get_started/installation.html).
78+
Please refer to the [installation guide](https://openvinotoolkit.github.io/training_extensions/1.5.0/guide/get_started/installation.html).
7979

8080
Note: Python 3.8, 3.9 and 3.10 were tested, along with Ubuntu 18.04, 20.04 and 22.04.
8181

@@ -91,22 +91,26 @@ Note: Python 3.8, 3.9 and 3.10 were tested, along with Ubuntu 18.04, 20.04 and 2
9191
- `otx demo` allows one to apply a trained model on the custom data or the online footage from a web camera and see how it will work in a real-life scenario.
9292
- `otx explain` runs explain algorithm on the provided data and outputs images with the saliency maps to show how your model makes predictions.
9393

94-
You can find more details with examples in the [CLI command intro](https://openvinotoolkit.github.io/training_extensions/stable/guide/get_started/cli_commands.html).
94+
You can find more details with examples in the [CLI command intro](https://openvinotoolkit.github.io/training_extensions/1.5.0/guide/get_started/cli_commands.html).
9595

9696
---
9797

9898
## Updates
9999

100-
### v1.4.0 (3Q23)
101-
102-
- Support encrypted dataset training (<https://github.com/openvinotoolkit/training_extensions/pull/2209>)
103-
- Add custom max iou assigner to prevent CPU OOM when large annotations are used (<https://github.com/openvinotoolkit/training_extensions/pull/2228>)
104-
- Auto train type detection for Semi-SL, Self-SL and Incremental: "--train-type" now is optional (<https://github.com/openvinotoolkit/training_extensions/pull/2195>)
105-
- Add per-class XAI saliency maps for Mask R-CNN model (<https://github.com/openvinotoolkit/training_extensions/pull/2227>)
106-
- Add new object detector Deformable DETR (<https://github.com/openvinotoolkit/training_extensions/pull/2249>)
107-
- Add new object detector DINO (<https://github.com/openvinotoolkit/training_extensions/pull/2266>)
108-
- Add new visual prompting task (<https://github.com/openvinotoolkit/training_extensions/pull/2203>, <https://github.com/openvinotoolkit/training_extensions/pull/2274>, <https://github.com/openvinotoolkit/training_extensions/pull/2311>, <https://github.com/openvinotoolkit/training_extensions/pull/2354>, <https://github.com/openvinotoolkit/training_extensions/pull/2318>)
109-
- Add new object detector ResNeXt101-ATSS (<https://github.com/openvinotoolkit/training_extensions/pull/2309>)
100+
### v1.5.0 (4Q23)
101+
102+
- Enable configurable confidence threshold for otx eval and export (<https://github.com/openvinotoolkit/training_extensions/pull/2388>)
103+
- Add YOLOX variants as new object detector models (<https://github.com/openvinotoolkit/training_extensions/pull/2402>)
104+
- Enable FeatureVectorHook to support action tasks (<https://github.com/openvinotoolkit/training_extensions/pull/2408>)
105+
- Add ONNX metadata to detection, instance segmantation, and segmentation models (<https://github.com/openvinotoolkit/training_extensions/pull/2418>)
106+
- Add a new feature to configure input size (<https://github.com/openvinotoolkit/training_extensions/pull/2420>)
107+
- Introduce the OTXSampler and AdaptiveRepeatDataHook to achieve faster training at the small data regime (<https://github.com/openvinotoolkit/training_extensions/pull/2428>)
108+
- Add a new object detector Lite-DINO (<https://github.com/openvinotoolkit/training_extensions/pull/2457>)
109+
- Add Semi-SL Mean Teacher algorithm for Instance Segmentation task (<https://github.com/openvinotoolkit/training_extensions/pull/2444>)
110+
- Official supports for YOLOX-X, YOLOX-L, YOLOX-S, ResNeXt101-ATSS (<https://github.com/openvinotoolkit/training_extensions/pull/2485>)
111+
- Add new argument to track resource usage in train command (<https://github.com/openvinotoolkit/training_extensions/pull/2500>)
112+
- Add Self-SL for semantic segmentation of SegNext families (<https://github.com/openvinotoolkit/training_extensions/pull/2215>)
113+
- Adapt input size automatically based on dataset statistics (<https://github.com/openvinotoolkit/training_extensions/pull/2499>)
110114

111115
### Release History
112116

‎docs/source/guide/release_notes/index.rst‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,35 @@ Releases
44
.. toctree::
55
:maxdepth: 1
66

7+
v1.5.0 (4Q23)
8+
-------------
9+
10+
- Enable configurable confidence threshold for otx eval and export
11+
- Add YOLOX variants as new object detector models
12+
- Enable FeatureVectorHook to support action tasks
13+
- Add ONNX metadata to detection, instance segmantation, and segmentation models
14+
- Add a new feature to configure input size
15+
- Introduce the OTXSampler and AdaptiveRepeatDataHook to achieve faster training at the small data regime
16+
- Add a new object detector Lite-DINO
17+
- Add Semi-SL Mean Teacher algorithm for Instance Segmentation task
18+
- Official supports for YOLOX-X, YOLOX-L, YOLOX-S, ResNeXt101-ATSS
19+
- Add new argument to track resource usage in train command
20+
- Add Self-SL for semantic segmentation of SegNext families
21+
- Adapt input size automatically based on dataset statistics
22+
- Refine input data in-memory caching
23+
- Adapt timeout value of initialization for distributed training
24+
- Optimize data loading by merging load & resize operations w/ caching support for cls/det/iseg/sseg
25+
- Support torch==2.0.1
26+
- Set "Auto" as default input size mode
27+
28+
29+
v1.4.4 (4Q23)
30+
-------------
31+
32+
- Update ModelAPI configuration
33+
- Add Anomaly modelAPI changes
34+
- Update Image numpy access
35+
736
v1.4.3 (4Q23)
837
-------------
938

‎requirements/base.txt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
2-
# Base Algo Requirements. #
2+
# Base Algo Requirements. #
33
natsort==8.1.*
44
prettytable==3.9.*
55
protobuf==3.20.*
66
pyyaml
7-
datumaro==1.5.1rc3
7+
datumaro~=1.5.1rc4
88
psutil==5.9.*
99
scipy==1.10.*
1010
bayesian-optimization==1.4.*

0 commit comments

Comments
 (0)