Skip to content

Commit 2cee01a

Browse files
SolenoidWGTSolenoidWGT
committed
fallback unit_test setting
1 parent 5cfc2fb commit 2cee01a

File tree

8 files changed

+21
-104
lines changed

8 files changed

+21
-104
lines changed

‎.github/workflows/unit_test.yml‎

Lines changed: 9 additions & 76 deletions
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,12 @@ jobs:
1111
if: "!contains(github.event.head_commit.message, 'ci skip')"
1212
strategy:
1313
matrix:
14-
python-version: ["3.7", "3.8", "3.9"]
14+
python-version: [3.7, 3.8, 3.9]
15+
1516
steps:
16-
- uses: actions/checkout@v3
17+
- uses: actions/checkout@v2
1718
- name: Set up Python ${{ matrix.python-version }}
18-
uses: actions/setup-python@v3
19+
uses: actions/setup-python@v2
1920
with:
2021
python-version: ${{ matrix.python-version }}
2122
- name: do_unittest
@@ -40,85 +41,17 @@ jobs:
4041
if: "!contains(github.event.head_commit.message, 'ci skip')"
4142
strategy:
4243
matrix:
43-
python-version: ["3.7", "3.8", "3.9"]
44-
steps:
45-
- uses: actions/checkout@v3
46-
- name: Set up Python ${{ matrix.python-version }}
47-
uses: actions/setup-python@v3
48-
env:
49-
AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache
50-
with:
51-
python-version: ${{ matrix.python-version }}
52-
- name: do_benchmark
53-
run: |
54-
python -m pip install .
55-
python -m pip install ".[test,k8s]"
56-
./ding/scripts/install-k8s-tools.sh
57-
make benchmark
58-
59-
test_multiprocess:
60-
runs-on: self-hosted
61-
if: "!contains(github.event.head_commit.message, 'ci skip')"
62-
strategy:
63-
matrix:
64-
python-version: ["3.7", "3.8", "3.9"]
65-
steps:
66-
- uses: actions/checkout@v3
67-
- name: Set up Python ${{ matrix.python-version }}
68-
uses: actions/setup-python@v3
69-
with:
70-
python-version: ${{ matrix.python-version }}
71-
- name: do_multiprocesstest
72-
timeout-minutes: 40
73-
run: |
74-
python -m pip install box2d-py
75-
python -m pip install .
76-
python -m pip install ".[test,k8s]"
77-
./ding/scripts/install-k8s-tools.sh
78-
make multiprocesstest
79-
80-
test_cuda:
81-
runs-on: self-hosted
82-
if: "!contains(github.event.head_commit.message, 'ci skip')"
83-
strategy:
84-
matrix:
85-
python-version: ["3.7", "3.8", "3.9"]
86-
steps:
87-
- uses: actions/checkout@v3
88-
- name: Set up Python ${{ matrix.python-version }}
89-
uses: actions/setup-python@v3
90-
env:
91-
AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache
92-
with:
93-
python-version: ${{ matrix.python-version }}
94-
- name: do_unittest
95-
timeout-minutes: 40
96-
run: |
97-
python -m pip install torch==1.12.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
98-
python -m pip install box2d-py
99-
python -m pip install .
100-
python -m pip install ".[test,k8s]"
101-
./ding/scripts/install-k8s-tools.sh
102-
make cudatest
44+
python-version: [3.7, 3.8, 3.9]
10345

104-
test_mq_benchmark:
105-
runs-on: self-hosted
106-
if: "!contains(github.event.head_commit.message, 'ci skip')"
107-
strategy:
108-
matrix:
109-
python-version: ["3.7", "3.8", "3.9"]
11046
steps:
111-
- uses: actions/checkout@v3
47+
- uses: actions/checkout@v2
11248
- name: Set up Python ${{ matrix.python-version }}
113-
uses: actions/setup-python@v3
114-
env:
115-
AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache
49+
uses: actions/setup-python@v2
11650
with:
11751
python-version: ${{ matrix.python-version }}
118-
- name: do_mqbenchmark
52+
- name: do_benchmark
11953
run: |
120-
python -m pip install torch==1.12.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
12154
python -m pip install .
12255
python -m pip install ".[test,k8s]"
12356
./ding/scripts/install-k8s-tools.sh
124-
make mqbenchmark
57+
make benchmark

‎Makefile‎

Lines changed: 2 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -57,29 +57,15 @@ benchmark:
5757
--durations=0 \
5858
-sv -m benchmark
5959

60-
multiprocesstest:
61-
pytest ${TEST_DIR} \
62-
--cov-report=xml \
63-
--cov-report term-missing \
64-
--cov=${COV_DIR} \
65-
${DURATIONS_COMMAND} \
66-
${WORKERS_COMMAND} \
67-
-sv -m multiprocesstest
68-
69-
mqbenchmark:
70-
pytest ${TEST_DIR} \
71-
--durations=0 \
72-
-sv -m mqbenchmark
73-
7460
test: unittest # just for compatibility, can be changed later
7561

7662
cpu_test: unittest algotest benchmark
7763

78-
all_test: unittest algotest cudatest benchmark multiprocesstest
64+
all_test: unittest algotest cudatest benchmark
7965

8066
format:
8167
yapf --in-place --recursive -p --verbose --style .style.yapf ${FORMAT_DIR}
8268
format_test:
8369
bash format.sh ${FORMAT_DIR} --test
8470
flake_check:
85-
flake8 ${FORMAT_DIR}
71+
flake8 ${FORMAT_DIR}

‎ding/data/tests/test_shm_buffer.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ def test_shm_buffer():
4848

4949
@pytest.mark.benchmark
5050
@pytest.mark.cudatest
51-
@pytest.mark.multiprocesstest
51+
# @pytest.mark.multiprocesstest
5252
def test_cuda_shm():
5353
if torch.cuda.is_available() and torch.cuda.device_count() >= 2:
5454
import torch.multiprocessing as mp

‎ding/framework/message_queue/perfs/tests/test_perf_nng.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
import pytest
44

55

6-
@pytest.mark.mqbenchmark
7-
@pytest.mark.multiprocesstest
6+
@pytest.mark.benchmark
7+
# @pytest.mark.multiprocesstest
88
def test_nng():
99
params = [
1010
("12376", None, "127.0.0.1", "learner", "0"), ("12378", "tcp://127.0.0.1:12376", "127.0.0.1", "collector", "1")

‎ding/framework/message_queue/perfs/tests/test_perf_shm.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,15 @@
66

77
@pytest.mark.mqbenchmark
88
@pytest.mark.cudatest
9-
@pytest.mark.multiprocesstest
9+
# @pytest.mark.multiprocesstest
1010
def test_shm_numpy_shm():
1111
if torch.cuda.is_available():
1212
shm_perf_main("shm")
1313

1414

1515
@pytest.mark.mqbenchmark
1616
@pytest.mark.cudatest
17-
@pytest.mark.multiprocesstest
17+
# @pytest.mark.multiprocesstest
1818
def test_shm_cuda_shared_tensor():
1919
if torch.cuda.is_available() and torch.cuda.device_count() >= 2:
2020
shm_perf_main("cuda_ipc")

‎ding/framework/message_queue/perfs/tests/test_perf_torchrpc_nccl.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@
66
import platform
77

88

9-
@pytest.mark.mqbenchmark
9+
@pytest.mark.benchmark
1010
@pytest.mark.cudatest
11-
@pytest.mark.multiprocesstest
11+
# @pytest.mark.multiprocesstest
1212
def test_perf_torchrpc_nccl():
1313
if platform.system().lower() != 'windows' and torch.cuda.is_available():
1414
if torch_ge_1121() and torch.cuda.device_count() >= 2:

‎ding/framework/message_queue/tests/test_torch_rpc.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -200,7 +200,7 @@ def torchrpc_args_parser(rank):
200200
logging.debug("[Pass] 6. Set n_parallel_workers > 1.")
201201

202202

203-
@pytest.mark.multiprocesstest
203+
@pytest.mark.unittest
204204
def test_torchrpc():
205205
ctx = get_context("spawn")
206206
if platform.system().lower() != 'windows' and torch_ge_1121():
@@ -209,7 +209,7 @@ def test_torchrpc():
209209

210210

211211
@pytest.mark.cudatest
212-
@pytest.mark.multiprocesstest
212+
@pytest.mark.unittest
213213
def test_torchrpc_cuda():
214214
if platform.system().lower() != 'windows':
215215
if torch_ge_1121() and torch.cuda.is_available() and torch.cuda.device_count() >= 2:
@@ -219,7 +219,7 @@ def test_torchrpc_cuda():
219219

220220

221221
@pytest.mark.cudatest
222-
@pytest.mark.multiprocesstest
222+
@pytest.mark.unittest
223223
def test_torchrpc_parser():
224224
if platform.system().lower() != 'windows' and torch_ge_1121() and torch.cuda.is_available():
225225
ctx = get_context("spawn")

‎pytest.ini‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,5 @@ markers =
1010
envpooltest
1111
other
1212
tmp
13-
multiprocesstest
14-
mqbenchmark
1513

1614
norecursedirs = ding/hpc_rl/tests

0 commit comments

Comments
 (0)