[Enhance] Support select gpu-ids in non-distribute testing time (#6781)

BIGWangYuDong · LJoson · Czm369 · web-flow · commit 5102f2f679bf · 2021-12-24T13:51:59.000+08:00
* Update README_zh-CN.md (#6652) * add abstract and sketch to the CONFIGS/algorithm/README.md (#6654) * Fix mosaic repr typo (#6523) * Include mmflow in readme (#6545) * Include mmflow in readme * Include mmflow in README_zh-CN * Add mmflow url into the document menu in docs/conf.py and docs_zh-CN/conf.py. * Make OHEM work with seesaw loss (#6514) * [Enhance] Support file_client in Datasets and evaluating panoptic results on Ceph (#6489) * first version * Replace with our api * Add copyright * Move the runtime error to multi_core interface * Add docstring * Fix comments * Add comments * Add unit test for pq_compute_single_core * Fix MMDetection model to ONNX command (#6558) * Update README.md (#6567) * [Feature] Support custom persistent_workers (#6435) * Fix aug test error when the number of prediction bboxes is 0 (#6398) * Fix aug test error when the number of prediction bboxes is 0 * test * test * fix lint * Support custom pin_memory and persistent_workers * fix comment * fix docstr * remove pin_memory * Fix SSD512 config error (#6574) * Fix mosaic repr typo (#6523) * Include mmflow in readme (#6545) * Include mmflow in readme * Include mmflow in README_zh-CN * Add mmflow url into the document menu in docs/conf.py and docs_zh-CN/conf.py. * Make OHEM work with seesaw loss (#6514) * Fix ssd512 config error Co-authored-by: Kyungmin Lee <30465912+lkm2835@users.noreply.github.com> Co-authored-by: Czm369 <40661020+Czm369@users.noreply.github.com> Co-authored-by: ohwi <supebulous@gmail.com> * Catch symlink failure on Windows (#6482) * Catch symlink failure on Windows Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * Set copy mode on Windows Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * Fix lint Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * Fix logic error Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * [Feature] Support Label Assignment Distillation (LAD) (#6342) * add LAD * inherit LAD from KnowledgeDistillationSingleStageDetector * add configs/lad/lad_r101_paa_r50_fpn_coco_1x.py * update LAD readme * update configs/lad/README.md * try not to use abbreviations for variable names * add unittest for lad_head * update test_lad_head * remove main in tests/test_models/test_dense_heads/test_lad_head.py * [Fix] Avoid infinite GPU waiting in dist training (#6501) * [#6495] fix infinite GPU waiting in dist training * print log_vars keys in assertion msg * linting issue * Support to collect the best models (#6560) * Fix mosaic repr typo (#6523) * Include mmflow in readme (#6545) * Include mmflow in readme * Include mmflow in README_zh-CN * Add mmflow url into the document menu in docs/conf.py and docs_zh-CN/conf.py. * Make OHEM work with seesaw loss (#6514) * update * support gather best model Co-authored-by: Kyungmin Lee <30465912+lkm2835@users.noreply.github.com> Co-authored-by: Czm369 <40661020+Czm369@users.noreply.github.com> Co-authored-by: ohwi <supebulous@gmail.com> * [Enhance]: Optimize augmentation pipeline to speed up training. (#6442) * Refactor YOLOX (#6443) * Fix aug test error when the number of prediction bboxes is 0 (#6398) * Fix aug test error when the number of prediction bboxes is 0 * test * test * fix lint * Support custom pin_memory and persistent_workers * [Docs] Chinese version of robustness_benchmarking.md (#6375) * Chinese version of robustness_benchmarking.md * Update docs_zh-CN/robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Update docs_zh-CN/robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Update docs_zh-CN/robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Update docs_zh-CN/robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Update docs_zh-CN/robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Update docs_zh-CN/robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Update robustness_benchmarking.md * Update robustness_benchmarking.md * Update robustness_benchmarking.md * Update robustness_benchmarking.md * Update robustness_benchmarking.md * Update robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * update yolox_s * update yolox_s * support dynamic eval interval * fix some error * support ceph * fix none error * fix batch error * replace resize * fix comment * fix docstr * Update the link of checkpoints (#6460) * [Feature]: Support plot confusion matrix. (#6344) * remove pin_memory * update * fix unittest * update cfg * fix error * add unittest * [Fix] Fix SpatialReductionAttention in PVT. (#6488) * [Fix] Fix SpatialReductionAttention in PVT * Add warning * Save coco summarize print information to logger (#6505) * Fix type error in 2_new_data_mode (#6469) * Always map location to cpu when load checkpoint (#6405) * configs: update groie README (#6401) Signed-off-by: Leonardo Rossi <leonardo.rossi@unipr.it> * [Fix] fix config path in docs (#6396) * [Enchance] Set a random seed when the user does not set a seed. (#6457) * fix random seed bug * add comment * enchance random seed * rename Co-authored-by: Haobo Yuan <yuanhaobo@whu.edu.cn> * [BugFixed] fix wrong trunc_normal_init use (#6432) * fix wrong trunc_normal_init use * fix wrong trunc_normal_init use * fix #6446 Co-authored-by: Uno Wu <st9007a@gmail.com> Co-authored-by: Leonardo Rossi <leonardo.rossi@unipr.it> Co-authored-by: BigDong <yudongwang@tju.edu.cn> Co-authored-by: Haian Huang(深度眸) <1286304229@qq.com> Co-authored-by: Haobo Yuan <yuanhaobo@whu.edu.cn> Co-authored-by: Shusheng Yang <shusheng.yang@qq.com> * bump version to v2.18.1 (#6510) * bump version to v2.18.1 * Update changelog.md * add some comment * fix some comment * update readme * fix lint * add reduce mean * update * update readme * update params Co-authored-by: Cedric Luo <luochunhua1996@outlook.com> Co-authored-by: RangiLyu <lyuchqi@gmail.com> Co-authored-by: Guangchen Lin <347630870@qq.com> Co-authored-by: Andrea Panizza <8233615+AndreaPi@users.noreply.github.com> Co-authored-by: Uno Wu <st9007a@gmail.com> Co-authored-by: Leonardo Rossi <leonardo.rossi@unipr.it> Co-authored-by: BigDong <yudongwang@tju.edu.cn> Co-authored-by: Haobo Yuan <yuanhaobo@whu.edu.cn> Co-authored-by: Shusheng Yang <shusheng.yang@qq.com> * [Refactor] Remove some code in `mmdet/apis/train.py` (#6576) * remove some code about custom hooks in apis/train.py * files were modified by yapf * Fix lad repeatedly output warning message (#6584) * drop support for pytorch 1.3.1. (#6589) * drop support for pytorch 1.3.1. * update ci * update ci * Skip CI on ignoring paths (#6578) * [test] Skip CI on ignoring paths Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * [skip ci] fix build.yaml Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * split lint in workflows Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * split lint in workflows Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * split lint in workflows Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * [Refactor]: refactor configs of FP16 models (#6592) * refactor configs of fp16 * update * update * update * Ignoring paths (#6615) Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * fix FP16 CI (#6616) * Bump to 2.19.0 (#6633) * bump to 2.19.0 * resolve comments * fix typo * fix * fix yolox cuda unittest (#6636) * Support circleci (#6630) * add ci * rename * fix mmtrack unittest * update mmcv version * add abstract and sketch to the CONFIGS/algorithm/README.md * add some ## Introduction * revert Co-authored-by: Kyungmin Lee <30465912+lkm2835@users.noreply.github.com> Co-authored-by: ohwi <supebulous@gmail.com> Co-authored-by: Guangchen Lin <347630870@qq.com> Co-authored-by: Rishit Dagli <rishit.dagli@gmail.com> Co-authored-by: RangiLyu <lyuchqi@gmail.com> Co-authored-by: Haian Huang(深度眸) <1286304229@qq.com> Co-authored-by: del-zhenwu <dele.zhenwu@gmail.com> Co-authored-by: Thuy Ng <thuypn9a4@gmail.com> Co-authored-by: Han Zhang <623606860@qq.com> Co-authored-by: Cedric Luo <luochunhua1996@outlook.com> Co-authored-by: Andrea Panizza <8233615+AndreaPi@users.noreply.github.com> Co-authored-by: Uno Wu <st9007a@gmail.com> Co-authored-by: Leonardo Rossi <leonardo.rossi@unipr.it> Co-authored-by: BigDong <yudongwang@tju.edu.cn> Co-authored-by: Haobo Yuan <yuanhaobo@whu.edu.cn> Co-authored-by: Shusheng Yang <shusheng.yang@qq.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com> Co-authored-by: Shilong Zhang <61961338+jshilong@users.noreply.github.com> * [Fix]fix init when densehead contains dcn (#6625) * fix init when densehead contains dcn * change to constant init * fix init of convfchead (#6624) * polish docs (#6623) * fix pseudosampler (#6622) * [Fix]Add an example of combining swin and one-stage models (#6621) * add an example of swin is used in one-stage model * fix comments * add a notice * add mmhuman3d in readme (#6699) * [Fix] Fix init weights in Swin and PVT. (#6663) * [Fix] update metafile (#6717) * [Fix] fix metafile and config file * minior fix * Update configs/seesaw_loss/metafile.yml Co-authored-by: RangiLyu <lyuchqi@gmail.com> Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Release YOLOX model (#6698) * Release YOLOX model * update * update * update * update * Add 'get_ann_info' to dataset_wrappers (#6526) * Add 'get_ann_info' to dataset_wrappers * fix format * Delete unimportant notes * [Enchance] Update FAQ docs (#6587) * Fix mosaic repr typo (#6523) * Include mmflow in readme (#6545) * Include mmflow in readme * Include mmflow in README_zh-CN * Add mmflow url into the document menu in docs/conf.py and docs_zh-CN/conf.py. * Make OHEM work with seesaw loss (#6514) * [Enhance] Support file_client in Datasets and evaluating panoptic results on Ceph (#6489) * first version * Replace with our api * Add copyright * Move the runtime error to multi_core interface * Add docstring * Fix comments * Add comments * Add unit test for pq_compute_single_core * Fix MMDetection model to ONNX command (#6558) * Update README.md (#6567) * [Feature] Support custom persistent_workers (#6435) * Fix aug test error when the number of prediction bboxes is 0 (#6398) * Fix aug test error when the number of prediction bboxes is 0 * test * test * fix lint * Support custom pin_memory and persistent_workers * fix comment * fix docstr * remove pin_memory * Fix SSD512 config error (#6574) * Fix mosaic repr typo (#6523) * Include mmflow in readme (#6545) * Include mmflow in readme * Include mmflow in README_zh-CN * Add mmflow url into the document menu in docs/conf.py and docs_zh-CN/conf.py. * Make OHEM work with seesaw loss (#6514) * Fix ssd512 config error Co-authored-by: Kyungmin Lee <30465912+lkm2835@users.noreply.github.com> Co-authored-by: Czm369 <40661020+Czm369@users.noreply.github.com> Co-authored-by: ohwi <supebulous@gmail.com> * Catch symlink failure on Windows (#6482) * Catch symlink failure on Windows Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * Set copy mode on Windows Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * Fix lint Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * Fix logic error Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com> * [Feature] Support Label Assignment Distillation (LAD) (#6342) * add LAD * inherit LAD from KnowledgeDistillationSingleStageDetector * add configs/lad/lad_r101_paa_r50_fpn_coco_1x.py * update LAD readme * update configs/lad/README.md * try not to use abbreviations for variable names * add unittest for lad_head * update test_lad_head * remove main in tests/test_models/test_dense_heads/test_lad_head.py * [Fix] Avoid infinite GPU waiting in dist training (#6501) * [#6495] fix infinite GPU waiting in dist training * print log_vars keys in assertion msg * linting issue * Support to collect the best models (#6560) * Fix mosaic repr typo (#6523) * Include mmflow in readme (#6545) * Include mmflow in readme * Include mmflow in README_zh-CN * Add mmflow url into the document menu in docs/conf.py and docs_zh-CN/conf.py. * Make OHEM work with seesaw loss (#6514) * update * support gather best model Co-authored-by: Kyungmin Lee <30465912+lkm2835@users.noreply.github.com> Co-authored-by: Czm369 <40661020+Czm369@users.noreply.github.com> Co-authored-by: ohwi <supebulous@gmail.com> * [Enhance]: Optimize augmentation pipeline to speed up training. (#6442) * Refactor YOLOX (#6443) * Fix aug test error when the number of prediction bboxes is 0 (#6398) * Fix aug test error when the number of prediction bboxes is 0 * test * test * fix lint * Support custom pin_memory and persistent_workers * [Docs] Chinese version of robustness_benchmarking.md (#6375) * Chinese version of robustness_benchmarking.md * Update docs_zh-CN/robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Update docs_zh-CN/robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Update docs_zh-CN/robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Update docs_zh-CN/robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Update docs_zh-CN/robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Update docs_zh-CN/robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * Update robustness_benchmarking.md * Update robustness_benchmarking.md * Update robustness_benchmarking.md * Update robustness_benchmarking.md * Update robustness_benchmarking.md * Update robustness_benchmarking.md Co-authored-by: RangiLyu <lyuchqi@gmail.com> * update yolox_s * update yolox_s * support dynamic eval interval * fix some error * support ceph * fix none error * fix batch error * replace resize * fix comment * fix docstr * Update the link of checkpoints (#6460) * [Feature]: Support plot confusion matrix. (#6344) * remove pin_memory * update * fix unittest * update cfg * fix error * add unittest * [Fix] Fix SpatialReductionAttention in PVT. (#6488) * [Fix] Fix SpatialReductionAttention in PVT * Add warning * Save coco summarize print information to logger (#6505) * Fix type error in 2_new_data_mode (#6469) * Always map location to cpu when load checkpoint (#6405) * configs: update groie README (#6401) Signed-off-by: Leonardo Rossi <leonardo.rossi@unipr.it> * [Fix] fix config path in docs (#6396) * [Enchance] Set a random seed when the user does not set a seed. (#6457) * fix random seed bug * add comment * enchance random seed * rename Co-authored-by: Haobo Yuan <yuanhaobo@whu.edu.cn> * [BugFixed] fix wrong trunc_normal_init use (#6432) * fix wrong trunc_normal_init use * fix wrong trunc_normal_init use * fix #6446 Co-authored-by: Uno Wu <st9007a@gmail.com> Co-authored-by: Leonardo Rossi <leonardo.rossi@unipr.it> Co-authored-by: BigDong <yudongwang@tju.edu.cn> Co-authored-by: Haian Huang(深度眸) <1286304229@qq.com> Co-authored-by: Haobo Yuan <yuanhaobo@whu.edu.cn> Co-authored-by: Shusheng Yang <shusheng.yang@qq.com> * bump version to v2.18.1 (#6510) * bump version to v2.18.1 * Update changelog.md * add some comment * fix some comment * update readme * fix lint * add reduce mean * update * update readme * update params Co-authored-by: Cedric Luo <luochunhua1996@outlook.com> Co-authored-by: RangiLyu <lyuchqi@gmail.com> Co-authored-by: Guangchen Lin <347630870@qq.com> Co-authored-by: Andrea Panizza <8233615+AndreaPi@users.noreply.github.com> Co-authored-by: Uno Wu <st9007a@gmail.com> Co-authored-by: Leonardo Rossi <leonardo.rossi@unipr.it> Co-authored-by: BigDong <yudongwang@tju.edu.cn> Co-authored-by: Haobo Yuan <yuanhaobo@whu.edu.cn> Co-authored-by: Shusheng Yang <shusheng.yang@qq.com> * [Refactor] Remove some code in `mmdet/apis/train.py` (#6576) * remove some code about custom hooks in apis/train.py * files were modified by yapf * Fix lad repeatedly output warning message (#6584) * update faq docs * update * update * update * fix lint * update * update * update * update readme * Rephrase Co-authored-by: Kyungmin Lee <30465912+lkm2835@users.noreply.github.com> Co-authored-by: Czm369 <40661020+Czm369@users.noreply.github.com> Co-authored-by: ohwi <supebulous@gmail.com> Co-authored-by: Guangchen Lin <347630870@qq.com> Co-authored-by: Rishit Dagli <rishit.dagli@gmail.com> Co-authored-by: RangiLyu <lyuchqi@gmail.com> Co-authored-by: del-zhenwu <dele.zhenwu@gmail.com> Co-authored-by: Thuy Ng <thuypn9a4@gmail.com> Co-authored-by: Han Zhang <623606860@qq.com> Co-authored-by: Cedric Luo <luochunhua1996@outlook.com> Co-authored-by: Andrea Panizza <8233615+AndreaPi@users.noreply.github.com> Co-authored-by: Uno Wu <st9007a@gmail.com> Co-authored-by: Leonardo Rossi <leonardo.rossi@unipr.it> Co-authored-by: BigDong <yudongwang@tju.edu.cn> Co-authored-by: Haobo Yuan <yuanhaobo@whu.edu.cn> Co-authored-by: Shusheng Yang <shusheng.yang@qq.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com> * Support keeping image ratio in the multi-scale training of YOLOX (#6732) * Support keeping ratio in the multi-scale training of YOLOX * fix typos * fix typos * [Doc]Add doc for detect_anomalous_params (#6697) * add doc for detect_anomalous_params * add min verision requirment * fix version and key * fix version * Fix dtype bug in base_dense_head * Support `bbox_clip_border` for the augmentations of YOLOX (#6730) * support 'bbox_clip_border' for the augmentations of YOLOX * update based on 1-st comments * add comments * fix typos * rename remove_ouside_bboxes to find_inside_bboxes * move comments to docstring * [Fix] Fix SimOTA with no valid bbox. (#6733) * [Enhance] support select gpu-ids in testing time * [Fix] fix link (#6796) * [Fix] fix lint * support select gpu-ids in testing time * minor fix Co-authored-by: Joson <1250377062@qq.com> Co-authored-by: Czm369 <40661020+Czm369@users.noreply.github.com> Co-authored-by: Kyungmin Lee <30465912+lkm2835@users.noreply.github.com> Co-authored-by: ohwi <supebulous@gmail.com> Co-authored-by: Guangchen Lin <347630870@qq.com> Co-authored-by: Rishit Dagli <rishit.dagli@gmail.com> Co-authored-by: RangiLyu <lyuchqi@gmail.com> Co-authored-by: Haian Huang(深度眸) <1286304229@qq.com> Co-authored-by: del-zhenwu <dele.zhenwu@gmail.com> Co-authored-by: Thuy Ng <thuypn9a4@gmail.com> Co-authored-by: Han Zhang <623606860@qq.com> Co-authored-by: Cedric Luo <luochunhua1996@outlook.com> Co-authored-by: Andrea Panizza <8233615+AndreaPi@users.noreply.github.com> Co-authored-by: Uno Wu <st9007a@gmail.com> Co-authored-by: Leonardo Rossi <leonardo.rossi@unipr.it> Co-authored-by: Haobo Yuan <yuanhaobo@whu.edu.cn> Co-authored-by: Shusheng Yang <shusheng.yang@qq.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com> Co-authored-by: Shilong Zhang <61961338+jshilong@users.noreply.github.com> Co-authored-by: ChaimZhu <zcm952742165@gmail.com> Co-authored-by: Shane Zhao <491498408@qq.com> Co-authored-by: Tao Gong <gt950513@mail.ustc.edu.cn> Co-authored-by: Yosuke Shinya <42844407+shinya7y@users.noreply.github.com>
diff --git a/tools/test.py b/tools/test.py
@@ -33,6 +33,12 @@ def parse_args():
         action='store_true',
         help='Whether to fuse conv and bn, this will slightly increase'
         'the inference speed')
+    parser.add_argument(
+        '--gpu-ids',
+        type=int,
+        nargs='+',
+        help='ids of gpus to use '
+        '(only applicable to non-distributed testing)')
     parser.add_argument(
         '--format-only',
         action='store_true',
@@ -155,9 +161,20 @@ def main():
             for ds_cfg in cfg.data.test:
                 ds_cfg.pipeline = replace_ImageToTensor(ds_cfg.pipeline)
 
+    if args.gpu_ids is not None:
+        cfg.gpu_ids = args.gpu_ids
+    else:
+        cfg.gpu_ids = range(1)
+
     # init distributed env first, since logger depends on the dist info.
     if args.launcher == 'none':
         distributed = False
+        if len(cfg.gpu_ids) > 1:
+            warnings.warn(
+                f'We treat {cfg.gpu_ids} as gpu-ids, and reset to '
+                f'{cfg.gpu_ids[0:1]} as gpu-ids to avoid potential error in '
+                'non-distribute testing time.')
+            cfg.gpu_ids = cfg.gpu_ids[0:1]
     else:
         distributed = True
         init_dist(args.launcher, **cfg.dist_params)
@@ -195,7 +212,7 @@ def main():
         model.CLASSES = dataset.CLASSES
 
     if not distributed:
-        model = MMDataParallel(model, device_ids=[0])
+        model = MMDataParallel(model, device_ids=cfg.gpu_ids)
         outputs = single_gpu_test(model, data_loader, args.show, args.show_dir,
                                   args.show_score_thr)
     else:
diff --git a/tools/train.py b/tools/train.py
@@ -117,6 +117,12 @@ def main():
     # init distributed env first, since logger depends on the dist info.
     if args.launcher == 'none':
         distributed = False
+        if len(cfg.gpu_ids) > 1:
+            warnings.warn(
+                f'We treat {cfg.gpu_ids} as gpu-ids, and reset to '
+                f'{cfg.gpu_ids[0:1]} as gpu-ids to avoid potential error in '
+                'non-distribute training time.')
+            cfg.gpu_ids = cfg.gpu_ids[0:1]
     else:
         distributed = True
         init_dist(args.launcher, **cfg.dist_params)
