推理代码

[qsyylstt-jmh]

能问下有开源推理的代码吗，没有的话能提供吗

[VoyagerXvoyagerx]

Hi, I was working for this too. LAE-DINO codebaserelies on the mmdet repo. Following the instructions in the mm-groundingdino will help. #11

[jaychempan]

We add the inference examples. If you have any questions, we welcome you to propose issues.

[qsyylstt-jmh]

我们添加推理示例。如果您有任何问题，我们欢迎您提出问题。

configs没有添加pipeline，给的推理的代码是跑不通的

[VoyagerXvoyagerx]

This code block worked for me

import torch
import cv2
import numpy as np
from mmdet.apis import init_detector, inference_detector
import torch

config_file = './mmdetection_lae/configs/lae_dino/lae_dino_swin-t_pretrain_LAE-1M.py' 
checkpoint_file = './lae_dino_swint_lae1m-28ca3a15.pth'
device = 'cuda:0' if torch.cuda.is_available() else 'cpu'
print(device)
model = init_detector(config_file, checkpoint_file, device=device)

img = cv2.imread(image_path)
result = inference_detector(model, img , text_prompt="boat. car. dock. jetski. lift.", custom_entities=True)
result

Output:

/home/zkcs/zyj/label-engine/baseline/LAE-DINO/mmdetection_lae/mmdet/models/layers/positional_encoding.py:103: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  dim_t = self.temperature**(2 * (dim_t // 2) / self.num_feats)
/home/zkcs/anaconda3/envs/lae/lib/python3.8/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  /opt/conda/conda-bld/pytorch_1639180588308/work/aten/src/ATen/native/TensorShape.cpp:2157.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/home/zkcs/zyj/label-engine/baseline/LAE-DINO/mmdetection_lae/mmdet/models/layers/transformer/utils.py:71: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  dim_t = temperature**(2 * (dim_t // 2) / num_feats)
/home/zkcs/anaconda3/envs/lae/lib/python3.8/site-packages/mmcv/cnn/bricks/transformer.py:524: UserWarning: position encoding of key ismissing in MultiheadAttention.
  warnings.warn(f'position encoding of key is'
/home/zkcs/zyj/label-engine/baseline/LAE-DINO/mmdetection_lae/mmdet/models/dense_heads/lae_dino_head.py:428: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  bbox_index = indexes // num_classes
<DetDataSample(

    META INFORMATION
    batch_input_shape: (800, 800)
    scale_factor: (0.78125, 0.78125)
    custom_entities: True
    pad_shape: (800, 800)
    img_shape: (800, 800)
    ori_shape: (1024, 1024)
    text: 'boat. car. dock. jetski. lift.'
    img_id: 0
    img_path: None

    DATA FIELDS
    ignored_instances: <InstanceData(
        
            META INFORMATION
        
            DATA FIELDS
            bboxes: tensor([], device='cuda:0', size=(0, 4))
            labels: tensor([], device='cuda:0', dtype=torch.int64)
        ) at 0x7f01af40d550>
    token_positive_map: 
        1: [1]
        2: [3]
        3: [5]
        4: [7, 8]
        5: [10]
    pred_instances: <InstanceData(
        
            META INFORMATION
        
            DATA FIELDS
            label_names: ['boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'lift', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'lift', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'dock', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'dock', 'car', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'lift', 'dock', 'boat', 'boat', 'lift', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'car', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'lift', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'dock', 'boat', 'boat', 'boat', 'car', 'dock', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'lift', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'car', 'boat', 'boat', 'dock', 'lift', 'boat', 'dock', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'lift', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat']
            bboxes: tensor([[ 522.9168,  387.5073,  795.4091,  736.4803],
                        [ 715.3210,  546.3906,  751.0032,  579.0335],
                        [ 224.4279,  305.3248,  254.7329,  323.8498],
                        ...,
                        [ 927.7061,  385.9805, 1023.9696,  581.9641],
                        [ 206.4692,  725.7925,  301.5588,  768.3694],
                        [ 483.0113,  602.0540,  520.6093,  648.0154]], device='cuda:0')
            scores: tensor([0.2122, 0.1376, 0.0998, 0.0994, 0.0936, 0.0933, 0.0912, 0.0901, 0.0859,
                        0.0850, 0.0831, 0.0793, 0.0759, 0.0729, 0.0711, 0.0697, 0.0659, 0.0621,
                        0.0614, 0.0610, 0.0608, 0.0603, 0.0598, 0.0593, 0.0582, 0.0576, 0.0564,
                        0.0558, 0.0556, 0.0556, 0.0534, 0.0532, 0.0523, 0.0516, 0.0516, 0.0514,
                        0.0512, 0.0506, 0.0504, 0.0492, 0.0492, 0.0489, 0.0477, 0.0477, 0.0464,
                        0.0457, 0.0456, 0.0455, 0.0452, 0.0444, 0.0439, 0.0436, 0.0430, 0.0427,
                        0.0426, 0.0420, 0.0417, 0.0415, 0.0411, 0.0410, 0.0410, 0.0396, 0.0395,
                        0.0394, 0.0388, 0.0386, 0.0386, 0.0385, 0.0378, 0.0377, 0.0369, 0.0369,
                        0.0368, 0.0367, 0.0364, 0.0361, 0.0359, 0.0358, 0.0355, 0.0353, 0.0353,
                        0.0351, 0.0346, 0.0344, 0.0343, 0.0342, 0.0342, 0.0341, 0.0340, 0.0339,
                        0.0339, 0.0336, 0.0332, 0.0330, 0.0330, 0.0327, 0.0325, 0.0323, 0.0322,
                        0.0321, 0.0319, 0.0318, 0.0318, 0.0318, 0.0317, 0.0316, 0.0315, 0.0315,
                        0.0311, 0.0309, 0.0308, 0.0305, 0.0303, 0.0303, 0.0303, 0.0303, 0.0302,
                        0.0302, 0.0301, 0.0300, 0.0298, 0.0297, 0.0297, 0.0296, 0.0296, 0.0295,
                        0.0295, 0.0294, 0.0292, 0.0291, 0.0290, 0.0288, 0.0288, 0.0287, 0.0287,
                        0.0287, 0.0284, 0.0280, 0.0278, 0.0278, 0.0275, 0.0274, 0.0273, 0.0272,
                        0.0270, 0.0270, 0.0267, 0.0266, 0.0266, 0.0265, 0.0265, 0.0264, 0.0263,
                        0.0260, 0.0260, 0.0258, 0.0257, 0.0256, 0.0256, 0.0256, 0.0255, 0.0255,
                        0.0255, 0.0254, 0.0254, 0.0254, 0.0254, 0.0251, 0.0250, 0.0249, 0.0249,
                        0.0248, 0.0247, 0.0247, 0.0246, 0.0245, 0.0245, 0.0243, 0.0243, 0.0243,
                        0.0243, 0.0243, 0.0243, 0.0243, 0.0242, 0.0242, 0.0242, 0.0240, 0.0239,
                        0.0239, 0.0239, 0.0238, 0.0237, 0.0235, 0.0235, 0.0234, 0.0233, 0.0232,
                        0.0231, 0.0231, 0.0230, 0.0229, 0.0229, 0.0229, 0.0229, 0.0228, 0.0228,
                        0.0227, 0.0224, 0.0222, 0.0222, 0.0221, 0.0220, 0.0219, 0.0219, 0.0218,
                        0.0218, 0.0218, 0.0218, 0.0218, 0.0217, 0.0217, 0.0217, 0.0217, 0.0215,
                        0.0214, 0.0214, 0.0214, 0.0214, 0.0213, 0.0211, 0.0211, 0.0211, 0.0211,
                        0.0211, 0.0210, 0.0210, 0.0210, 0.0209, 0.0209, 0.0208, 0.0208, 0.0208,
                        0.0208, 0.0206, 0.0206, 0.0205, 0.0204, 0.0204, 0.0203, 0.0203, 0.0202,
                        0.0202, 0.0202, 0.0200, 0.0200, 0.0200, 0.0200, 0.0199, 0.0198, 0.0197,
                        0.0197, 0.0197, 0.0197, 0.0197, 0.0196, 0.0196, 0.0196, 0.0195, 0.0195,
                        0.0195, 0.0194, 0.0194, 0.0193, 0.0193, 0.0192, 0.0192, 0.0192, 0.0192,
                        0.0192, 0.0192, 0.0191, 0.0191, 0.0190, 0.0190, 0.0190, 0.0190, 0.0188,
                        0.0187, 0.0187, 0.0187, 0.0187, 0.0187, 0.0187, 0.0186, 0.0186, 0.0186,
                        0.0186, 0.0186, 0.0185], device='cuda:0')
            labels: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                        0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0,
                        0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 2, 0, 0, 2, 0, 2, 0, 0, 0, 0, 2,
                        0, 0, 0, 0, 2, 1, 0, 0, 2, 0, 0, 0, 0, 0, 2, 0, 4, 2, 0, 0, 4, 0, 0, 0,
                        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 4,
                        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                        0, 0, 0, 2, 2, 0, 0, 0, 1, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0,
                        0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,
                        0, 2, 4, 0, 2, 0, 2, 0, 0, 0, 0, 2, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 4, 0,
                        0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0], device='cuda:0')
        ) at 0x7f01af40db50>
    gt_instances: <InstanceData(
        
            META INFORMATION
        
            DATA FIELDS
            bboxes: tensor([], device='cuda:0', size=(0, 4))
            labels: tensor([], device='cuda:0', dtype=torch.int64)
        ) at 0x7f01af40d4c0>
) at 0x7f01af40d790>

[Hranran]

This code block worked for me

import torch
import cv2
import numpy as np
from mmdet.apis import init_detector, inference_detector
import torch

config_file = './mmdetection_lae/configs/lae_dino/lae_dino_swin-t_pretrain_LAE-1M.py'
checkpoint_file = './lae_dino_swint_lae1m-28ca3a15.pth'
device = 'cuda:0' if torch.cuda.is_available() else 'cpu'
print(device)
model = init_detector(config_file, checkpoint_file, device=device)

img = cv2.imread(image_path)
result = inference_detector(model, img , text_prompt="boat. car. dock. jetski. lift.", custom_entities=True)
result
Output:

/home/zkcs/zyj/label-engine/baseline/LAE-DINO/mmdetection_lae/mmdet/models/layers/positional_encoding.py:103: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  dim_t = self.temperature**(2 * (dim_t // 2) / self.num_feats)
/home/zkcs/anaconda3/envs/lae/lib/python3.8/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  /opt/conda/conda-bld/pytorch_1639180588308/work/aten/src/ATen/native/TensorShape.cpp:2157.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/home/zkcs/zyj/label-engine/baseline/LAE-DINO/mmdetection_lae/mmdet/models/layers/transformer/utils.py:71: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  dim_t = temperature**(2 * (dim_t // 2) / num_feats)
/home/zkcs/anaconda3/envs/lae/lib/python3.8/site-packages/mmcv/cnn/bricks/transformer.py:524: UserWarning: position encoding of key ismissing in MultiheadAttention.
  warnings.warn(f'position encoding of key is'
/home/zkcs/zyj/label-engine/baseline/LAE-DINO/mmdetection_lae/mmdet/models/dense_heads/lae_dino_head.py:428: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  bbox_index = indexes // num_classes
<DetDataSample(

    META INFORMATION
    batch_input_shape: (800, 800)
    scale_factor: (0.78125, 0.78125)
    custom_entities: True
    pad_shape: (800, 800)
    img_shape: (800, 800)
    ori_shape: (1024, 1024)
    text: 'boat. car. dock. jetski. lift.'
    img_id: 0
    img_path: None

    DATA FIELDS
    ignored_instances: <InstanceData(
        
            META INFORMATION
        
            DATA FIELDS
            bboxes: tensor([], device='cuda:0', size=(0, 4))
            labels: tensor([], device='cuda:0', dtype=torch.int64)
        ) at 0x7f01af40d550>
    token_positive_map: 
        1: [1]
        2: [3]
        3: [5]
        4: [7, 8]
        5: [10]
    pred_instances: <InstanceData(
        
            META INFORMATION
        
            DATA FIELDS
            label_names: ['boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'lift', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'lift', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'dock', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'dock', 'car', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'lift', 'dock', 'boat', 'boat', 'lift', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'car', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'lift', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'dock', 'boat', 'boat', 'boat', 'car', 'dock', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'lift', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'car', 'boat', 'boat', 'dock', 'lift', 'boat', 'dock', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'lift', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat', 'dock', 'boat', 'boat', 'boat', 'boat', 'boat', 'boat']
            bboxes: tensor([[ 522.9168,  387.5073,  795.4091,  736.4803],
                        [ 715.3210,  546.3906,  751.0032,  579.0335],
                        [ 224.4279,  305.3248,  254.7329,  323.8498],
                        ...,
                        [ 927.7061,  385.9805, 1023.9696,  581.9641],
                        [ 206.4692,  725.7925,  301.5588,  768.3694],
                        [ 483.0113,  602.0540,  520.6093,  648.0154]], device='cuda:0')
            scores: tensor([0.2122, 0.1376, 0.0998, 0.0994, 0.0936, 0.0933, 0.0912, 0.0901, 0.0859,
                        0.0850, 0.0831, 0.0793, 0.0759, 0.0729, 0.0711, 0.0697, 0.0659, 0.0621,
                        0.0614, 0.0610, 0.0608, 0.0603, 0.0598, 0.0593, 0.0582, 0.0576, 0.0564,
                        0.0558, 0.0556, 0.0556, 0.0534, 0.0532, 0.0523, 0.0516, 0.0516, 0.0514,
                        0.0512, 0.0506, 0.0504, 0.0492, 0.0492, 0.0489, 0.0477, 0.0477, 0.0464,
                        0.0457, 0.0456, 0.0455, 0.0452, 0.0444, 0.0439, 0.0436, 0.0430, 0.0427,
                        0.0426, 0.0420, 0.0417, 0.0415, 0.0411, 0.0410, 0.0410, 0.0396, 0.0395,
                        0.0394, 0.0388, 0.0386, 0.0386, 0.0385, 0.0378, 0.0377, 0.0369, 0.0369,
                        0.0368, 0.0367, 0.0364, 0.0361, 0.0359, 0.0358, 0.0355, 0.0353, 0.0353,
                        0.0351, 0.0346, 0.0344, 0.0343, 0.0342, 0.0342, 0.0341, 0.0340, 0.0339,
                        0.0339, 0.0336, 0.0332, 0.0330, 0.0330, 0.0327, 0.0325, 0.0323, 0.0322,
                        0.0321, 0.0319, 0.0318, 0.0318, 0.0318, 0.0317, 0.0316, 0.0315, 0.0315,
                        0.0311, 0.0309, 0.0308, 0.0305, 0.0303, 0.0303, 0.0303, 0.0303, 0.0302,
                        0.0302, 0.0301, 0.0300, 0.0298, 0.0297, 0.0297, 0.0296, 0.0296, 0.0295,
                        0.0295, 0.0294, 0.0292, 0.0291, 0.0290, 0.0288, 0.0288, 0.0287, 0.0287,
                        0.0287, 0.0284, 0.0280, 0.0278, 0.0278, 0.0275, 0.0274, 0.0273, 0.0272,
                        0.0270, 0.0270, 0.0267, 0.0266, 0.0266, 0.0265, 0.0265, 0.0264, 0.0263,
                        0.0260, 0.0260, 0.0258, 0.0257, 0.0256, 0.0256, 0.0256, 0.0255, 0.0255,
                        0.0255, 0.0254, 0.0254, 0.0254, 0.0254, 0.0251, 0.0250, 0.0249, 0.0249,
                        0.0248, 0.0247, 0.0247, 0.0246, 0.0245, 0.0245, 0.0243, 0.0243, 0.0243,
                        0.0243, 0.0243, 0.0243, 0.0243, 0.0242, 0.0242, 0.0242, 0.0240, 0.0239,
                        0.0239, 0.0239, 0.0238, 0.0237, 0.0235, 0.0235, 0.0234, 0.0233, 0.0232,
                        0.0231, 0.0231, 0.0230, 0.0229, 0.0229, 0.0229, 0.0229, 0.0228, 0.0228,
                        0.0227, 0.0224, 0.0222, 0.0222, 0.0221, 0.0220, 0.0219, 0.0219, 0.0218,
                        0.0218, 0.0218, 0.0218, 0.0218, 0.0217, 0.0217, 0.0217, 0.0217, 0.0215,
                        0.0214, 0.0214, 0.0214, 0.0214, 0.0213, 0.0211, 0.0211, 0.0211, 0.0211,
                        0.0211, 0.0210, 0.0210, 0.0210, 0.0209, 0.0209, 0.0208, 0.0208, 0.0208,
                        0.0208, 0.0206, 0.0206, 0.0205, 0.0204, 0.0204, 0.0203, 0.0203, 0.0202,
                        0.0202, 0.0202, 0.0200, 0.0200, 0.0200, 0.0200, 0.0199, 0.0198, 0.0197,
                        0.0197, 0.0197, 0.0197, 0.0197, 0.0196, 0.0196, 0.0196, 0.0195, 0.0195,
                        0.0195, 0.0194, 0.0194, 0.0193, 0.0193, 0.0192, 0.0192, 0.0192, 0.0192,
                        0.0192, 0.0192, 0.0191, 0.0191, 0.0190, 0.0190, 0.0190, 0.0190, 0.0188,
                        0.0187, 0.0187, 0.0187, 0.0187, 0.0187, 0.0187, 0.0186, 0.0186, 0.0186,
                        0.0186, 0.0186, 0.0185], device='cuda:0')
            labels: tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                        0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0,
                        0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 2, 0, 0, 2, 0, 2, 0, 0, 0, 0, 2,
                        0, 0, 0, 0, 2, 1, 0, 0, 2, 0, 0, 0, 0, 0, 2, 0, 4, 2, 0, 0, 4, 0, 0, 0,
                        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 4,
                        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
                        0, 0, 0, 2, 2, 0, 0, 0, 1, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0,
                        0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,
                        0, 2, 4, 0, 2, 0, 2, 0, 0, 0, 0, 2, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 4, 0,
                        0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0], device='cuda:0')
        ) at 0x7f01af40db50>
    gt_instances: <InstanceData(
        
            META INFORMATION
        
            DATA FIELDS
            bboxes: tensor([], device='cuda:0', size=(0, 4))
            labels: tensor([], device='cuda:0', dtype=torch.int64)
        ) at 0x7f01af40d4c0>
) at 0x7f01af40d790>

I tried your method and encountered this error: AttributeError: 'list' object has no attribute 'get'

I suspect this might be due to version incompatibility. Could you share the versions of mmcv and mmdetection you’re using?

Error log:

(lae) prime@RansComputer:~/code/projects/LAE-DINO/mmdetection_lae$ /home/prime/miniconda3/envs/lae/bin/python /home/prime/code/projects/LAE-DINO/test_3.py
cuda:0
Loads checkpoint by local backend from path: /home/prime/code/projects/LAE-DINO/lae_dino_swint_lae1m-28ca3a15.pth
/home/prime/code/projects/LAE-DINO/mmdetection_lae/mmdet/apis/inference.py:90: UserWarning: dataset_meta or class names are not saved in the checkpoint's meta data, use COCO classes by default.
  warnings.warn(
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
loading annotations into memory...
Done (t=0.06s)
creating index...
index created!
loading annotations into memory...
Done (t=0.34s)
creating index...
index created!
Traceback (most recent call last):
  File "/home/prime/code/projects/LAE-DINO/test_3.py", line 14, in <module>
    model = init_detector(config_file, checkpoint_file, device=device)
  File "/home/prime/code/projects/LAE-DINO/mmdetection_lae/mmdet/apis/inference.py", line 103, in init_detector
    cfg_palette = metainfo.get('palette', None)
AttributeError: 'list' object has no attribute 'get'
/home/prime/miniconda3/envs/lae/lib/python3.8/tempfile.py:827: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmp_g6v7esz'>
  _warnings.warn(warn_message, ResourceWarning)

[VoyagerXvoyagerx]

mmcv                      2.2.0                    pypi_0    pypi
mmdet                     3.3.0                     dev_0    <develop>
mmengine                  0.10.7                   pypi_0    pypi
python                    3.8.20          h4a871b0_2_cpython    conda-forge
pytorch                   1.10.1          py3.8_cuda11.3_cudnn8.2.0_0    pytorch

FYI @Hranran

[Andytonglove]

@Hranran I encountered the same problem, the inference code doesn't seem to be the latest version, the pipeline hasn't been added in configs, and the code can't run. The code block above can be executed, and a temporary solution is to modify line 103 in LAE-DINO/mmdetection_lae/mmdet/apis/inference.py .

[Justlovesmile]

我们添加推理示例。如果您有任何问题，我们欢迎您提出问题。

configs没有添加pipeline，给的推理的代码是跑不通的

在configs/_base_/datasets/lae_1m_detection.py中的val_dataloader添加pipeline字段即可：

val_dataloader = dict(
    batch_size=1,
    num_workers=2,
    persistent_workers=True,
    drop_last=False,
    sampler=dict(type='DefaultSampler', shuffle=False),
    dataset=dict(type='ConcatDataset', datasets=all_val_dataset, pipeline=test_pipeline)) # add pipeline

然后测试：

python demo/image_demo.py \
       --inputs images/ \
       --model configs/lae_dino/lae_dino_swin-t_pretrain_LAE-1M.py \
       --weights ../weights/lae_dino_swint_lae1m-28ca3a15.pth \
       --texts 'playground . road . tank . airplane . vehicle . bridge' -c \
       --palette random \
       --out-dir outputs \
       --pred-score-thr 0.4

[saraalemadi]

Hi,

I tried using the latest version with pipeline included in the val_dataloader, but this resulted in the following error: TypeError: __init__() got an unexpected keyword argument 'pipeline'. The code runs when removing pipeline but the results of testing using the command suggested in the README, don't match those reported in the paper. Therefore, I would appreciate any tips on how to resolve this issue.

Thanks