Skip to content

Commit 743bb5f

Browse files
chore: update Deformable_Detr model card (#39902)
* chore: update Deformable_Detr model card * fix: added pipeline, automodel examples and checkpoints link * Update deformable_detr.md --------- Co-authored-by: Steven Liu <[email protected]>
1 parent ac0b468 commit 743bb5f

File tree

1 file changed

+61
-21
lines changed

1 file changed

+61
-21
lines changed

docs/source/en/model_doc/deformable_detr.md

Lines changed: 61 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -14,43 +14,83 @@ rendered properly in your Markdown viewer.
1414
1515
-->
1616

17-
# Deformable DETR
18-
19-
<div class="flex flex-wrap space-x-1">
20-
<img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-DE3412?style=flat&logo=pytorch&logoColor=white">
17+
<div style="float: right;">
18+
<div class="flex flex-wrap space-x-1">
19+
<img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-DE3412?style=flat&logo=pytorch&logoColor=white">
20+
</div>
2121
</div>
2222

23-
## Overview
24-
25-
The Deformable DETR model was proposed in [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://huggingface.co/papers/2010.04159) by Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, Jifeng Dai.
26-
Deformable DETR mitigates the slow convergence issues and limited feature spatial resolution of the original [DETR](detr) by leveraging a new deformable attention module which only attends to a small set of key sampling points around a reference.
27-
28-
The abstract from the paper is the following:
23+
# Deformable DETR
2924

30-
*DETR has been recently proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance. However, it suffers from slow convergence and limited feature spatial resolution, due to the limitation of Transformer attention modules in processing image feature maps. To mitigate these issues, we proposed Deformable DETR, whose attention modules only attend to a small set of key sampling points around a reference. Deformable DETR can achieve better performance than DETR (especially on small objects) with 10 times less training epochs. Extensive experiments on the COCO benchmark demonstrate the effectiveness of our approach.*
25+
[Deformable DETR](https://huggingface.co/papers/2010.04159) improves on the original [DETR](./detr) by using a deformable attention module. This mechanism selectively attends to a small set of key sampling points around a reference. It improves training speed and improves accuracy.
3126

3227
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/deformable_detr_architecture.png"
3328
alt="drawing" width="600"/>
3429

3530
<small> Deformable DETR architecture. Taken from the <a href="https://huggingface.co/papers/2010.04159">original paper</a>.</small>
3631

37-
This model was contributed by [nielsr](https://huggingface.co/nielsr). The original code can be found [here](https://github.com/fundamentalvision/Deformable-DETR).
32+
You can find all the available Deformable DETR checkpoints under the [SenseTime](https://huggingface.co/SenseTime) organization.
3833

39-
## Usage tips
34+
> [!TIP]
35+
> This model was contributed by [nielsr](https://huggingface.co/nielsr).
36+
>
37+
> Click on the Deformable DETR models in the right sidebar for more examples of how to apply Deformable DETR to different object detection and segmentation tasks.
4038
41-
- Training Deformable DETR is equivalent to training the original [DETR](detr) model. See the [resources](#resources) section below for demo notebooks.
39+
The example below demonstrates how to perform object detection with the [`Pipeline`] and the [`AutoModel`] class.
4240

43-
## Resources
41+
<hfoptions id="usage">
42+
<hfoption id="Pipeline">
43+
44+
```python
45+
from transformers import pipeline
46+
import torch
47+
48+
pipeline = pipeline(
49+
"object-detection",
50+
model="SenseTime/deformable-detr",
51+
torch_dtype=torch.float16,
52+
device_map=0
53+
)
54+
55+
pipeline("http://images.cocodataset.org/val2017/000000039769.jpg")
56+
```
4457

45-
A list of official Hugging Face and community (indicated by 🌎) resources to help you get started with Deformable DETR.
58+
</hfoption>
59+
<hfoption id="AutoModel">
4660

47-
<PipelineTag pipeline="object-detection"/>
61+
```python
62+
from transformers import AutoImageProcessor, AutoModelForObjectDetection
63+
from PIL import Image
64+
import requests
65+
import torch
4866

49-
- Demo notebooks regarding inference + fine-tuning on a custom dataset for [`DeformableDetrForObjectDetection`] can be found [here](https://github.com/NielsRogge/Transformers-Tutorials/tree/master/Deformable-DETR).
50-
- Scripts for finetuning [`DeformableDetrForObjectDetection`] with [`Trainer`] or [Accelerate](https://huggingface.co/docs/accelerate/index) can be found [here](https://github.com/huggingface/transformers/tree/main/examples/pytorch/object-detection).
51-
- See also: [Object detection task guide](../tasks/object_detection).
67+
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
68+
image = Image.open(requests.get(url, stream=True).raw)
69+
70+
image_processor = AutoImageProcessor.from_pretrained("SenseTime/deformable-detr")
71+
model = AutoModelForObjectDetection.from_pretrained("SenseTime/deformable-detr")
72+
73+
# prepare image for the model
74+
inputs = image_processor(images=image, return_tensors="pt")
75+
76+
with torch.no_grad():
77+
outputs = model(**inputs)
78+
79+
results = image_processor.post_process_object_detection(outputs, target_sizes=torch.tensor([image.size[::-1]]), threshold=0.3)
80+
81+
for result in results:
82+
for score, label_id, box in zip(result["scores"], result["labels"], result["boxes"]):
83+
score, label = score.item(), label_id.item()
84+
box = [round(i, 2) for i in box.tolist()]
85+
print(f"{model.config.id2label[label]}: {score:.2f} {box}")
86+
```
87+
88+
</hfoption>
89+
</hfoptions>
90+
91+
## Resources
5292

53-
If you're interested in submitting a resource to be included here, please feel free to open a Pull Request and we'll review it! The resource should ideally demonstrate something new instead of duplicating an existing resource.
93+
- Refer to this set of [notebooks](https://github.com/NielsRogge/Transformers-Tutorials/tree/master/Deformable-DETR) for inference and fine-tuning [`DeformableDetrForObjectDetection`] on a custom dataset.
5494

5595
## DeformableDetrImageProcessor
5696

0 commit comments

Comments
 (0)