Skip to content

Commit 7531238

Browse files
authored
Add COCO model for RKNN (#2026)
## Description This adds the RKNN model trained on the COCO dataset as one of the models shipped with PV. This model is fairly general, and has been trained to identify a number of objects, including people, animals, cars, and more. This model is meant for teams to test object detection, particularly for teams who might not have access to the game elements that our other models are trained on. ## Meta Merge checklist: - [x] Pull Request title is [short, imperative summary](https://cbea.ms/git-commit/) of proposed changes - [x] The description documents the _what_ and _why_ - [x] If this PR changes behavior or adds a feature, user documentation is updated - [ ] If this PR touches photon-serde, all messages have been regenerated and hashes have not changed unexpectedly - [ ] If this PR touches configuration, this is backwards compatible with settings back to v2024.3.1 - [ ] If this PR touches pipeline settings or anything related to data exchange, the frontend typing is updated - [ ] If this PR addresses a bug, a regression test for it is added
1 parent ba1c0db commit 7531238

File tree

3 files changed

+92
-1
lines changed

3 files changed

+92
-1
lines changed

docs/source/docs/objectDetection/about-object-detection.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
PhotonVision supports object detection using neural network accelerator hardware built into Orange Pi 5/5+ coprocessors. Please note that the Orange Pi 5/5+ are the only coprocessors that are currently supported. The Neural Processing Unit, or NPU, is [used by PhotonVision](https://github.com/PhotonVision/rknn_jni/tree/main) to massively accelerate certain math operations like those needed for running ML-based object detection.
66

7-
For the 2025 season, PhotonVision ships with a pretrained ALGAE model. A model to detect coral is not currently stable, and interested teams should ask in the Photonvision discord.
7+
PhotonVision currently ships with a model trained on the [COCO dataset](https://cocodataset.org/). This model is meant to be used for testing and other miscellaneous purposes. It is not meant to be used in competition. For the 2025 post-season, PhotonVision also ships with a pretrained ALGAE model. A model to detect coral is available in the PhotonVision discord, but will not be distributed with PhotonVision.
88

99
## Tracking Objects
1010

photon-core/src/main/java/org/photonvision/common/configuration/NeuralNetworkModelManager.java

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,97 @@ private NeuralNetworkPropertyManager getShippedProperties(File modelsDirectory)
7171
Family.RKNN,
7272
Version.YOLOV8));
7373

74+
nnProps.addModelProperties(
75+
new ModelProperties(
76+
Path.of(modelsDirectory.getAbsolutePath(), "yolov8nCOCO.rknn"),
77+
"COCO",
78+
new LinkedList<String>(
79+
List.of(
80+
"person",
81+
"bicycle",
82+
"car",
83+
"motorcycle",
84+
"airplane",
85+
"bus",
86+
"train",
87+
"truck",
88+
"boat",
89+
"traffic light",
90+
"fire hydrant",
91+
"stop sign",
92+
"parking meter",
93+
"bench",
94+
"bird",
95+
"cat",
96+
"dog",
97+
"horse",
98+
"sheep",
99+
"cow",
100+
"elephant",
101+
"bear",
102+
"zebra",
103+
"giraffe",
104+
"backpack",
105+
"umbrella",
106+
"handbag",
107+
"tie",
108+
"suitcase",
109+
"frisbee",
110+
"skis",
111+
"snowboard",
112+
"sports ball",
113+
"kite",
114+
"baseball bat",
115+
"baseball glove",
116+
"skateboard",
117+
"surfboard",
118+
"tennis racket",
119+
"bottle",
120+
"wine glass",
121+
"cup",
122+
"fork",
123+
"knife",
124+
"spoon",
125+
"bowl",
126+
"banana",
127+
"apple",
128+
"sandwich",
129+
"orange",
130+
"broccoli",
131+
"carrot",
132+
"hot dog",
133+
"pizza",
134+
"donut",
135+
"cake",
136+
"chair",
137+
"couch",
138+
"potted plant",
139+
"bed",
140+
"dining table",
141+
"toilet",
142+
"tv",
143+
"laptop",
144+
"mouse",
145+
"remote",
146+
"keyboard",
147+
"cell phone",
148+
"microwave",
149+
"oven",
150+
"toaster",
151+
"sink",
152+
"refrigerator",
153+
"book",
154+
"clock",
155+
"vase",
156+
"scissors",
157+
"teddy bear",
158+
"hair drier",
159+
"toothbrush")),
160+
640,
161+
640,
162+
Family.RKNN,
163+
Version.YOLOV8));
164+
74165
return nnProps;
75166
}
76167

Binary file not shown.

0 commit comments

Comments
 (0)