You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/technical_documents/jointly_prediction_planning_evaluator.md
+39-7Lines changed: 39 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,12 +1,16 @@
1
-
# JOINTLY PREDICTION PLANNING EVALUATOR
1
+
# Jointly VectorNet-TNT-Interaction Evaluator
2
2
3
3
# Introduction
4
4
5
-
The prediction module comprises 4 main functionalities: Container, Scenario, Evaluator and Predictor.
5
+
The prediction module comprises 4 main functionalities: Container, Scenario, Evaluator and Predictor.
6
6
7
-
The Evaluator predicts path and speed separately for any given obstacle. An evaluator evaluates a path by outputting a probability for it (lane sequence) using the given model stored in prediction/data/.
7
+
An Evaluator predicts paths and speeds for any given obstacles. An evaluator evaluates a path(lane sequence) with a probability by the given model stored in prediction/data/.
8
8
9
-
Jointly prediction planning evaluator is used in the new Interactive Obstacle(vehicle-type) model to generate short term trajectory points which are calculated using Vectornet and LSTM. By considering ADC's trajectory info, the obstacle trajectory prediction can be more accurate under interaction scenario.
9
+
In Apollo 7.0, a new model named Jointly VectorNet-TNT-Interaction Evaluator is
10
+
used to generate short term trajectory points. This model uses VectorNet as
11
+
encoder and TNT as decoder, in which the planning information of ADC is inserted
12
+
at the end. For scenarios at junction, the obstacle trajectory prediction by the
13
+
new model is more accurate according to the results of our experiments and road tests.
1. The encoder of jointly prediction planning evaluator is Vectornet, before model inference, we need to process obstacle and map data into the correct format.
33
+
# Algorithm Detail
34
+
## Structure
35
+

36
+
37
+
## Encoder
38
+
Basically, the encoder is mainly using an [VectorNet](https://arxiv.org/abs/2005.04259). The ADC trajectory and all obstacle trajectories in the form of coordinate points are transformed into polylines. For each polyline, vectors are connected sequentially, where it contains start point, end point, obstacle length and some other attributes of vector. All points are transformed in the coordinate of the ADC, with North as the direction and (0, 0) as the position for ADC at time 0.
39
+
40
+
After that, map information is extracted from HDMap files. As structures of lane/road/junction/crosswalk are depicted in points in HDMap, they are also processed as polylines whose vectors contain the types of traffic elements. In order to speed up calculation in Network, only information in ROI is taken into consideration.
41
+
42
+
A Subgraph is used to deal with each polyline and the feature of polyline is acquired. After that, the polyline features are inputted into a Global Graph, which is a GNN Network substantially.
43
+
44
+
The encoding feature is gained after encoding of VectorNet. For more information in detail, please find the References.
45
+
46
+
## Decoder
47
+
The structure of Decoder is a [TNT model](https://arxiv.org/abs/2008.08294).
48
+
49
+
There are three parts in TNT.
50
+
1)Target Prediction. N points around the ADC is grid-sampled and M points are selected as target points. These target points are used as the final points of prediction trajectories in the next step.
51
+
2)Motion Estimation. With VectorNet feature and target points, a prediction trajectory is generated for each selected target point. To train more efficiently, a teaching method usingtrue trajectory is also used in this step.
52
+
3)Scoring and Selection. A score is calculated for each motion trajectory in the second step, which is used to select final trajectories.
53
+
54
+
## Interactive Planning
55
+
After three steps in TNT, K predicted trajectories are assumed to be outputted. There is a latest planning trajectory with the same form, with which the distance of position and velocity can be calculated, named as (cost1, cost2) respectively. Meanwhile, weights of (cost1, cost2) are also outputted by VectorNet, outputting the final cost by multiplication.
56
+
57
+
Note we can also find a cost with the truth obstacle trajectory and ADC planning, thus producing the true value of cost. That's how the loss is calculated in this step.
58
+
59
+
60
+
# References
61
+
1. Gao, Jiyang, et al. "Vectornet: Encoding hd maps and agent dynamics from vectorized representation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.
0 commit comments