Skip to content

Commit ef47989

Browse files
jpinkney-awsjpinkney-aws
authored
docs(amazonq): sequence diagram for performance telemetry #5743
## Problem It's hard to know what certain performance metrics are tracking ## Solution Add documentation
1 parent 65ba658 commit ef47989

File tree

1 file changed

+260
-0
lines changed

1 file changed

+260
-0
lines changed

‎docs/telemetry-perf.md‎

Lines changed: 260 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,260 @@
1+
# Telemetry Performance Metrics
2+
3+
Visual representations of performance telemetry metrics
4+
5+
## Amazon Q Inline
6+
7+
### codewhispererFirstCompletionLatency
8+
9+
How long it took to receive the first suggestion after we started calling the getRecommendations API
10+
11+
```mermaid
12+
sequenceDiagram
13+
participant User
14+
participant invoke as Inline invoked
15+
participant rService as Recommendation Service
16+
participant rHandler as Recommendation Handler
17+
participant backend as CWSPR backend
18+
participant sdk as Create CWSPR SDK
19+
participant token as Toolkit auth
20+
21+
22+
User->>invoke: Finished typing
23+
invoke->>rService: calls
24+
rService->>rHandler: calls
25+
rHandler->>sdk: calls
26+
27+
sdk->>token: Start getting bearer token
28+
token->>sdk: Finished getting bearer token
29+
30+
sdk->>rHandler: Return client
31+
note over rHandler, backend: codewhispererFirstCompletionLatency
32+
rect rgb(230, 230, 230, 0.5)
33+
loop Get paginated recommendations
34+
rHandler->>backend: calls
35+
end
36+
backend->>rHandler: first response received
37+
end
38+
rHandler->>User: show results
39+
backend->>rHandler: all the other responses
40+
rHandler->>User: add to already shown results
41+
```
42+
43+
### codewhispererEndToEndLatency
44+
45+
How long it took from when we started calling the getRecommendations API to when the first suggestion was shown
46+
47+
```mermaid
48+
sequenceDiagram
49+
participant User
50+
participant invoke as Inline invoked
51+
participant rService as Recommendation Service
52+
participant rHandler as Recommendation Handler
53+
participant backend as CWSPR backend
54+
participant sdk as Create CWSPR SDK
55+
participant token as Toolkit auth
56+
57+
User->>invoke: Finished typing
58+
invoke->>rService: calls
59+
rService->>rHandler: calls
60+
rHandler->>sdk: calls
61+
62+
sdk->>token: Start getting bearer token
63+
token->>sdk: Finished getting bearer token
64+
65+
sdk->>rHandler: Return client
66+
note over User, backend: codewhispererEndToEndLatency
67+
rect rgb(230, 230, 230, 0.5)
68+
loop Get paginated recommendations
69+
rHandler->>backend: calls
70+
end
71+
backend->>rHandler: first response received
72+
rHandler->>User: show results
73+
end
74+
75+
backend->>rHandler: all the other responses
76+
rHandler->>User: add to already shown results
77+
```
78+
79+
### codewhispererAllCompletionsLatency
80+
81+
How long it took to complete all paginated calls
82+
83+
```mermaid
84+
sequenceDiagram
85+
participant User
86+
participant invoke as Inline invoked
87+
participant rService as Recommendation Service
88+
participant rHandler as Recommendation Handler
89+
participant backend as CWSPR backend
90+
participant sdk as Create CWSPR SDK
91+
participant token as Toolkit auth
92+
93+
94+
User->>invoke: Finished typing
95+
invoke->>rService: calls
96+
rService->>rHandler: calls
97+
rHandler->>sdk: calls
98+
99+
sdk->>token: Start getting bearer token
100+
token->>sdk: Finished getting bearer token
101+
102+
sdk->>rHandler: Return client
103+
note over User, backend: codewhispererAllCompletionsLatency
104+
rect rgb(230, 230, 230, 0.5)
105+
loop Get paginated recommendations
106+
rHandler->>backend: calls
107+
end
108+
backend->>rHandler: first response received
109+
rHandler->>User: show results
110+
backend->>rHandler: all the other responses
111+
end
112+
113+
114+
rHandler->>User: add to already shown results
115+
```
116+
117+
### codewhispererPostprocessingLatency
118+
119+
How long it took to display the first suggestion after it received the first response from the API
120+
121+
```mermaid
122+
sequenceDiagram
123+
participant User
124+
participant invoke as Inline invoked
125+
participant rService as Recommendation Service
126+
participant rHandler as Recommendation Handler
127+
participant backend as CWSPR backend
128+
participant sdk as Create CWSPR SDK
129+
participant token as Toolkit auth
130+
131+
132+
User->>invoke: Finished typing
133+
invoke->>rService: calls
134+
rService->>rHandler: calls
135+
rHandler->>sdk: calls
136+
137+
sdk->>token: Start getting bearer token
138+
token->>sdk: Finished getting bearer token
139+
140+
sdk->>rHandler: Return client
141+
loop Get paginated recommendations
142+
rHandler->>backend: calls
143+
end
144+
note over User, backend: codewhispererPostprocessingLatency
145+
rect rgb(230, 230, 230, 0.5)
146+
backend->>rHandler: first response received
147+
rHandler->>User: show results
148+
end
149+
150+
backend->>rHandler: all the other responses
151+
rHandler->>User: add to already shown results
152+
```
153+
154+
### codewhispererCredentialFetchingLatency
155+
156+
How long it took to get the bearer token
157+
158+
```mermaid
159+
sequenceDiagram
160+
participant User
161+
participant invoke as Inline invoked
162+
participant rService as Recommendation Service
163+
participant rHandler as Recommendation Handler
164+
participant backend as CWSPR backend
165+
participant sdk as Create CWSPR SDK
166+
participant token as Toolkit auth
167+
168+
User->>invoke: Finished typing
169+
invoke->>rService: calls
170+
rService->>rHandler: calls
171+
rHandler->>sdk: calls
172+
173+
note over sdk, token: codewhispererCredentialFetchingLatency
174+
rect rgb(230, 230, 230, 0.5)
175+
sdk->>token: Start getting bearer token
176+
token->>sdk: Finished getting bearer token
177+
end
178+
sdk->>rHandler: Return client
179+
loop Get paginated recommendations
180+
rHandler->>backend: calls
181+
end
182+
183+
backend->>rHandler: first response received
184+
rHandler->>User: show results
185+
186+
backend->>rHandler: all the other responses
187+
rHandler->>User: add to already shown results
188+
```
189+
190+
### codewhispererPreprocessingLatency
191+
192+
How long it took to create the client and get ready to start sending getRecommendation API calls
193+
194+
```mermaid
195+
sequenceDiagram
196+
participant User
197+
participant invoke as Inline invoked
198+
participant rService as Recommendation Service
199+
participant rHandler as Recommendation Handler
200+
participant backend as CWSPR backend
201+
participant sdk as Create CWSPR SDK
202+
participant token as Toolkit auth
203+
204+
User->>invoke: Finished typing
205+
invoke->>rService: calls
206+
rService->>rHandler: calls
207+
rHandler->>sdk: calls
208+
209+
note over rHandler, token: codewhispererPreprocessingLatency
210+
rect rgb(230, 230, 230, 0.5)
211+
sdk->>token: Start getting bearer token
212+
token->>sdk: Finished getting bearer token
213+
sdk->>rHandler: Return client
214+
end
215+
loop Get paginated recommendations
216+
rHandler->>backend: calls
217+
end
218+
219+
backend->>rHandler: first response received
220+
rHandler->>User: show results
221+
222+
backend->>rHandler: all the other responses
223+
rHandler->>User: add to already shown results
224+
```
225+
226+
### codewhisperer_perceivedLatency duration
227+
228+
How long it took from when the user stopped pressing a key to when they were shown a response
229+
230+
```mermaid
231+
sequenceDiagram
232+
participant User
233+
participant invoke as Inline invoked
234+
participant rService as Recommendation Service
235+
participant rHandler as Recommendation Handler
236+
participant backend as CWSPR backend
237+
participant sdk as Create CWSPR SDK
238+
participant token as Toolkit auth
239+
240+
User->>invoke: Finished typing
241+
note over User, token: codewhisperer_perceivedLatency duration
242+
rect rgb(230, 230, 230, 0.5)
243+
invoke->>rService: calls
244+
rService->>rHandler: calls
245+
rHandler->>sdk: calls
246+
sdk->>token: Start getting bearer token
247+
token->>sdk: Finished getting bearer token
248+
sdk->>rHandler: Return client
249+
250+
loop Get paginated recommendations
251+
rHandler->>backend: calls
252+
end
253+
254+
backend->>rHandler: first response received
255+
rHandler->>User: show results
256+
257+
backend->>rHandler: all the other responses
258+
rHandler->>User: add to already shown results
259+
end
260+
```

0 commit comments

Comments
 (0)