@@ -277,10 +277,10 @@ past two and a half weeks, but after successfully deploying version 3.6.0 of
277
277
the `baker` service this morning, she's escaped early to try to unwind a bit.
278
278
279
279
Her shoulders are just starting to unknot when her phone pings with a text
280
- from Charlie , down in the NOC. Waterproof phones are a blessing, but also a
280
+ from Chihiro , down in the NOC. Waterproof phones are a blessing, but also a
281
281
curse.
282
282
283
- **Charlie **: _Hey Ana. Things are still running, more or less, but latencies
283
+ **Chihiro **: _Hey Ana. Things are still running, more or less, but latencies
284
284
on everything in the `baker` namespace are crazy high after your last rollout,
285
285
and `baker` itself has a weirdly high load. Sorry to interrupt you on the lake
286
286
but can you take a look? Thanks!!_
@@ -297,30 +297,30 @@ duplicates? Ana checks her HTTPRoute again, though she's pretty sure you
297
297
can't configure retries there, and finds nothing. But it definitely looks like
298
298
clients are retrying when they shouldn’t be.
299
299
300
- She pings Charlie .
300
+ She pings Chihiro .
301
301
302
- **Ana**: _Hey Charlie . Something weird is up, looks like requests to `baker`
302
+ **Ana**: _Hey Chihiro . Something weird is up, looks like requests to `baker`
303
303
are failing but getting retried??_
304
304
305
305
A minute later they answer.
306
306
307
- **Charlie **: 🤷 _Did you configure retries?_
307
+ **Chihiro **: 🤷 _Did you configure retries?_
308
308
309
309
**Ana**: _Dude. I don’t even know how to._ 😂
310
310
311
- **Charlie **: _You just attach a RetryPolicy to your HTTPRoute._
311
+ **Chihiro **: _You just attach a RetryPolicy to your HTTPRoute._
312
312
313
313
**Ana**: _Nope. Definitely didn’t do that._
314
314
315
315
She types `kubectl get retrypolicy -n baker` and gets a permission error.
316
316
317
317
**Ana**: _Huh, I actually don’t have permissions for RetryPolicy._ 🤔
318
318
319
- **Charlie **: 🤷 _Feels like you should but OK, guess that can’t be it._
319
+ **Chihiro **: 🤷 _Feels like you should but OK, guess that can’t be it._
320
320
321
321
Minutes pass while both look at logs.
322
322
323
- **Charlie **: _I’m an idiot. There’s a RetryPolicy for the whole namespace –
323
+ **Chihiro **: _I’m an idiot. There’s a RetryPolicy for the whole namespace –
324
324
sorry, too many policies in the dashboard and I missed it. Deleting that since
325
325
you don’t want retries._
326
326
@@ -332,17 +332,17 @@ through them: there’s one for every single service in the `baker` namespace.
332
332
333
333
**Ana**: _PUT IT BACK!!_
334
334
335
- **Charlie **: _Just did. Be glad you couldn't hear all the alarms here._ 😕
335
+ **Chihiro **: _Just did. Be glad you couldn't hear all the alarms here._ 😕
336
336
337
337
**Ana**: _What the hell just happened??_
338
338
339
- **Charlie **: _At a guess, all the workloads in the `baker` namespace actually
339
+ **Chihiro **: _At a guess, all the workloads in the `baker` namespace actually
340
340
fail a lot, but they seem OK because there are retries across the whole
341
341
namespace?_ 🤔
342
342
343
343
Ana's blood runs cold.
344
344
345
- **Charlie **: _Yeah. Looking a little closer, I think your `baker` rollout this
345
+ **Chihiro **: _Yeah. Looking a little closer, I think your `baker` rollout this
346
346
morning would have failed without those retries._ 😕
347
347
348
348
There is a pause while Ana's mind races through increasingly unpleasant
@@ -351,40 +351,40 @@ possibilities.
351
351
**Ana**: _I don't even know where to start here. How long did that
352
352
RetryPolicy go in? Is it the only thing like it?_
353
353
354
- **Charlie **: _Didn’t look closely before deleting it, but I think it said a few
354
+ **Chihiro **: _Didn’t look closely before deleting it, but I think it said a few
355
355
months ago. And there are lots of different kinds of policy and lots of
356
356
individual policies, hang on a minute..._
357
357
358
- **Charlie **: _Looks like about 47 for your chunk of the world, a couple hundred
358
+ **Chihiro **: _Looks like about 47 for your chunk of the world, a couple hundred
359
359
system-wide._
360
360
361
361
**Ana**: 😱 _Can you tell me what they’re doing for each of our services? I
362
362
can’t even_ look _at these things._ 😕
363
363
364
- **Charlie **: _That's gonna take awhile. Our tooling to show us which policies
364
+ **Chihiro **: _That's gonna take awhile. Our tooling to show us which policies
365
365
bind to a given workload doesn't go the other direction._
366
366
367
367
**Ana**: _...wait. You have to_ build tools _to know if retries are turned on??_
368
368
369
369
Pause.
370
370
371
- **Charlie **: _Policy attachment is more complex than we’d like, yeah._ 😐
372
- _Look, how ‘bout roll back your `baker` change for now? We can get together in
371
+ **Chihiro **: _Policy attachment is more complex than we’d like, yeah._ 😐
372
+ _Look, how about roll back your `baker` change for now? We can get together in
373
373
the morning and start sorting this out._
374
374
375
375
Ana shakes her head and rolls back her edits to the `baker` Deployment, then
376
376
sits looking out over the lake as the deployment progresses.
377
377
378
378
**Ana**: _Done. Are things happier now?_
379
379
380
- **Charlie **: _Looks like, thanks. Reckon you can get back to your sailboard._ 🙂
380
+ **Chihiro **: _Looks like, thanks. Reckon you can get back to your sailboard._ 🙂
381
381
382
382
Ana sighs.
383
383
384
384
**Ana**: _Wish I could. Wind’s died down, though, and it'll be dark soon.
385
385
Just gonna head home._
386
386
387
- **Charlie **: _Ouch. Sorry to hear that._ 😐
387
+ **Chihiro **: _Ouch. Sorry to hear that._ 😐
388
388
389
389
One more look out at the lake.
390
390
@@ -401,13 +401,13 @@ listed in increasing order of desirability:
401
401
- _Which_ Policy is (or Policies are) affecting a particular object
402
402
- _What_ settings in the Policy are affecting the object.
403
403
404
- In the parable, if Ana and Charlie had known that there were Policies affecting
404
+ In the parable, if Ana and Chihiro had known that there were Policies affecting
405
405
the relevant object, then they could have gone looking for the relevant Policies
406
406
and things would have played out differently. If they knew which Policies, they
407
407
would need to look less hard, and if they knew what the settings being applied
408
408
were, then the parable would have been able to be very short indeed.
409
409
410
- (There’s also another use case to consider, in that Charlie should have been able
410
+ (There’s also another use case to consider, in that Chihiro should have been able
411
411
to see that the Policy on the namespace was in use in many places before deleting
412
412
it.)
413
413
@@ -432,7 +432,7 @@ ways at an API level to the Application Developer's concerns.
432
432
433
433
An important note here is that a key piece of information for Policy Admins and
434
434
Cluster Operators is “How many things does this Policy affect?”. In the parable,
435
- this would have enabled Charlie to know that deleting the Namespace Policy would
435
+ this would have enabled Chihiro to know that deleting the Namespace Policy would
436
436
affect many other people than just Ana.
437
437
438
438
# ## Problems we need to solve
0 commit comments