Client#watch returns an unhandled error message on Forbidden response

[pre]

Let

  lolcats = K8s::Client.in_cluster_config
    .api('something/v1')
    .resource('lolcats', namespace: 'default')

Calling lolcats.watch will give misleading error message caused by permission denied:

[applikator-6469bf9b68-n5k2p applikator] I, [2019-01-09T09:09:30.623639 #1]  INFO -- K8s::Transport<https://10.33.0.1:443>: GET /apis/something/v1 => HTTP 200: <K8s::API::MetaV1::APIResourceList> in 0.014s
[applikator-6469bf9b68-n5k2p applikator] /usr/local/bundle/gems/dry-struct-0.5.1/lib/dry/struct/class_interface.rb:208:in `rescue in new': [K8s::API::MetaV1::WatchEvent.new] :type is missing in Hash input (Dry::Struct::Error) (Excon::Error::Socket)

Calling lolcats.list gives an error message which gives you out a reason:

WARN -- K8s::Transport<https://10.33.0.1:443>: GET /apis/something/v1/namespaces/default/lolcats => HTTP 403 Forbidden in 0.004s
[operator-6478f68bcd-t6w75 operator] /usr/local/bundle/gems/k8s-client-0.6.4/lib/k8s/transport.rb:211:in `parse_response': GET /apis/something/v1/namespaces/default/lolcats => HTTP 403 Forbidden: lolcats.xyz is forbidden: User "system:serviceaccount:kube-system:operator" cannot list resource "lolcats" in API group "xyz" in the namespace "default" (K8s::Error::Forbidden)

It took me some time to figure out the reason without a reasonable failure message.

[kke]

I believe the problem here is that Excon's response_block is called even though the server responds with an error. There's no way to access the response or response headers from inside the response_block to determine this.

With the current excon, I believe the only options are:

1. do a HEAD request to the watch endpoint first, I don't know if kube understands this
2. try to guess from the JSON content that we're dealing with an error before calling the WatchEvent.new with the json-doc.

(see excon/excon#628)

[vitobotta]

Hi @kke @pre I am trying this gem and am having exactly this problem. How can I fix? Thanks!

[vitobotta]

Got rid of the error by specifying the correct service account. Although the watch doesn't seem to work... it only reports the events when I shutdown the program...

[kke]

Did you try from master branch? (gem 'k8s-client', github: 'kontena/k8s-client', branch: 'master' in gemfile)

May not make any difference, but maybe it does.

[vitobotta]

Hi @kke I tried the master branch and I had to require watch_events manually for some reason. I had the same problem though with the watch not firing when events happen. What can I try? :)