endpints missing when try to monitor custom node

[ghost]

Hello guys,

I am using kube-prometheus version 2.11.0 and try to monitor nodes running centos7 that not the worker nodes of the k8s cluster with service monitor. I have installed and run node_exporter on each node and configured: endpoints, service and service monitoring.
Below is my endpoints configure:
`apiVersion: v1
kind: Endpoints
metadata:
labels:
run: node-exporter-test
name: node-exporter-test
namespace: monitoring
subsets:

• addresses:
  • ip: 192.168.x.x
    ports:
  • name: metrics
    port: 9100
    protocol: TCP
    `
    However, It happened to me that, after few hours I will see 0/0 up from the "targets" tab and if I check the endpoints information using kubectl get endpoints node-exporter-test -o yaml -n monitoring command, I will see configure information below "subset"
    will be missing from the yaml print, any advice?

[ArthurSens]

Hey, if you see no targets on the Prometheus's target page, I'd say that your ServiceMonitor object is not configured correctly.
Have you tried the troubleshooting guide?

https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/troubleshooting.md

[ghost]

Hello ArthurSens,
Thanks for your quick reply. The fact is, I can see the target is up at the first few hours after I confgured endpoints, service and service monitor. When issue happens, I will see (0/0 up) at the "Targets" page and also the endpoints configure information is missing when I try to get it by "kubectl get endpoints endpoints_name -o yaml " . The annotation part will keep the last-configured information though.