#sre - March 2020

prometheus-operator users: how much memory have you seen your prometheus operator consume?

Hi folks, what’s the right way to handle the KubeletDown alerts that comes with prometheus operator on a public cloud where nodes gets replaced at times ?

    - alert: KubeletDown
      annotations:
        message: Kubelet has disappeared from Prometheus target discovery.
        runbook_url: <https://github.com/kubernetes-monitoring/kubernetes-mixin/tree/master/runbook.md#alert-name-kubeletdown>
      expr: |
        absent(up{job="kubelet", metrics_path="/metrics"} == 1)
      for: 15m
      labels:
        severity: critical

Adding @U010XGY9B46 bot

What's your preferred APM platform (no Appdynamics) ? Need container support, .net , Java, more, etc? I want to simplify telemetry and monitoring metrics to a central service and give business a self service telemetry metrics source so it's all centralized.

I want a system ideally that automatically pulls in aws tags on instances to, do I can stop writing complicated chocolatey packages for configuring the app.

Right now gut feeling is SignalFX ( can manage with terraform to), datadog are the promising solutions.