Prometheus Not Discovering ArgoCD ServiceMonitor Targets in Kubernetes

Below is one common cause and its solution when ServiceMonitors aren’t being discovered by Prometheus:

What’s Happening

Even though you set
    serviceMonitorSelectorNilUsesHelmValues: false,
your Prometheus CR (custom resource) now has an empty ServiceMonitor selector (i.e. {}). When the selector is empty, Prometheus won’t automatically match any ServiceMonitor—even if your ArgoCD ServiceMonitor is deployed—because there’s no label matching criteria defined.

How to Resolve It

You must explicitly configure the Prometheus CR to select ServiceMonitors that have the labels you want. In other words, update your Prometheus configuration so that its serviceMonitorSelector (and possibly serviceMonitorNamespaceSelector) matches the labels on the ArgoCD ServiceMonitor.

For example, if your ArgoCD ServiceMonitor contains the label:

YAMLCopied
metadata:
  labels:
    app: argocd

then update your Prometheus CR as follows:

YAMLCopied
spec:
  serviceMonitorSelector:
    matchLabels:
      app: argocd
  serviceMonitorNamespaceSelector:
    matchNames:
      - <namespace-where-argocd-is-deployed>

Replace <namespace-where-argocd-is-deployed> with the correct namespace.

Additional Checks

• Verify the ServiceMonitor Resource: Run
  kubectl get servicemonitors -A
  to ensure the ArgoCD ServiceMonitor is actually created with the expected labels.

• Operator Logs: Check the logs of the Prometheus operator; they’ll often indicate if no ServiceMonitor matches the selector.

• Namespace Selection: Make sure that if ArgoCD’s ServiceMonitor lives in a different namespace, the Prometheus CR’s serviceMonitorNamespaceSelector allows cross-namespace discovery. If you want to include all namespaces, you might set it to an empty spec, or explicitly list them.

After these changes, Prometheus should pick up the ArgoCD ServiceMonitor, and you’ll be able to see its targets in the Prometheus UI, which in turn will be reflected on your ArgoCD Grafana Dashboard.

This way, your monitoring stacks should work in tandem as intended.