How can I integrate Azure AD authentication into my Kubernetes Dashboard via oauth2-proxy?

The bounty expires in 5 days. Answers to this question are eligible for a +100 reputation bounty.
arjunbnair is looking for a canonical answer:

An answer that solves the whole problem is appreciated.

I have an Azure Kubernetes Cluster where I deployed the Kubernetes Dashboard following the instructions from https://github.com/kubernetes/dashboard using the configuration in my below values.yaml:

YAMLCopied
auth:
  role: auth
  image:
    repository: kubernetesui/dashboard-auth
    tag: 1.2.2
  scaling:
    replicas: 1
    revisionHistoryLimit: 10
  containers:
    ports:
      - name: auth
        containerPort: 8000
        protocol: TCP
    volumeMounts:
      - mountPath: /tmp
        name: tmp-volume
    # TODO: Validate configuration
    resources:
      requests:
        cpu: 100m
        memory: 200Mi
      limits:
        cpu: 250m
        memory: 400Mi
  automountServiceAccountToken: true
  volumes:
    # Create on-disk volume to store exec logs (required)
    - name: tmp-volume
      emptyDir: {}
  nodeSelector: {}
  # Labels & annotations for Auth related resources
  labels: {}
  annotations: {}
  serviceLabels: {}
  serviceAnnotations: {}


# API deployment configuration
api:
  role: api
  image:
    repository: kubernetesui/dashboard-api
    tag: 1.10.1
  scaling:
    replicas: 1
    revisionHistoryLimit: 10
  containers:
    ports:
      - name: api
        containerPort: 8000
        protocol: TCP
    volumeMounts:
      - mountPath: /tmp
        name: tmp-volume
    # TODO: Validate configuration
    resources:
      requests:
        cpu: 100m
        memory: 200Mi
      limits:
        cpu: 250m
        memory: 400Mi
  automountServiceAccountToken: true
  # Additional volumes
  # - name: dashboard-kubeconfig
  #   secret:
  #     defaultMode: 420
  #     secretName: dashboard-kubeconfig
  volumes:
    # Create on-disk volume to store exec logs (required)
    - name: tmp-volume
      emptyDir: {}
  nodeSelector: {}
  # Labels & annotations for API related resources
  labels: {}
  annotations: {}
  serviceLabels: {}
  serviceAnnotations: {}

# WEB UI deployment configuration
web:
  role: web
  image:
    repository: kubernetesui/dashboard-web
    tag: 1.6.0
  scaling:
    replicas: 1
    revisionHistoryLimit: 10
  containers:
    ports:
      - name: web
        containerPort: 8000
        protocol: TCP
    # Additional container arguments
    # Full list of arguments: https://github.com/kubernetes/dashboard/blob/master/docs/common/arguments.md
    # args:
    # - --system-banner="Welcome to the Kubernetes Dashboard"
    args: []
    # Additional container environment variables
    # env:
    # - name: SOME_VAR
    #   value: 'some value'
    env: []
    # Additional volume mounts
    # - mountPath: /kubeconfig
    #   name: dashboard-kubeconfig
    #   readOnly: true
    volumeMounts:
      # Create volume mount to store logs (required)
      - mountPath: /tmp
        name: tmp-volume
    # TODO: Validate configuration
    resources:
      requests:
        cpu: 100m
        memory: 200Mi
      limits:
        cpu: 250m
        memory: 400Mi
  automountServiceAccountToken: true
  volumes:
    # Create on-disk volume to store exec logs (required)
    - name: tmp-volume
      emptyDir: {}
  nodeSelector: {}
  # Labels & annotations for WEB UI related resources
  labels: {}
  annotations: {}
  serviceLabels: {}
  serviceAnnotations: {}

### Metrics Scraper
### Container to scrape, store, and retrieve a window of time from the Metrics Server.
### refs: https://github.com/kubernetes/dashboard/tree/master/modules/metrics-scraper
metricsScraper:
  enabled: true
  role: metrics-scraper
  image:
    repository: kubernetesui/dashboard-metrics-scraper
    tag: 1.2.1
  scaling:
    replicas: 1
    revisionHistoryLimit: 10
  containers:
    ports:
      - containerPort: 8000
        protocol: TCP
    args: []
    # Additional container environment variables
    # env:
    # - name: SOME_VAR
    #   value: 'some value'
    env: []
    # Additional volume mounts
    # - mountPath: /kubeconfig
    #   name: dashboard-kubeconfig
    #   readOnly: true
    volumeMounts:
      # Create volume mount to store logs (required)
      - mountPath: /tmp
        name: tmp-volume
    # TODO: Validate configuration
    resources:
      requests:
        cpu: 100m
        memory: 200Mi
      limits:
        cpu: 250m
        memory: 400Mi
    livenessProbe:
      httpGet:
        scheme: HTTP
        path: /
        port: 8000
      initialDelaySeconds: 30
      timeoutSeconds: 30
  automountServiceAccountToken: true
  # Additional volumes
  # - name: dashboard-kubeconfig
  #   secret:
  #     defaultMode: 420
  #     secretName: dashboard-kubeconfig
  volumes:
    - name: tmp-volume
      emptyDir: {}
  nodeSelector: {}
  # Labels & annotations for Metrics Scraper related resources
  labels: {}
  annotations: {}
  serviceLabels: {}
  serviceAnnotations: {}

## Optional Metrics Server sub-chart configuration
## Enable this if you don't already have metrics-server enabled on your cluster and
## want to use it with dashboard metrics-scraper
## refs:
##  - https://github.com/kubernetes-sigs/metrics-server
##  - https://github.com/kubernetes-sigs/metrics-server/tree/master/charts/metrics-server
metrics-server:
  enabled: false
  args:
    - --kubelet-preferred-address-types=InternalIP
    - --kubelet-insecure-tls

## Required Kong sub-chart with DBless configuration to act as a gateway
## for our all containers.
kong:
  enabled: true
  image:
    repository: kong
    tag: "3.6"
  ## Configuration reference: https://docs.konghq.com/gateway/3.6.x/reference/configuration
  env:
    dns_order: LAST,A,CNAME,AAAA,SRV
    plugins: 'off'
    nginx_worker_processes: 1
  ingressController:
    enabled: false
  manager:
    enabled: false
  dblessConfig:
    configMap: kong-dbless-config
  proxy:
    type: ClusterIP
    http:
      enabled: false

## Optional Cert Manager sub-chart configuration
## Enable this if you don't already have cert-manager enabled on your cluster.
cert-manager:
  enabled: false
  installCRDs: true

## Optional Nginx Ingress sub-chart configuration
## Enable this if you don't already have nginx-ingress enabled on your cluster.
nginx:
  enabled: false
  controller:
    electionID: ingress-controller-leader
    ingressClassResource:
      name: internal-nginx
      default: false
      controllerValue: k8s.io/internal-ingress-nginx
    service:
      type: ClusterIP

## Extra configurations:
## - manifests
## - predefined roles
## - prometheus
## - etc...
extras:
  # Extra Kubernetes manifests to be deployed
  # manifests:
  # - apiVersion: v1
  #   kind: ConfigMap
  #   metadata:
  #     name: additional-configmap
  #   data:
  #     mykey: myvalue
  manifests: []
  serviceMonitor:
    # Whether to create a Prometheus Operator service monitor.
    enabled: false
    # Here labels can be added to the serviceMonitor
    labels: {}
    # Here annotations can be added to the serviceMonitor
    annotations: {}
    # metrics.serviceMonitor.metricRelabelings Specify Metric Relabelings to add to the scrape endpoint
    # ref: https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#relabelconfig
    metricRelabelings: []
    # metrics.serviceMonitor.relabelings [array] Prometheus relabeling rules
    relabelings: []
    # ServiceMonitor connection scheme. Defaults to HTTPS.
    scheme: https
    # ServiceMonitor connection tlsConfig. Defaults to {insecureSkipVerify:true}.
    tlsConfig:
      insecureSkipVerify: true

The Dashboard works when I temporarily set the kong-proxy service to LoadBalancer, and I can access it via the public DNS https://arjun-test-k8s-dashboard-2.eastus.cloudapp.azure.com. However, it always prompts for a service token login.

I want to authenticate using Azure AD. Following the documentation at https://github.com/weinong/k8s-dashboard-with-aks-aad, I set up Azure AD integration using the exact same front-end IP and DNS. Below are the files I used:

oauth2.yaml

YAMLCopied
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: oauth2-proxy
  name: oauth2-proxy
  namespace: kubernetes-dashboard
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: oauth2-proxy
  template:
    metadata:
      labels:
        k8s-app: oauth2-proxy
    spec:
      containers:
      - args:
        - --provider=azure
        - --oidc-issuer-url=https://sts.windows.net/44444444-3333-222222-1111111111/
        - --email-domain=*
        - --http-address=0.0.0.0:4180
        - --azure-tenant=44444444-3333-222222-1111111111
        - --client-id=44444444-3333-222222-1111111111
        - --client-secret=44444444-3333-222222-1111111111
        - --cookie-secret=N5dG88dXYxzI7fgjgsM8tJWxaXZ9Zw==
        - --pass-access-token=true
        - --resource=6dae42f8-4368-4678-94ff-3960e28e3630
        - --set-xauthrequest=true
        image: docker.io/weinong/oauth2-proxy:v6.1.1-109-g49746b8
        imagePullPolicy: Always
        name: oauth2-proxy
        ports:
        - containerPort: 4180
          protocol: TCP

---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: oauth2-proxy
  name: oauth2-proxy
  namespace: kubernetes-dashboard
spec:
  ports:
  - name: http
    port: 4180
    protocol: TCP
    targetPort: 4180
  selector:
    k8s-app: oauth2-proxy

ingress.yaml

YAMLCopied
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
    nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
    nginx.ingress.kubernetes.io/auth-response-headers: 'X-Auth-Request-Access-Token'
    nginx.ingress.kubernetes.io/configuration-snippet: |
      auth_request_set $token $upstream_http_x_auth_request_access_token;
      proxy_set_header Authorization "Bearer $token";
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
  name: external-auth-oauth2
  namespace: kubernetes-dashboard
spec:
  rules:
  - host: arjun-test-k8s-dashboard-2.eastus.cloudapp.azure.com  # Replace with your valid FQDN
    http:
      paths:
      - path: /login
        pathType: Prefix
        backend:
          service:
            name: kubernetes-dashboard-kong-proxy
            port:
              number: 443
  tls:
  - hosts:
    - arjun-test-k8s-dashboard-2.eastus.cloudapp.azure.com  # Replace with your valid FQDN
    secretName: k8s-dashboard-ingress-tls-secret
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: oauth2-proxy
  namespace: kubernetes-dashboard
  annotations:
    nginx.ingress.kubernetes.io/proxy-buffer-size: "64k"
    nginx.ingress.kubernetes.io/proxy-buffers-number: "4"
spec:
  rules:
  - host: arjun-test-k8s-dashboard-2.eastus.cloudapp.azure.com  # Replace with your valid FQDN
    http:
      paths:
      - path: /oauth2
        pathType: Prefix
        backend:
          service:
            name: oauth2-proxy
            port:
              number: 4180
  tls:
  - hosts:
    - arjun-test-k8s-dashboard-2.eastus.cloudapp.azure.com  # Replace with your valid FQDN
    secretName: k8s-dashboard-ingress-tls-secret

To create the TLS secret, I ran:

BASHCopied
openssl req -x509 -newkey rsa:4096 -keyout tls.key -out tls.crt -days 365
openssl rsa -in tls.key -out tls.decrypted.key
kubectl create secret tls k8s-dashboard-ingress-tls-secret --cert=tls.crt --key=tls.decrypted.key --namespace=kubernetes-dashboard

When accessing https://arjun-test-k8s-dashboard-2.eastus.cloudapp.azure.com/, I get no response. I have tried the following troubleshooting steps:

1. Reverted the kong-proxy service to ClusterIP.
2. Added an Azure Load Balancing rule for TCP/443.
3. Configured a health probe targeting TCP/443.
4. Verified the front-end IP matches the DNS FQDN.
5. Allowed all traffic in the NSG.

Does anyone know how to resolve this issue? I would like the Dashboard to authenticate via Azure AD instead of using a manual service token login.