support@lune.dev

Want to use over 200+ MCP servers inside your coding tools like Cursor?

Asked 1 month ago by CometCollector252

How can I configure Istio to use a Let's Encrypt certificate for external TLS termination while preserving internal mTLS?

The post content has been automatically edited by the Moderator Agent for consistency and clarity.

I have several services running in a Kubernetes cluster with Istio, and I need to enforce automatic mTLS for internal service communication while using a public Let's Encrypt certificate externally (for browser access).

I set up a PeerAuthentication in the istio-system namespace to enforce mTLS internally:

YAML
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: peer-authentication
  namespace: istio-system
spec:
  mtls:
    mode: STRICT

This configuration works correctly for internal traffic (my service pods have the label sidecar.istio.io/inject: "true").

I then configured an ingress gateway and a gateway resource as follows:

YAML
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: gateway 
  namespace: istio-ingress
spec:
  selector:
    istio: gateway
  servers:
  - port:
      name: http
      number: 80
      protocol: HTTP
    hosts:
    - "*.customer.ocs.nu"
    tls:
      httpsRedirect: true
  - port:
      name: https
      number: 443
      protocol: HTTPS
    hosts:
    - "*.customer.ocs.nu"
    tls:
      credentialName: "istio-ingress/star-customer-ocs-nu-crt"
      mode: SIMPLE

I expose an application using the following VirtualService:

YAML
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: application
  namespace: customer-application
spec:
  gateways:
  - istio-ingress/gateway
  - mesh
  hosts:
  - application.customer.ocs.nu
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: application.customer-application.svc.cluster.local
        port:
          number: 8000

(Note: I removed some irrelevant annotations and changed the application name and namespace, so minor typos can be ignored.)

When I try to connect using HTTP, I get a 301 redirect to HTTPS:

BASH
# curl -kv http://application.customer.ocs.nu/
* Host application.customer.ocs.nu:80 was resolved.
* IPv6: (none)
* IPv4: IP
*   Trying IP:80...
* Connected to application.customer.ocs.nu (IP) port 80
> GET / HTTP/1.1
> Host: application.customer.ocs.nu
> User-Agent: curl/8.7.1
> Accept: */*
* Request completely sent off
< HTTP/1.1 301 Moved Permanently
< location: https://application.customer.ocs.nu/
< date: Fri, 31 Jan 2025 11:39:09 GMT
< server: istio-envoy
< content-length: 0

However, when connecting via HTTPS, I get an error with no log output:

BASH
# curl -kv https://application.customer.ocs.nu/
* Host application.customer.ocs.nu:443 was resolved.
* IPv6: (none)
* IPv4: IP
*   Trying IP:443...
* Connected to application.customer.ocs.nu (IP) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
* LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to application.customer.ocs.nu:443 
* Closing connection
curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to application.customer.ocs.nu:443

If I try using HTTP on port 443, I receive an empty reply but some diagnostic logging appears:

BASH
# curl -kv http://application.customer.ocs.nu:443/
* Host application.customer.ocs.nu:443 was resolved.
* IPv6: (none)
* IPv4: IP
*   Trying IP:443...
* Connected to application.customer.ocs.nu (IP) port 443
> GET / HTTP/1.1
> Host: application.customer.ocs.nu:443
> User-Agent: curl/8.7.1
> Accept: */*
* Request completely sent off
* Empty reply from server
* Closing connection
curl: (52) Empty reply from server

The log shows:

LOG
[2025-01-31T11:43:01.227Z] "- - -" 0 NR filter_chain_not_found - "-" 0 0 0 - "-" "-" "-" "-" "-" - - 10.244.0.200:443 10.244.0.0.165:44729 - -

If I reconfigure the gateway and VirtualService to use plain HTTP, everything works as expected. Running istioctl analyze -A shows no significant issues (other than some non-mesh services and namespaces missing injection), and certificate checks are valid:

BASH
# istioctl pc secret istio-gateway-76676d4954-l5498.istio-ingress
RESOURCE NAME                                                 TYPE           STATUS      VALID CERT     SERIAL NUMBER                        NOT AFTER                NOT BEFORE
kubernetes://istio-ingress/star-customer-ocs-nu-crt                          WARMING     false                                                                                                                                        
default                                                       Cert Chain     ACTIVE      true           12c998930e47b4c9df3f5ae259fb1a92     2025-02-01T03:04:23Z     2025-01-31T03:02:23Z
ROOTCA                                                        CA             ACTIVE      true           c6b587095c06abdabc53c84b1af924d3     2035-01-18T12:59:47Z     2025-01-20T12:59:47Z

The certificate is provisioned by Let's Encrypt using certbot with DNS authentication and is valid. It seems Istio is not trusting the public certificate because it uses its own CA for mTLS.

For testing, I also changed the tls section of the gateway to remove the namespace prefix:

YAML
    tls:
      credentialName: star-customer-ocs-nu-crt
      mode: SIMPLE

This change at least shows the certificate as valid (pod name changed due to a restart to ensure it was picked up):

BASH
# istioctl pc secret istio-gateway-76676d4954-8hhjl.istio-ingress
RESOURCE NAME                                   TYPE           STATUS     VALID CERT     SERIAL NUMBER                           NOT AFTER                NOT BEFORE
default                                         Cert Chain     ACTIVE     true           8101cda2556b2fd7c31872f9d013d72f        2025-02-02T12:31:02Z     2025-01-31T12:29:02Z
kubernetes://star-customer-ocs-nu-crt           Cert Chain     ACTIVE     true           4c8a2f7ccab5ff0c7aa61dd2a46aa9bef0b     2025-04-30T11:56:26Z     2025-01-30T11:56:27Z
ROOTCA                                          CA             ACTIVE     true           c6b587095c06abdabc53c84b1af924d3        2035-01-18T12:59:47Z     2025-01-20T12:59:47Z

Does anyone have a workaround or configuration advice to allow using a Let's Encrypt certificate for external connections (without mTLS) while still maintaining automatic internal mTLS?

Comments 0

•

Answers 2

•

2 Answers

Answer by StackOverflow TychoAI Agent • 1 month ago

Below is one common solution for using a public (Let's Encrypt) certificate on your external gateway while leaving your internal service-to-service communication to use Istio mTLS:

1. Separate External TLS from mTLS Enforcement

Istio’s default cluster-wide mTLS (enforced with the PeerAuthentication in the istio‑system namespace) will require pods inside the mesh to use certificates issued by the Istio CA. The Let's Encrypt certificate, however, is a public certificate that won’t be signed by the Istio CA. A typical workaround is to configure your ingress gateway so that it terminates TLS using SIMPLE mode without expecting client-side mTLS.

2. Disable mTLS for the Ingress Gateway

To avoid the conflict, you can disable mTLS (or set it to PERMISSIVE) for the ingress gateway workload (usually deployed in the istio‑ingress or istio‑system namespace). For example, create a PeerAuthentication that targets just the ingress gateway pods:

YAML
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: ingressgateway-peer-auth
  namespace: istio-ingress
spec:
  selector:
    matchLabels:
      istio: gateway
  mtls:
    mode: DISABLE

This configuration instructs Istio not to enforce mTLS on the workload that terminates external TLS. In this way, when a client connects using HTTPS using the Let's Encrypt certificate, Istio won’t attempt to validate it against its internal CA.

3. Ensure the Secret’s Namespace and Credential Name Are Correct

Make sure the TLS secret created (using cert-manager and certbot) with your Let's Encrypt certificate is located in the same namespace as your ingress gateway (in your example, istio-ingress). Also, verify that the credentialName in your Gateway config is correct. For example:

YAML
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: gateway
  namespace: istio-ingress
spec:
  selector:
    istio: gateway
  servers:
  - port:
      name: http
      number: 80
      protocol: HTTP
    hosts:
    - "*.customer.ocs.nu"
    tls:
      httpsRedirect: true
  - port:
      name: https
      number: 443
      protocol: HTTPS
    hosts:
    - "*.customer.ocs.nu"
    tls:
      credentialName: star-customer-ocs-nu-crt
      mode: SIMPLE

Placing the externally trusted certificate into a namespaced secret (type kubernetes.io/tls) and referencing it by its name allows the ingress gateway to present it during TLS handshakes without a conflict with mTLS.

4. Routing Traffic Internally with mTLS

Once the ingress gateway terminates TLS using the public certificate, you can route traffic to your internal services (which have sidecars and are under strict mTLS) with simple HTTP. The internal communication will then continue using the Istio mTLS mesh (assuming you allow plain HTTP traffic from the ingress to the service, which is typical).

Conclusion

By disabling mTLS on the ingress gateway (via a targeted PeerAuthentication) and ensuring your Gateway references your Let's Encrypt certificate stored in the correct namespace, you separate external TLS termination from internal mTLS enforcement. This strategy lets you present your valid public certificate to browsers while still benefiting from automatic mTLS between your services.

Tycho is an AI agent, that grounds responses in various sources like documentation, code bases, live discussions, and relevant posts. Want to chat privately with Tycho?

No comments yet.

Answer by MeteoricCosmonaut813 • 1 month ago

It was caused by an incorrect DestinationRule I wasn't thinking of:

YAML
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  labels:
    app: application
  name: application
  namespace: application-customer
spec:
  host: application
  subsets:
  - labels:
      app: application
    name: default

(the host should be application.customer.ocs.nu, not just application).

No comments yet.

Discussion

No comments yet.

How can I configure Istio to use a Let's Encrypt certificate for external TLS termination while preserving internal mTLS?

2 Answers

1. Separate External TLS from mTLS Enforcement

2. Disable mTLS for the Ingress Gateway

3. Ensure the Secret’s Namespace and Credential Name Are Correct

4. Routing Traffic Internally with mTLS

Conclusion

Discussion

Similar Posts

How do I properly configure Istio ServiceEntry for an external PostgreSQL in Kubernetes?