AWS EKS – Part 30 – Ingress with Application Load Balancer

The AWS Load Balancer Controller(LBC) introduced in the previous lesson also provides a feature to create an AWS Application Load Balancer(ALB) that can be used as an Ingress Controller for our Kubernetes cluster. In this lesson, we want to create Ingress resources and expose our applications outside the cluster to the Internet.

Follow our social media:

https://linkedin.com/in/ssbostan

https://linkedin.com/company/kubedemy

https://youtube.com/@kubedemy

https://telegram.me/kubedemy

Register for the FREE EKS Tutorial:

If you want to access the course materials, register from the following link:

Register for the FREE AWS EKS Black Belt Course

What is Kubernetes Ingress?

As the name suggests, it is a way to accept external traffic(ingress) to applications within the cluster. It provides a way to expose applications and accept external traffic to them. Applications are typically exposed over HTTP protocol “when using Ingress” and may have SSL/TLS termination, load balancing, virtual hosting, etc.

What is the Ingress Controller?

Ingress Controller is a web server or a reverse proxy service that understands Kubernetes Ingress resources, configures its core to handle incoming requests and passes them to the desired application within the cluster. Just think you have an Nginx web server running on the edge of your network and configure it to accept requests and forward them to the proper services based on the request’s host, headers, etc., and you may configure the SSL/TLS feature. The Kubernetes ingress controller is the same, but it is configured using Kubernetes Ingress resources. You can have multiple Ingress controllers in your cluster, but in most cases, one is enough.

AWS ALB vs. other Ingress Controllers:

Dozens of Ingress controllers, including Nginx, HAProxy, Traefik, Istio, Skipper, etc., are available for Kubernetes. Each may provide some specific features or fit for specific use cases, but mostly, they are the same. In most cases, you deploy the Ingress controller as Deployment within the cluster, and it’s responsible for managing Ingress resources and accepting external traffic. The most important thing here is you only have one Ingress controller that manages all the requests. When using AWS Load Balancer Controller(LBC) to create Application Load Balancers for Ingress resources, it creates an Application Load Balancer(ALB) for every Ingress resource you have within the cluster, which causes you a big bill at the end of the month. So, You must share the same Application Load Balancer with different Ingress resources to prevent this behaviour.

AWS Ingress Controller setup procedure:

• Install the AWS Load Balancer Controller into the cluster.
• Expose the application using an Ingress resource and ALB service.
• Share the Load Balancer between different Ingress resources.
• Work with LBC Annotations for Ingress resources.

Step 1 – Install AWS LBC Ingress Controller:

To install the AWS LBC, follow the procedure mentioned in the previous lesson:

AWS EKS – Part 29 – Setup and Work with AWS Network Load Balancer

You must be able to see the ALB Ingress class within the cluster:

kubectl get ingressclasses

Step 2 – Expose Kubernetes Application:

Assume we have an application exposed in the cluster using a ClusterIP service:

To expose the application outside the cluster using an ALB Load Balancer, create an Ingress resource for the service and use alb as the Ingress class name:

cat <<EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
  labels:
    app: nginx
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  ingressClassName: alb
  rules:
    - host: nginx.20302040.xyz
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nginx
                port:
                  number: 80
EOF

This manifest will ask the LBC to create an Internet-facing “publicly available over the Internet” ALB Load Balancer and use the Pod IP addresses to send the traffic from the Load Balancer to the application instances. As mentioned in the previous lesson, you can only use ip target type when using AWS VPC CNI, and for other CNIs, instance type must be used, and the applications must be exposed using NodePort Services.

It takes a couple of minutes for the ALB to be created.

You will find the following flow if you check the ALB resource within your AWS account. Based on what we requested, the traffic comes to the external load balancer on port 80; it checks for the first rule as we asked for, “the Host header must be our domain”, and it sends the requests to the first target group, which contains Pod IP Addresses.

Setup your domain and create a CNAME record to the provided address:

Now, you can access your application over the Internet.

Step 3 – Share ALB for multiple Kubernetes Ingresses:

The AWS Load Balancer Controller provides a way to group multiple Ingress resources using the same AWS Application Load Balancer. To do so, we must use two annotations to group our Ingresses and set their order on the ALB rules:

alb.ingress.kubernetes.io/group.name: NAME
alb.ingress.kubernetes.io/group.order: "ORDER"

The group NAME must be lowercase alphanumeric, dash and dot, and it must start and end with alphanumeric. The group ORDER is between -1000 and 1000, and the lower number has more priority. So, an Ingress rule with an order of -1000 will be checked first. The default order is 0 if not provided. In the case of having multiple resources in the same order, they will be sorted lexicographically by the Ingress namespace/name.

Important notice: Sharing the same Application Load Balancer may cause a security risk, as anyone with the privilege to create/modify Kubernetes Ingress resources can compromise the load balancer rules by adding a rule with more priority and sending the traffic to a malicious application. Note that the ALB can be shared within multiple namespaces. So, be careful when sharing the load balancer; setup the proper access boundaries and use OPA to restrict Kubernetes users from using specific groups.

Step 4 – AWS LBC Annotations for Ingress:

The AWS Load Balancer Controller supports many Kubernetes annotations for ALB Ingress to set the load balancer behaviour. Here are some of the most important ones. You can also check them here in the official documentation.

AWS Resource Tagging:

You can add custom tags to the created load balancer using the following annotation.

alb.ingress.kubernetes.io/tags: owner=kubedemy, env=test

Use different security groups:

AWS LBC creates at least two security groups, “frontend and backend”, for the load balancer. You can change them using the following annotation.

alb.ingress.kubernetes.io/security-groups: sg1-id, sg2-id, sg3-id

Change load balancer ports:

The ALB resource listens to ports 80 and 443 “if TLS is enabled for the ingress,” and you can change them using the following annotation.

alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 8080}, {"HTTPS":8443}]'

SSL/TLS Redirection:

You can redirect HTTP requests to HTTPS using the following annotation. This annotation enables the SSL redirect and mentions the target port for SSL/TLS requests.

alb.ingress.kubernetes.io/ssl-redirect: "443"

SSL/TLS certificate using AWS ACM:

The following annotation helps you to provide the certificate ARN to be used for the ingress. It’s only effective when enabling the SSL/TLS feature.

alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:eu-west-2:xxxxx:certificate/xxxxxxx

Custom health check port and path:

Using the following annotations, you can customise the health check port, path, protocol, timeout, threshold, and success codes for each ingress you have.

alb.ingress.kubernetes.io/healthcheck-port: "8081"
alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
alb.ingress.kubernetes.io/healthcheck-path: /health
alb.ingress.kubernetes.io/healthcheck-interval-seconds: "15"
alb.ingress.kubernetes.io/healthcheck-timeout-seconds: "5"
alb.ingress.kubernetes.io/healthy-threshold-count: "1"
alb.ingress.kubernetes.io/unhealthy-threshold-count: "2"
alb.ingress.kubernetes.io/success-codes: 200-300

Conclusion:

AWS Application Load Balancer is a high-performance and reliable load balancer that can be used as an Ingress Controller for Kubernetes clusters. Although it can be used on all CNIs, when using AWS VPC CNI, it can send traffic directly to the Pods instead of instances, which increases performance and reduces latency. The only thing you need to be aware of is the cost of using different ALBs for each Ingress resource you have, which can be managed through load balancer grouping and sharing the same ALB across multiple Ingress resources, but you need to secure the environment using OPA policies, Kyverno, etc., to prevent possible breaches and rule compromisation.

If you like this series of articles, please share them and write your thoughts as comments here. Your feedback encourages me to complete this massively planned program. Just share them and provide feedback. I’ll make you an AWS EKS black belt.

Follow my LinkedIn https://linkedin.com/in/ssbostan

Follow Kubedemy’s LinkedIn https://linkedin.com/company/kubedemy

Follow Kubedemy’s Telegram https://telegram.me/kubedemy