AWS Fargate is a serverless pay-as-you-go compute engine that lets you deploy applications without managing servers. It can be used on both ECS “to run tasks” and EKS “to run Pods” services. As it’s provided in the PAYG model, it reduces your running cost, and you can even mix it with Spot and Graviton instances to reduce more than ever. In this lesson, we’ll work on Fargate to deploy Kubernetes Pods in AWS EKS clusters.
To allow Fargate instances to join the EKS cluster, the EKS cluster should be deployed with a private API endpoint, or the public API endpoint should be accessible over the internet. If you want to use the public endpoint, the Fargate instance should be deployed behind a NAT gateway to be able to communicate.
Although you provide an IAM role to Fargate instances, the Fargate Pods cannot consume that IAM role. If the Pod needs to get access to the AWS account, you must use the IAM Role for Service Accounts (IRSA) method.
Kubernetes affinity and antiAffinity options are ignored on Fargate. In Fargate, one node equals one Pod. 1 Instance (Worker node) = 1 Pod.
You can’t change the Fargate profiles after creation. To change, a new one should be created, and the old one should be removed. If the Fargate profile gets deleted, all the running Pods which are using that profile will be deleted and rescheduled again.
You can’t use dynamic storage provisioning with Fargate nodes “as the CSI node driver cannot be deployed there”, but you can use statically provisioned volumes in Fargate Pods. So, Fargate is not a good choice for StatefulSet applications with volumeClaimTemplates option. Remember: EBS volumes are not available at all, but you can easily mount EFS volumes in Fargate instances/Pods.
Fargate instances and Pods deployment procedure:
Create private subnets and NAT Gateway for Fargate instances.
Create a Fargate IAM role for new EKS Fargate profiles.
Create separate Fargate profiles for each private subnet.
Deploy a Kubernetes Pod and test the environment.
Play with Fargate Pods and discover more about Fargate in EKS clusters.
Investigate Fargate inside and outside of the Kubernetes cluster.
Step 1 – Create private subnets and NAT gateway:
To make Fargate work, we need some private subnets as well as a NAT gateway. We deployed all of them as a part of cluster requirements in the first lesson.
AWS deploys a fully managed VM behind the scenes. The kubelet component on the managed machine needs to join the cluster. To do so, we must create a new IAM role for our Fargate instances and attach a proper policy to that.
Important: AWS will add the IAM role ARN in aws-auth configmap.
Important: It’s better to create a separate Fargate IAM role for each cluster.
To create a trust policy for the Fargate IAM role, run this:
Now attach the AmazonEKSFargatePodExecutionRolePolicy policy to the role. This policy allows Fargate instance to communicate and download images from AWS ECR service. To attach the policy, run the following command:
aws iam attach-role-policy \
--policy-arn arn:aws:iam::aws:policy/AmazonEKSFargatePodExecutionRolePolicy \
Step 3 – Create Fargate Profiles for the EKS cluster:
You can create a Fargate profile and assign a couple of private subnets “in different zones” to it. Normally, Fargate instances are spread across different subnets and zones, but not in an equal manner, and there is a chance to deploy all of the application replicas into the same subnet/zone. To avoid this behaviour, which can cause application downtime on AZ failures, the best practice is to create a Fargate profile per zone. In our example, we have three private subnets and will create three Fargate profiles.
Fargate uses Namespace and Label to mutate the Pod manifest.
You can use * and ? wildcards in both of them.
If the Pod matches nothing, it remains in the Pending state.
If the Pod matches more than one profile, it uses the first one alphabetically.
After matching to a profile, Fargate mutates the Pod and changes the scheduler to fargate-scheduler instead of default-scheduler.
To use a specific profile, set eks.amazonaws.com/fargate-profile label.
You are allowed to provide up to 5 labels for each Fargate profile.
To create Fargate profiles for the EKS cluster, run these commands:
If you don’t provide any resource requests, the minimum resources will be provisioned. The minimum one is 0.25vCPU and 0.5GB memory.
Resource limits should be the same as resource requests because, in Fargate, one node is one Pod. So, a different amount of limit doesn’t make sense.
As every Fargate node has only one Pod, there is no Pod Eviction process.
All Fargate Pods will be run in the Guaranteed QoS class.
If you run a Pod with minimum resources “0.25vCPU 0.5GB” and check the node capacity, you will find that the amount of allocated resources is different. It’s because Kubernetes components also need some resources. An annotation CapacityProvisioned on the Pod shows the exact amount of resources available to the Pod.
Pod resource calculation:
To calculate the Pod resource requests, follow this algorithm:
Find the highest resources needed in initContainers containers.
Sum all resources needed by normal containers.
Take the biggest number from the above calculations.
Sum the biggest value + 256MB for Kubernetes components.
The final value will round up to the next available one in the above table.
Select Fargate profile for Pod:
As mentioned, if the Pod matches more than one Fargate profile, the first one will be chosen automatically “alphabetically”. You can also set an exact Fargate profile to run your Pod. To do so, set the following label to the Pod:
Step 6 – Investigate Fargate in/out of EKS Cluster:
For each Node/Pod, one network interface is deployed in the selected subnet.
Fargate Mutating Webhook:
When you create a Pod in the proper namespaces and set the required labels, the Fargate Mutating Webhook changes the Pod scheduler field to fargate-scheduler. Fargate Scheduler is a controller which is run as a managed service along with the cluster control plane. So you don’t need to deploy or manage it.
Here are the results of the previous commands; we need them in the next articles:
In this lesson, you learned many things about EKS+Fargate. In future, you will learn more about monitoring, logging, storage and networking for Fargate instances. Fargate is a cool AWS feature that can minimize your management procedure.
If you like this series of articles, please share them and write your thoughts as comments here. Your feedback encourages me to complete this massively planned program. Just share them and provide feedback. I’ll make you an AWS EKS black belt.