AWS EKS – Part 34 – Kubernetes Storage using EBS CSI Driver

AWS EBS “Elastic Block Store” is a high-performance, easy-to-use block storage service designed for use with EC2 instances. It can also be integrated with EKS for Kubernetes workloads that need Persistent storage, like StatefulSet, to deploy stateful applications. In this lesson, you will learn how to deploy and configure the EBS CSI Driver to create and manage EBS volumes for your workloads using Kubernetes resources like StorageClass, PersistentVolume, PersistentVolumeClaim and within StatefulSet volume templates.

Follow our social media:

https://linkedin.com/in/ssbostan

https://linkedin.com/company/kubedemy

https://youtube.com/@kubedemy

https://telegram.me/kubedemy

Register for the FREE EKS Tutorial:

If you want to access the course materials, register from the following link:

Register for the FREE AWS EKS Black Belt Course

What is AWS EBS?

AWS EBS is a high-performance, easy-to-use block storage service which provides persistent storage that can be attached to running instances, offering high availability and durability for critical workloads. It supports various volume types, such as SSD and HDD options, to cater to performance needs. With snapshots, encryption, and scalability, EBS allows users to manage storage efficiently and securely. Additionally, it enables quick scaling of storage capacity and performance based on demand.

How does EBS fit in Kubernetes?

EBS volumes can be used statically using the PersistentVolume resource or provisioned dynamically using the StorageClass resource with the EBS CSI Driver. It provides a ReadWriteOnce access mode, meaning you can simultaneously mount it to only one worker node. It also supports Snapshots, Resizing, Backup, and Restore.

EBS + EKS Limitations:

• Single Availability Zone: EBS volumes are tied to a single Availability Zone. Suppose a Kubernetes Pod using an EBS volume needs to be scheduled on a node in a different AZ. In that case, the volume cannot follow, potentially impacting high availability and failover scenarios. For example, if a StatefulSet Pod needs to reschedule due to node failure and no worker node is available in the same AZ, the Pod will not come up and be stuck in the Pending state for good, or if the AZ goes out of reach, you can’t deploy the Pod to another availability zone.
• Node Dependency: EBS volumes can only be attached to one EC2 instance “worker node” at a time in ReadWriteOnce mode. This restricts the volume to Pods running on a single node, limiting scenarios requiring shared access to the volume across multiple nodes. This limitation also exists for most of the block storage.

AWS EBS in EKS setup procedure:

• Create the required IAM role for the EBS CSI Driver.
• Install the EBS CSI Driver using EKS Addons.
• Create Kubernetes PVC backed with EBS volume.
• Deploy test application using Kubernetes StatefulSet.

Step 1 – Create IAM Role for EBS CSI Driver:

The EBS CSI Driver needs AmazonEBSCSIDriverPolicy AWS provides this policy. You can add it to the Node IAM Role or, if IRSA is used, create a new IAM role for the driver and attach the policy. To learn more about IRSA, read the following article:

AWS EKS – Part 13 – Setup IAM Roles for Service Accounts (IRSA)

aws iam attach-role-policy \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
  --role-name Kubedemy_EKS_Managed_Nodegroup_Role

Step 2 – Install the EBS CSI Driver:

The EBS CSI Driver can be installed using both Helm and EKS Addons. In this lesson, we will deploy the EBS CSI driver using EKS Addons. If you want to deploy the driver in a self-managed model using Helm, read the official documentation here.

cat <<EOF > ebs-csi-driver.yaml
controller:
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 1
          preference:
            matchExpressions:
              - key: eks.amazonaws.com/compute-type
                operator: NotIn
                values:
                  - fargate
        - weight: 100
          preference:
            matchExpressions:
              - key: node.kubernetes.io/scope
                operator: In
                values:
                  - system
EOF

aws eks create-addon \
  --cluster-name kubedemy \
  --addon-name aws-ebs-csi-driver \
  --addon-version v1.33.0-eksbuild.1 \
  --configuration-values file://ebs-csi-driver.yaml \
  --tags owner=kubedemy

The above configuration adds a Node Affinity to prefer system nodes for scheduling the EBS CSI Driver Controller. The other rule, which is by default, prefers not to schedule on Fargate nodes. The EBS CSI Driver NODE component, a DaemonSet, will only be deployed to EC2 worker nodes as the EBS doesn’t work on Fargate nodes at all.

Step 3 – Create EBS PersistentVolumeClaim:

To create a new Persistent Volume backed by an EBS volume, create the following resource. As the EBS StorageClass VolumeBindingMode is WaitForFirstConsumer to find the proper AZ to create the volume, the PVC resource remains in the Pending state until a new Pod, which uses that PVC, is scheduled to a worker node.

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-ebs
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: gp2
  resources:
    requests:
      storage: 1Gi
EOF

Deploy a Pod and recheck the PVC:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  containers:
    - name: alpine
      image: alpine:latest
      volumeMounts:
        - name: test-ebs
          mountPath: /data
  volumes:
    - name: test-ebs
      persistentVolumeClaim:
        claimName: test-ebs
EOF

Step 4 – Deploy StatefulSet with EBS Volume:

As we have EBS in place, we can deploy any Stateful application to our Kubernetes cluster. For example, to deploy a MySQL instance:

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
        - name: mysql
          image: mysql:8
          volumeMounts:
            - name: mysql-data
              mountPath: /var/lib/mysql
  volumeClaimTemplates:
    - metadata:
        name: mysql-data
      spec:
        storageClassName: gp2
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
EOF

Conclusion:

Using EBS with EKS provides scalable, persistent storage for Kubernetes applications. Although this integration enhances data durability and availability, its limited single availability zone can significantly make your environment vulnerable to failures. If you need high-performance storage, EBS is the right choice, but if you’re looking for high availability, EFS is much better, as will be explained in the next article.

If you like this series of articles, please share them and write your thoughts as comments here. Your feedback encourages me to complete this massively planned program. Just share them and provide feedback. I’ll make you an AWS EKS black belt.

Follow my LinkedIn https://linkedin.com/in/ssbostan

Follow Kubedemy’s LinkedIn https://linkedin.com/company/kubedemy

Follow Kubedemy’s Telegram https://telegram.me/kubedemy