Amazon EKS Anywhere and Amazon EKS Connector

What is EKS Anywhere? It’s an option that AWS created so that we can run and operate like Kubernetes clusters that are EKS. But outside of AWS infrastructure. we can have EKS clusters running in our on-premise data center, for example, or, depending on how far this goes. And even in other cloud providers, you can just set up a few servers in Google or Azure, install a cluster using EKS Anywhere, and then you have EKS in another cloud, right. But for today, I’m going to do the on-premise approach. we will not be using any other cloud provider. I’m going to show you that after. But this is basically what EKS Anywhere is. It’s a way to have EKS clusters that are not running in an AWS account or an AWS infrastructure.

What does it look like? To set up an EKS Anywhere cluster, you need to use the EKS CTL. Some of you have probably already used this to provision a regular EKS cluster in AWS. You use the same tool to provision the Anywhere cluster. The only thing extra that you need to do if you already have EKS CTL is if you have to install the Anywhere plugin to the EKS CTL CLI. once you install that, you get another set of subcommands. everything you run is EKS CTL space Anywhere. And then the commands are pretty similar. if you can see it on this diagram, the command is EKS CTL Anywhere and then create a cluster. And that starts the provision. This diagram is showing what it would look like if we use VMware and vSphere. Right now, that’s the only third-party-supported provisioning that it supports right now. You can do one of two things – you can provision a local testing cluster using Docker and EKS CTL, or you can provision a production-grade cluster in vSphere, using VMware. And that’s what is described right here. What it does is it provisions a small bootstrapping cluster and from that one, it runs the provisioning on vSphere to provision all the virtual machines and set up the actual production cluster.

Okay, so what are we going to be building for this demo? We’re going to simulate the on-premise part of the EKS Anywhere. I created this diagram to explain. First, I have my regular workstation here. This is the computer that I am on right now. Then I set up an Ubuntu server on an old laptop. I have a laptop in my local network running an Ubuntu server. And that’s going to be my on-premise server. On that one, I have EKS CTL and my cluster configurations. And what I’m going to do is I’m going to provision EKS Anywhere cluster on this machine that will include my control plane and I could set up one or more workers. We’re going to stay with just a single worker. And once we have that EKS cluster running on my local infrastructure, I’m going to use the EKS Connector to have that cluster show up as an EKS cluster in the AWS console.

Okay, so let’s go to the demo part. As I mentioned, I have my local workstation and Ubuntu server here. I’m going to connect to my Ubuntu server through SSH. Here we go. You can see this is running Ubuntu 2204 and not much else. I’m going to show you first what we have installed here. As I mentioned, I have the EKS CTL. And it’s Bradley version 0.95. Then, we need the Anywhere plugin for this EKS CTL, right. If we do EKSCTL Anywhere, that’s its own thing. It’s a different version. The Anywhere plugin is installed as a separate binary, but it doesn’t like it if you try to use it as its own thing. The other binary is called EKSCTL-Anywhere. But if I do this, it just tells me, please use it as a plugin for the original CLI. But it is a separate program. We have these two things installed already. I have AWS credentials that we’re going to be using afterward. But those are not necessary to provision the cluster because this is on-premise. And I also have a KubeCTL here, so we can interact with the cluster after.

The first thing we need to do is we need to set up our trust. We’re going to set it as an environment variable. To start provisioning a cluster again, we use CTL Anywhere, then the first thing is to generate the config. Use our cluster name. And then we need to tell it which type of cluster we’re provisioning. And that’s done with a provider. As I was mentioning before, the only two setups right now are either a local development cluster using Docker, or vSphere, or the regular AWS cluster using EKSCTL. Alright, so in this case, I’m going to say my provider is Docker. And this will output a configuration file, and we’re just going to save it to what’s there. This generated a file, we can see it here, demo. If I open this, this will look very, very similar to a normal EKS CTL cluster, and it’s configuring things like the network, the version, and the number of worker nodes, as you can see, here, it’s 122, Kubernetes version and worker count is one. And that’s it. This is a super simple one because it’s in Docker. If you tried to do one with vSphere, that configuration file is a bit larger because it includes VMware-specific configurations that you need to modify here after generating the file like credentials and things like that. But for Docker, it’s simpler, of course. Now that we have this file here, the cluster configuration. We can do EKS CTL Anywhere. Now the command would be Create cluster and you give it a file. It will be Demo.YAML.

This part right here, it can take up to 20 minutes, maybe depending on where you’re running it. For me, it has taken between 15 and 20 minutes for the times I’ve tested this, so we’re not going to wait 15 minutes. If we exit this folder, you’ll see that I already have some other things here. This Friday demo, EKSAnywhere.YAML, was the cluster configuration file I generated before. This one is the same thing we just saw inside the demo folder. It’s just the one I created before. And once you run the Create cluster command, it will provision the cluster using Docker. If we do Docker ps here, for example, you see I have some containers running. And if you notice there’s also this demo folder I created when we started just to do the initial point. But we have this Friday demo EKS Anywhere folder. And that was created after I ran the Create cluster command. In there, we have some more configurations. But this is the important one. The Kube config file for this cluster is inside that folder. To be able to run Kube CTL commands against that local cluster, I need to export my Kube config. I’m going to export this. It’s my current directory. Then the folder with the cluster name and then the Kube config file here. Now that is exported, I can do Kube CTL commands, and those should work. For example, get namespaces.

Now that it’s properly configured, I can run Kube CTL commands, you can see here we have quite a few namespaces. We can also check for running thoughts to check in all namespaces. This is what’s currently running. It’s just a cluster requirement. The Kube proxy, the API server, and some extra things that are part of the Anywhere in the EKS Anywhere cluster bootstrapping. This is a fully functional Kubernetes cluster already. Granted, it’s not a super-powerful cluster right now because it’s running on a laptop, but you get the idea.

This is an EKS Anywhere cluster, but if we go to the AWS console here, you’ll notice that we still don’t have any clusters. What we need is to deploy the EKS Connector into our EKS Anywhere cluster, and then we will be able to see it here. To do that, I’m going to set up my AWS credentials. And again, there are a couple of ways to do this. we could use the AWS CLI to register a new cluster, or we could do it here from the console. if you click on the console and add a cluster, you’ll see that you have the option to register. If we tried to register one, it will ask for a name and a provider. Here, we would choose EKS Anywhere. This needs a role, an IAM role with specific trust relationships and permissions. And then once you registered, it will give you some files that you can download either from here or from the CLI. That’s the complicated way of doing it. The easy way of doing it would be to again use the EKS CTL Anywhere CLI. The command to do that is this one. Because CTL registered cluster name, so Friday demo Anywhere. The provider is EKS Anywhere and the region I want to use is US press two. Let’s see what we get here. This provision I’m going to enroll for us. When it’s done, we will get three files. Those three files are what we need to apply to the cluster. Once we apply those, the cluster registers. Once this command finishes, it creates a roll, and then it gives us back a couple of Kubernetes manifests to apply.

On those, by default, it will have the current user that ran the command, that’s the user that will be allowed to see the workloads connect to the cluster. What we do there is we edit those files to include any other IAM users or IAM roles that we want to have access to the cluster in the AWS console. And then we apply those manifests. That creates a bunch of things like secrets, config maps. Now we have some more files here. And this last blue line gives us a command. It says from Kube CTL apply EKS Connector yellow, because Connector cluster role and EKS Connector console dashboard full access. This one is Connector. This one, we don’t have to modify anything. This is just the deployment of the Connector pod and a bunch of config maps or secrets or somewhere in here there is a token that was generated by AWS when you do it manually, you add cluster here register at the end. We’ll put a token in the files and that token lasts for like three days. you have three days to register your cluster. And if you don’t do it on those three days, then you have to run the register command again to get a fresh token. All that is here now. No need to modify this one. We do need to modify the cluster role. Here it’s just the permissions to be able to do stuff in the cluster from AWS to be able to see things so what I’m going to put here is the role the SSL role that we use in AWS. We say this and the other one is the dashboard full access. Again, this is just permissions. And here you can see my user is already set if we want to add the role just add another block. Remember we didn’t have any cluster here before running the register. Now if we refresh we have the Friday demo Anywhere cluster but it’s pending and it will remain on a pending state until we apply these files. Okay, so everything was created. Now we should have some new pods trying to start. Here we have the EKS Connector pod. It’s initializing which gives it a few seconds to complete. Okay, almost there, we have the first one started. And I think the first one is enough for it to show. Now we have our cluster name here for the demo Anywhere. It’s on its status of active we can see the version. And on the provider, we can see this is an EKS Anywhere cluster.

Here we can see this cluster has two notes running, and we can see the current resources. The same stuff you will be able to see on a regular EKS cluster, but this is not running on AWS. This is running an on-premise machine. I forgot to show you this. This is when you run the EKS Anywhere create cluster command. This is what you see. I took a screenshot because as I mentioned, it takes like 15 minutes to complete. But yeah, this is the output of that.