There are a lot of guides on the internet describing how to build a Kubernetes Cluster on a Raspberry Pi. I’ve followed quite a few of them to build my cluster. A lot of those guides are out of date now since I want to continue using my Raspberry Pi 3 hardware. Last weekend I embarked on a journey to rebuild my cluster. I’m going to try and report the journey to you now.
One note is that I built this cluster using Ubuntu Server 20.04 instead of Raspberry Pi OS. I did this mainly because I’m disappointed with the Raspberry Pi organization for changing the name of Raspbian to Raspberry Pi OS. Raspbian is really easy to google with Bing. Raspberry Pi OS comes up with too much static.
There are several steps to take to build this cluster:
- Flash the Micro SD Cards
- Setup Ubuntu 20.04 on each machine
- Disable services we won’t use
- Bootstrap the Cluster
- Join the Nodes
Flash
Not Gordon, and not The, but you must flash your Micro SD cards with Ubuntu 20.04 Server. I used the 64bit version of Ubuntu 20.04 for the Raspberry Pi 3 and balenaEtcher to flash the cards .
Once each card was flashed I also added a file to the card called ssh in the root. This file must be blank and it is used to enable SSH on the Raspberry Pi without having a monitor or keyboard connected to it.
About Networking
Some people do this on a separate network, and my first attempt at this I put it in the 192.168.50.0/24 network. However, I couldn’t figure out how to punch a hole through the firewall to gain access to the cluster from my main network after I finished. It was painful. For this attempt I logged into each Raspberry Pi and copied down the MAC address for each of them then added IP Address reservations in my DHCP server for each of them. Now they are on my main network and I can access the cluster from my main machine. I’m sure there’s a good reason that this is bad, and if you know please let me know. I like to learn.
WiFi Network
You can add a file called wpa_supplicant.conf to automatically join your Raspberry Pi to the wireless network, but I wouldn’t do this as a wired network will be much more reliable for the Kubernetes Cluster.
Setup Ubuntu on Each Machine
I’m not as well versed with Ubuntu as I am with Windows. I do know a bit of Linux and am getting better at it all the time.
Setting up Ubuntu on the Raspberry Pi consists of these steps:
- Change the Default Password
- Change the Hostname
- Update
- Set Boot Options
- SSH Config
Change the Default Password
This step is really easy. Once you ssh into your Raspberry Pi with ssh ubuntu@{IP Address} you are forced to change the password. The default password is “ubuntu”. After changing the password you are logged out and you must log in again.
Change the Hostname
By default the hostname is Ubuntu. If you are like me having 6 computers on your network with the same hostname is crazy. So let’s change the hostname. First make sure your favorite text editor is on the Pi and then fire it up and edit the file /etc/hostname by running a command similar to: sudo vim /etc/hostname. There should only be one line in this file. Change it to what you want the hostname to be. Then open the file /etc/hosts and add a line at the top for 127.0.1.1 followed by a tab and then the hostname that you set earlier. Like this:
127.0.1.1 leader
I have 6 Pis that are named:
- leader
- worker1
- worker2
- worker3
- worker4
- worker5
You can name them as you see fit. Once you change both the hsotname file and the hosts file you should reboot with sudo reboot.
Update
Once the Pi has rebooted log in and run the following commmand:
sudo apt-get update && sudo apt-get upgrade -y
This will take a few minutes to complete.
Set Boot Options
Next we have to enable the memory cgroup options. Why? Well from what I can tell Kubernetes needs it. I don’t really know why, but if you do, please drop me a line. 😀
This is where a lot of the directions are different. I’m using Ubuntu 20.04 so the file /boot/firmware/nobtcmd.txt doesn’t exist. It has been depricated in favor of include commands. So open /boot/firmware/cmdline.txt in your favorite editor and add this to the end of it:
cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1
Once saved reboot your Raspberry Pi.
SSH Config
If you are anything like me, you have a very strong password that you’ve had to enter a bunch of times already and I bet it’s either getting old or you are thinking about using a simple/insecure password for this. Well help is here. On your computer (not one of the Pis) see if you have a public key for SSH. This is usually stored in the .ssh folder in your home folder. Inside this .ssh folder would be a file typically named id_rsa.pub. If you have this file great, hold on a second.
If not generate one by typing:
ssh-keygen -t rsa -b 4096
and I usually give it no password when it asks.
Now that you have one (or if you had one before), type this:
ssh-copy-id -i ~/.ssh/id_rsa.pub ubuntu@{Your Pis address}
If you get an error:
'ssh-copy-id' is not recognized as an internal or external command, operable program or batch file.
Then try this command using PowerShell on Windows:
type $env:USERPROFILE\.ssh\id_rsa.pub | ssh ubuntu@{Your Pis address} "cat >> .ssh/authorized_keys"
Obviously replace
{Your Pis address}with the address of your Raspberry Pi. At this point though you should be able to use the hostname that you provided in a previous step.
Now you should be able to log in to your Ubuntu Server without needing your password.
Disable Services We Won’t Use
There are 4 services that Ubuntu provides by default that I won’t be using for my Cluster. You might want the same or not. This is up to you. Enter these lines:
- For no WiFi on the server:
sudo systemctl disable wpa_supplicant.service
- To disable time based tasks
sudo systemctl disable atd.service
sudo systemctl disable cron
- I like to run my own updates on my schedule.
sudo systemctl disable unattended-upgrades.service
Then reboot.
Now repeat everything up to this point on each of your Raspberry Pis.
Bootstrap the Cluster
Now for the fun part. The Raspberry Pis are setup and running. Let’s give them something to do. On your main computer download k3sup by typing:
curl -sSL https://getk3sup.dev | sh
sudo install k3sub /usr/local/bin
Then type the following command:
k3sup install --user ubuntu --sudo --ip {Your leader's IP address}
Next run this command:
export KUBECONFIG=/home/upgrade/kubeconfig
Finally run the command:
kubectl get pod -A
You should see your leader has several system pods running. Great! Try typing in kubectl cluster-info and you should see some basic information about your cluster. Now you should copy the config file to ~/.kube/config so you won’t have to export the environment variable KUBECONFIG to access the cluster.
Join the Nodes
Finally, we need to add workers to the cluster. This is really simple. Just type:
k3sup join --user ubuntu --sudo --server-ip {Your leader's IP address} --ip {Your worker's IP address}
Wait about a minute and then type kubectl get node and you’ll see your worker listed!
k3sup makes building these clusters so much easier. Now you have a Kubernetes cluster what are you going to do with it? Please let me know.
Buy Raspberry Pi 3s on Amazon !
