Skip to main content

The upgrade rancher dilemna

OK.

It's been awhile since I have not upgraded Rancher and the K3S cluster infra and I think it's maybe thge good time to upgrade my infra to be up to date.

Problem is that I fear this upgrade because I went in the past into some big issues like the rancher webgui not accessible and haf to restore back all the vms of the infra.

But well, that's not a good reason to not to upgrade!

So, I have decided to make it more clean and have a way to:

  1. Backup
  2. Plan
  3. Test
  4. Do
  5. Validate

the update tasks.

This time I really be organized and do this update once for all!

So, my initial plan is to use rancher 2.7.5 because the 2.7.6 and the 2.7.7 seems to be in RC with and not sure exactly what is the difference between the 2...

Anyway!

Let's have a look at my current setup:

  • 3 master nodes (centos 7.9)
  • 3 workers (centos 7.9)
  • kube-vip used as the VIP manager
  • curent version of rancher 2.6.9 (to date, 2.6.13 is the latest of the 2.6.x serie)

Current version of K3s (I used k3sup to install it)

[root@node-1 ~]# kubectl get nodes -o wide
NAME       STATUS   ROLES                       AGE      VERSION         INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION                CONTAINER-RUNTIME
node-1     Ready    control-plane,etcd,master   2y142d   v1.21.12+k3s1   192.168.2.100   <none>        CentOS Linux 7 (Core)   3.10.0-1160.88.1.el7.x86_64   containerd://1.4.13-k3s1
node-2     Ready    control-plane,etcd,master   442d     v1.21.12+k3s1   192.168.2.101   <none>        CentOS Linux 7 (Core)   3.10.0-1160.49.1.el7.x86_64   containerd://1.4.13-k3s1
node-3     Ready    control-plane,etcd,master   431d     v1.21.12+k3s1   192.168.2.102   <none>        CentOS Linux 7 (Core)   3.10.0-1160.88.1.el7.x86_64   containerd://1.4.13-k3s1
worker-1   Ready    <none>                      2y142d   v1.21.12+k3s1   192.168.2.103   <none>        CentOS Linux 7 (Core)   3.10.0-1160.49.1.el7.x86_64   containerd://1.4.13-k3s1
worker-2   Ready    <none>                      2y142d   v1.21.12+k3s1   192.168.2.104   <none>        CentOS Linux 7 (Core)   3.10.0-1160.88.1.el7.x86_64   containerd://1.4.13-k3s1
worker-3   Ready    <none>                      356d     v1.21.12+k3s1   192.168.2.105   <none>        CentOS Linux 7 (Core)   3.10.0-1160.49.1.el7.x86_64   containerd://1.4.13-k3s1

Now we need to check what is que OS, K3S requirements to do the upgade to 2.7.5.

There is a website that describe the support matrix for rancher: https://www.suse.com/suse-rancher/support-matrix/all-supported-versions/rancher-v2-7-5/

So we we look at what we want vs what we got:

Rancher Version
 Low
 High
2.7.5
 1.23
 1.26
2.6.9
 1.20
 1.24
2.6.13
 1.20
 1.24

And in the K/S section of the page you can check what K3S version is supported per OS version:

Let say that we want to stick with Centos or Rocky linux:

K3S Version
 Low
 High

v1.24.15+k3s1

 Centos 7.9
 Rocky Linux 8.4/8.5/8.6

v1.26.6+k3s1

 N/A
 Rocky Linux 8.7/8.8/8.9/9.0/9.1/9.2
v1.21.12+k3s1
 ??
 ??

So it seems to be pretty clear that if I want to go to 1.26, I will will to go with Rocky 9. Or switch to Ubuntu...

I honestly prefer Debian so I guess I will go with an upgrade to Rocky linux. Which mean than the upgrade plan will be a little more complex as we have to move from Centos To Rocky..

My first thoughts about this is it could be pretty simple even if it's going to take some times:

  1. Add 3 more masters with Rocky Linux 8.4 to have v1.21.12+k3s1 (sse here https://www.suse.com/assets/EN-2.6.5SupportMatrix-300422-0116-26.pdf)
  2. Add 3 more workers with Rocky Linux 8.4
  3. Remove the Centos workers
  4. Remove the Centos masters
  5. Make sure the vip with KUBE-VIP is configured properly.

Once all is migrated to Rocky Linux, then I can think about migrating to 1.24 for k3s.

  1. Upgrade k3s to v1.24.15+k3s1
  2. Upgrade rancher to 2.6.13 (not sure if it is needed)

Then

  • Should I take the risk to upgrade RL to 8.9 and upgrade k3s to 1.26 at the same time and hope that I can upgrade to Rancher 2.7.5 at the end?
  • Any option option?

Well, another option could be this one:

  1. Create 3 masters, 3 workers with RL 9.2
  2. Install K3S 1.26
  3. Install Rancher 2.7.5
  4. Install KUBe-VIP with a new vip
  5. Configure pretty much the same settings
  6. Copy/Paste all the deployments to the new cluster

I meam that could be an option too! We just have to be sure the app that I will copy paste will not access their config (most of them are stored on NFS) at the same time.

Back to top