This morning I had the need to change the IP address configuration for the load balancer in our k8s cluster. The basics of changing the main pool in metallb were straightforward enough, but when I applied my changes, I didn't get what I needed.

So, what happened? Originally, I wasn't thinking like a kubernetes cluster, so I'd not realized that the load balancer itself drives addressing. As such, I was focussed on things like restarting the pods that had been assigned addresses. This did no good, although I did learn how to force fluxcd to detect and correct helm chart drift which was super useful when I accidentally deleted a deployment that didn't come back.

Once I realized that metallb is a kubernetes operator that looks at the Service CRD for type LoadBalancer and plumbs the address and network for those services, I needed to focus on the metallb controller pod, as that was what wasn't effecting the changes.

I looked at the logs and realized it would not pick up the new configuration because there were IP addresses in the old range in use. It turns out that there's no way to automatically update these, but if you restart your controller deployment:

kubectl rollout restart deployment metallb-controller -n metallb-system

Then it will pick up the new configuration and assign new addresses to any service already in use.