Built on November 11, 2026.
What we built and why
I wanted a small Kubernetes cluster at home, no distro wrappers and no single-binary shortcuts. As lazy as I am, I wanted to go back to the roots of Kubernetes by just using upstream Kubernetes installed with kubeadm.
The cluster runs on two Intel NUCs for now so I can show more than one node and add as I go. k8s-cp1 is the control plane. k8s-worker1 is the first worker. Both sit on the same home LAN, and I all cluster traffic stays on that network.
Both NUCs are also on my Tailscale tailnet. I use that to run kubectl from my Mac wherever I am. The API server certificate includes the Tailscale IP and MagicDNS name, so remote access works without exposing anything to the internet.
One more wrinkle. k8s-cp1 already runs Docker with a monitoring stack and some automation tools. Those containers had to keep running through the whole build. That affected how I set up the container runtime so I figured I would show that part of the process as well.
The cluster is called labcluster for the docs here but I’d suggest that you choose a more fun name for yours 🙂
Hardware and network
| Item | k8s-cp1 | k8s-worker1 |
|---|---|---|
| Role | Control plane | Worker |
| CPU | Intel Core i5-6260U, 4 threads | Intel Core i5-5250U, 4 threads |
| RAM | 32 GB | 16 GB |
| OS | Ubuntu 24.04.4 LTS | Ubuntu 24.04.4 LTS |
| Kernel | 6.8.0-136-generic | 6.8.0-147-generic |
| LAN interface | eno1 | eno1 |
| LAN IP | 192.168.10.11 | 192.168.10.21 |
| Tailscale IP | 100.64.0.11 | 100.64.0.21 |
| Existing workloads | Docker (a monitoring stack and a few self-hosted apps) | None |
| Container runtime | containerd 2.2.4 (from Docker) | containerd 2.4.2 (from Docker’s repo) |
Network plan
| Range | Use |
|---|---|
| 192.168.10.0/24 | Home LAN and node IPs |
| 10.244.0.0/16 | Pod network (Calico) |
| 10.96.0.0/12 | Kubernetes services |
| 172.17-19.0.0/16 | Existing Docker bridges on k8s-cp1 (left alone) |
| 100.64.0.0/10 | Tailscale |
The pod and service ranges avoid the LAN, the Docker bridges and Tailscale. Overlapping ranges cause routing problems that are painful to track down later…trust me on this as a life lesson to avoid.
Versions
| Component | Version |
|---|---|
| Kubernetes (kubeadm, kubelet, kubectl) | v1.37.2 |
| containerd on k8s-cp1 | 2.2.4 (containerd.io from Docker) |
| containerd on k8s-worker1 | 2.4.2 (containerd.io from Docker’s apt repo) |
| Calico | v3.33.0 (Tigera operator) |
| kubectl on my Mac | v1.36.1 |
Prerequisites
- Two Ubuntu 24.04 machines on the same LAN
- Root or sudo on both
- Tailscale installed on both nodes and on the machine you will run kubectl from
- At least 2 CPUs and 2 GB RAM per node (kubeadm checks this)
- Unique hostnames, MAC addresses and product UUIDs on each node
Check the current stable release before you start.
curl -s https://dl.k8s.io/release/stable.txt
# v1.37.2 at build timeStep 1. Record what is already running
k8s-cp1 had Docker containers I could not afford to lose. I captured their state first so I had something to compare against.
docker ps --format '{{.Names}}\t{{.Status}}'All nine containers showed Up. I checked this list again after every risky step.
Step 2. Turn off swap
The kubelet expects swap to be off by default. Run this on both nodes.
swapoff -a
sed -i.bak-k8s -E 's@^([^#].*\sswap\s.*)$@#\1@' /etc/fstab
grep swap /etc/fstab
swapon --showThe sed line comments out the swap entry so it stays off after a reboot. It also keeps a backup at /etc/fstab.bak-k8s.
Step 3. Kernel modules and sysctl
Kubernetes networking needs the overlay and br_netfilter modules. It also needs bridged traffic to pass through iptables and IP forwarding turned on. Run this on both nodes.
cat > /etc/modules-load.d/k8s.conf <<EOT
overlay
br_netfilter
EOT
modprobe overlay
modprobe br_netfilter
cat > /etc/sysctl.d/99-kubernetes.conf <<EOT
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOT
sysctl --system
sysctl net.bridge.bridge-nf-call-iptables net.ipv4.ip_forwardStep 4. Container runtime
k8s-cp1: reuse Docker’s containerd
Docker already installs containerd as the containerd.io package, and Kubernetes can use that same containerd. There is one catch…Docker ships a config file that turns off the CRI plugin, and the kubelet needs CRI to talk to containerd.
This is the whole Docker-provided config.
disabled_plugins = ["cri"]I backed it up and replaced it with the full default config. Then I switched the runc cgroup driver to systemd to match the kubelet.
cp -a /etc/containerd/config.toml /etc/containerd/config.toml.bak-pre-k8s
containerd config default > /tmp/ctd.toml
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /tmp/ctd.toml
install -m 644 /tmp/ctd.toml /etc/containerd/config.tomlThe only change from the stock default is one line.
- SystemdCgroup = false
+ SystemdCgroup = trueThat line lives under this section in the containerd 2.x config.
[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runc.options]
SystemdCgroup = trueNow the part that made me nervous. Restarting containerd on a box with live Docker workloads. Each running container has its own containerd shim process, and the shims keep running when containerd restarts. Once containerd is back up, Docker reconnects. The default config also keeps the same root at /var/lib/containerd, so Docker’s images and snapshots stay where Docker expects them.
systemctl restart containerd
sleep 8
systemctl is-active containerd docker
docker ps -a --format '{{.Names}}\t{{.Status}}'
ctr plugins ls | grep -i criEvery container still showed its original uptime. None of them restarted. The CRI plugin showed ok. Woot!
io.containerd.cri.v1 images - ok
io.containerd.cri.v1 runtime linux/amd64 ok
io.containerd.grpc.v1 cri - okIf Docker has live-restore off, you can still expect this to work. Do it during a quiet window anyway, and have your docker compose files handy.
k8s-worker1: install containerd from Docker’s repo
k8s-worker1 had no runtime. I installed containerd.io from Docker’s apt repo so both nodes run containerd 2.x.
export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y ca-certificates curl gpg
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" \
> /etc/apt/sources.list.d/docker.list
apt-get update
apt-get install -y containerd.io
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
systemctl enable containerd
systemctl restart containerd
containerd --versionThe resulting repo line looks like this.
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu noble stableThis only installs containerd, not the Docker engine.
Step 5. Install kubeadm, kubelet and kubectl
Use the official pkgs.k8s.io repo. Each Kubernetes minor version has its own repo path, so set it to match the stable release. Run this on both nodes.
export DEBIAN_FRONTEND=noninteractive
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.37/deb/Release.key \
| gpg --dearmor --yes -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.37/deb/ /' \
> /etc/apt/sources.list.d/kubernetes.list
apt-get -o DPkg::Lock::Timeout=300 update
apt-get -o DPkg::Lock::Timeout=300 install -y kubelet kubeadm kubectl cri-tools
apt-mark hold kubelet kubeadm kubectl
systemctl enable kubeletThe hold keeps routine apt upgrades from moving Kubernetes versions under you. Upgrades should go through kubeadm upgrade.
Point crictl at containerd so you can debug pods at the runtime level.
cat > /etc/crictl.yaml <<EOT
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10
EOT
kubeadm version -o short
kubelet --version
crictl info > /dev/null && echo crictl-okStep 6. Initialize the control plane
I used a kubeadm config file instead of a long list of flags. It is easier to read and easy to keep in git.
Find the node’s MagicDNS name first.
tailscale status --json | python3 -c 'import json,sys; print(json.load(sys.stdin)["Self"]["DNSName"])'Here is the config I used on k8s-cp1, saved as /root/kubeadm/init.yaml. Replace with your own tailnet name.
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.10.11
bindPort: 6443
nodeRegistration:
name: k8s-cp1
criSocket: unix:///run/containerd/containerd.sock
kubeletExtraArgs:
- name: node-ip
value: 192.168.10.11
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
kubernetesVersion: v1.37.2
clusterName: labcluster
controlPlaneEndpoint: 192.168.10.11:6443
networking:
podSubnet: 10.244.0.0/16
serviceSubnet: 10.96.0.0/12
apiServer:
certSANs:
- 192.168.10.11
- 100.64.0.11
- k8s-cp1
- k8s-cp1.<tailnet>.ts.net
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemdA few choices in this file are worth calling out.
- advertiseAddress and node-ip pin the control plane to the LAN IP. Without them the kubelet may pick the Tailscale or Docker bridge address.
- certSANs adds the Tailscale IP, the short hostname and the MagicDNS name to the API server certificate. That is what lets kubectl connect over Tailscale with TLS checks intact.
- cgroupDriver systemd matches the containerd setting from Step 4.
Run the init and set up kubectl for root and my regular user.
kubeadm init --config /root/kubeadm/init.yaml
mkdir -p /root/.kube
cp /etc/kubernetes/admin.conf /root/.kube/config
install -d -o <user> -g <user> /home/<user>/.kube
install -m 600 -o <user> -g <user> /etc/kubernetes/admin.conf /home/<user>/.kube/config
kubectl get nodesThe node shows NotReady at this point. That is normal until a pod network is installed.
Step 7. Install Calico
I used the Tigera operator for Calico v3.33.0. From 3.30 on, the CRDs ship in their own manifest.
V=v3.33.0
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/$V/manifests/operator-crds.yaml
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/$V/manifests/tigera-operator.yaml
kubectl wait --for=condition=Established crd/installations.operator.tigera.io --timeout=60sThe stock custom-resources.yaml uses 192.168.0.0/16 for pods. I changed it to match the podSubnet in the kubeadm config. I also told Calico which network to use for node addresses.
Saved as /root/kubeadm/calico-cr.yaml.
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
name: default
spec:
calicoNetwork:
nodeAddressAutodetectionV4:
cidrs:
- 192.168.10.0/24
ipPools:
- name: default-ipv4-ippool
blockSize: 26
cidr: 10.244.0.0/16
encapsulation: VXLANCrossSubnet
natOutgoing: Enabled
nodeSelector: all()
---
apiVersion: operator.tigera.io/v1
kind: APIServer
metadata:
name: default
spec: {}
---
apiVersion: operator.tigera.io/v1
kind: Goldmane
metadata:
name: default
---
apiVersion: operator.tigera.io/v1
kind: Whisker
metadata:
name: defaultkubectl create -f /root/kubeadm/calico-cr.yamlWhy LAN IP autodetection
Calico picks a node address on its own by default, however each NUC has many interfaces. There is the LAN NIC, tailscale0 and on k8s-cp1 several Docker bridges. If Calico picks tailscale0, pod traffic between nodes rides the Tailscale tunnel. If it picks a Docker bridge, cross-node pod traffic breaks. Pinning detection to 192.168.10.0/24 makes Calico use the LAN every time.
Why VXLANCrossSubnet
Both nodes share one subnet, so Calico routes pod traffic directly with no encapsulation. If I add a node on another subnet later, Calico wraps that traffic in VXLAN automatically. VXLAN also avoids needing BGP on my home router.
The IPPool Calico created matches the plan.
{"allowedUses":["Workload","Tunnel"],"assignmentMode":"Automatic","blockSize":26,"cidr":"10.244.0.0/16","ipipMode":"Never","natOutgoing":true,"nodeSelector":"all()","vxlanMode":"CrossSubnet"}Check that each node got its LAN address.
kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name} {.metadata.annotations.projectcalico\.org/IPv4Address}{"\n"}{end}'k8s-worker1 192.168.10.21/24
k8s-cp1 192.168.10.11/24Give the operator a minute or two to settle. Some components report Degraded early on while the Calico API server starts. They clear on their own.
kubectl get tigerastatusStep 8. Join the worker
Print a fresh join command on the control plane.
kubeadm token create --print-join-commandI used a join config file instead of the raw command so I could pin the node IP. Saved on k8s-worker1 as /root/kubeadm/join.yaml with mode 600.
apiVersion: kubeadm.k8s.io/v1beta4
kind: JoinConfiguration
discovery:
bootstrapToken:
apiServerEndpoint: 192.168.10.11:6443
token: <token>
caCertHashes:
- sha256:<hash>
nodeRegistration:
name: k8s-worker1
criSocket: unix:///run/containerd/containerd.sock
kubeletExtraArgs:
- name: node-ip
value: 192.168.10.21kubeadm join --config /root/kubeadm/join.yaml
rm -f /root/kubeadm/join.yamlBootstrap tokens expire after 24 hours. Delete the file after the join anyway since it holds a credential.
Back on the control plane, label the worker and wait for both nodes.
kubectl label node k8s-worker1 node-role.kubernetes.io/worker= --overwrite
kubectl wait --for=condition=Ready node --all --timeout=300sStep 9. Verify
Nodes
kubectl get nodes -o wideNAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8s-worker1 Ready worker 8m31s v1.37.2 192.168.10.21 <none> Ubuntu 24.04.4 LTS 6.8.0-147-generic (amd64) containerd://2.4.2
k8s-cp1 Ready control-plane 9m12s v1.37.2 192.168.10.11 <none> Ubuntu 24.04.4 LTS 6.8.0-136-generic (amd64) containerd://2.2.4Pods
kubectl get pods -ANAMESPACE NAME READY STATUS RESTARTS AGE
calico-system calico-apiserver-69c6474754-pp4xx 1/1 Running 0 8m34s
calico-system calico-apiserver-69c6474754-zgbmm 1/1 Running 0 8m34s
calico-system calico-kube-controllers-7c9969d9c5-wbls4 1/1 Running 0 8m33s
calico-system calico-node-6kdzm 1/1 Running 0 8m33s
calico-system calico-node-r722q 1/1 Running 0 8m30s
calico-system calico-typha-69789f988-h6lxg 1/1 Running 0 8m33s
calico-system csi-node-driver-fnngm 2/2 Running 0 8m33s
calico-system csi-node-driver-lskjd 2/2 Running 0 8m30s
calico-system goldmane-7fb6b9c74b-794qb 1/1 Running 0 8m34s
calico-system whisker-578fb565d5-b6hsz 2/2 Running 0 8m13s
kube-system coredns-559f6c778d-gdkbh 1/1 Running 0 9m2s
kube-system coredns-559f6c778d-jbf64 1/1 Running 0 9m2s
kube-system etcd-k8s-cp1 1/1 Running 0 9m8s
kube-system kube-apiserver-k8s-cp1 1/1 Running 0 9m8s
kube-system kube-controller-manager-k8s-cp1 1/1 Running 0 9m9s
kube-system kube-proxy-6p9ph 1/1 Running 0 8m30s
kube-system kube-proxy-fsnzz 1/1 Running 0 9m2s
kube-system kube-scheduler-k8s-cp1 1/1 Running 0 9m9s
tigera-operator tigera-operator-7c96ff8686-dfwx6 1/1 Running 0 8m45skubectl get tigerastatusNAME AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE
apiserver True False False 7m39s All objects available
calico True False False 7m49s All objects available
goldmane True False False 7m29s All objects available
ippools True False False 8m29s All objects available
tiers True False False 7m34s All objects available
whisker True False False 7m9s All objects availableWorkload test
This runs two nginx pods on the worker and hits them through a ClusterIP from the control plane. It proves scheduling, cross-node pod networking and kube-proxy all work.
kubectl create deployment nginx-test --image=nginx:stable --replicas=2
kubectl patch deployment nginx-test \
-p '{"spec":{"template":{"spec":{"nodeSelector":{"kubernetes.io/hostname":"k8s-worker1"}}}}}'
kubectl expose deployment nginx-test --port 80
kubectl rollout status deployment nginx-test --timeout=180s
kubectl get pods -l app=nginx-test -o wide
IP=$(kubectl get svc nginx-test -o jsonpath='{.spec.clusterIP}')
curl -s -m 10 http://$IP | grep -o '<title>.*</title>'<title>Welcome to nginx!</title>DNS test
kubectl run dnstest --image=busybox:1.36 --restart=Never \
--overrides='{"spec":{"nodeSelector":{"kubernetes.io/hostname":"k8s-worker1"}}}' \
--command -- nslookup kubernetes.default.svc.cluster.local
kubectl wait --for=jsonpath='{.status.phase}'=Succeeded pod/dnstest --timeout=60s
kubectl logs dnstestName: kubernetes.default.svc.cluster.local
Address: 10.96.0.1Clean up.
kubectl delete pod dnstest
kubectl delete svc,deployment nginx-testDocker workloads
Last check on k8s-cp1.
docker ps --format '{{.Names}}\t{{.Status}}'All nine containers were still Up with their original uptime of weeks and months. The published ports still answered locally.
Step 10. kubectl from my Mac over Tailscale
The admin kubeconfig points at the LAN address 192.168.10.11. I copied it to my Mac and swapped the server for the Tailscale IP. The cert already covers that IP from Step 6.
I kept it in its own file so it would not touch any existing ~/.kube/config.
mkdir -p ~/.kube
umask 077
ssh root@k8s-cp1 'cat /etc/kubernetes/admin.conf' \
| sed 's#server: https://192.168.10.11:6443#server: https://100.64.0.11:6443#' \
> ~/.kube/labcluster.config
chmod 600 ~/.kube/labcluster.config
kubectl --kubeconfig ~/.kube/labcluster.config get nodesNAME STATUS ROLES AGE VERSION
k8s-worker1 Ready worker 2m44s v1.37.2
k8s-cp1 Ready control-plane 3m25s v1.37.2You can use the MagicDNS name instead of the IP. Both are in the certificate, thankfully.
This file is full cluster admin, so please treat it like a private key! For daily use I create a less privileged user and only elevate rights when needed.
Gotchas that Got Me
Docker’s containerd config disables CRI
The containerd.io package from Docker writes a one-line config that disables the CRI plugin. kubeadm fails its preflight checks until you fix that. Back up the file, generate the default config and set SystemdCgroup to true. Restarting containerd did not restart any Docker containers.
apt lock from unattended upgrades
The first package install on k8s-worker1 failed with “Could not get lock /var/lib/dpkg/lock-frontend”. An automatic upgrade run was holding it. Adding a lock timeout to apt fixed it without killing anything.
apt-get -o DPkg::Lock::Timeout=300 install -y kubelet kubeadm kubectl cri-toolsneedrestart held back docker.service
After the package installs on k8s-cp1, Ubuntu’s needrestart listed docker.service as a deferred restart. I left it deferred on purpose. Restarting Docker would bounce every container on the box. Do that later in a maintenance window.
kubectl version skew
My Mac had kubectl v1.36.1 and the cluster runs v1.37.2. kubectl supports one minor version of skew in either direction, so it works. I will upgrade the Mac client to stay current.
Calico status looks broken at first
Right after applying the Calico resources, tigerastatus showed goldmane and whisker as Degraded with API errors. The Calico API server was still starting, and everything went green within about two minutes. Sometimes patience truly is a virtue.
Next in the Backlog…Digital Rebar
Part two of this project is getting Digital Rebar from RackN. The plan is to run the Digital Rebar endpoint on k8s-cp1 and enroll both NUCs as existing machines. No PXE reimaging to start.
Then I will rebuild this same kubeadm cluster as a Digital Rebar workflow and compare it with the hand-built version above. More on that soon!