Kubernetes on Your Desk: Cross-Architecture Home Lab

The Setup

Most Kubernetes learning environments are single-node sandboxes. Minikube, Kind, Docker Desktop: they teach you kubectl syntax, not Kubernetes behavior. Scheduling decisions, cross-node networking, failover, workload migration: none of these exist when there's only one node. This project connects two machines with different CPU architectures into a single cluster using k3s.

Component	Specification	Role	Architecture
Ubuntu Desktop	Ryzen 5 5600X, 16GB RAM	Control plane + worker	amd64
MacBook Pro M4 Pro	24GB RAM, via Multipass VM	Worker node	arm64
Network	Home LAN (192.168.101.x)	Bridged connectivity	N/A

Total cost beyond hardware: $0.

Why k3s

Standard Kubernetes (kubeadm) requires managing etcd, certificate rotation, and multiple system services separately. Minimum 2GB RAM for the control plane alone. k3s is a CNCF-certified Kubernetes distribution that bundles everything into a single binary at under 100MB. Same API, same kubectl, same manifests. It ships containerd, flannel, CoreDNS, and Traefik out of the box. Control plane footprint: ~512MB. For learning, the trade-off is obvious. One install command, and the control plane was running.

The Architecture

┌─────────────────────────────────────────────────────────┐
│                    HOME NETWORK                          │
│                                                          │
│  ┌─────────────────────┐    ┌──────────────────────────┐│
│  │   Ubuntu PC (amd64) │    │   Mac M4 Pro (arm64)     ││
│  │   192.168.101.26     │    │   192.168.101.6          ││
│  │                      │    │                          ││
│  │  ┌────────────────┐ │    │  ┌──────────────────────┐││
│  │  │ Control Plane  │ │    │  │  Multipass VM        │││
│  │  │ • API Server   │ │    │  │  (Ubuntu arm64)      │││
│  │  │ • Scheduler    │ │    │  │                      │││
│  │  │ • etcd         │ │    │  │  ┌──────────────┐   │││
│  │  └────────────────┘ │    │  │  │ k3s agent    │   │││
│  │  ┌────────────────┐ │    │  │  │ (worker)     │   │││
│  │  │ k3s agent      │ │    │  │  └──────────────┘   │││
│  │  │ (worker)       │ │    │  └──────────────────────┘││
│  │  └────────────────┘ │    │                          ││
│  └─────────────────────┘    └──────────────────────────┘│
│              ▲                          ▲                 │
│              └──────── k3s cluster ─────┘                 │
└─────────────────────────────────────────────────────────┘

The Critical Gotcha

k3s doesn't run natively on macOS. Multipass (Canonical's lightweight VM tool) provides the Ubuntu environment, but the default NAT networking creates an isolated subnet. The Ubuntu desktop can't reach the VM. The fix: bridged networking via --network en0, which gives the VM an IP on the home network's subnet. But even with bridging, there's a subtler problem. The VM has two interfaces: one NAT (default), one bridged. Both k3s and flannel (the overlay network) default to the NAT interface for VXLAN tunnels. The node shows Ready, but cross-node pod communication silently fails. The solution is two flags when joining the cluster:

--node-ip=<VM_BRIDGED_IP> --flannel-iface=enp0s2

This tells kubelet to advertise the bridged IP and flannel to route tunnels over the correct interface. Without these, you get a cluster that looks healthy but can't actually distribute work.

What the Cluster Demonstrates

Cross-architecture scheduling. Four nginx replicas deployed: two landed on amd64, two on arm64. The multi-arch image was pulled automatically for each architecture. No manual intervention.

NAME                     NODE                     ARCH
nginx-66686b6766-8tfkt   master-fluffy-ms-7c94    amd64
nginx-66686b6766-gdnt6   k3s-worker               arm64
nginx-66686b6766-r6st2   master-fluffy-ms-7c94    amd64
nginx-66686b6766-xsrb8   k3s-worker               arm64

Cross-node networking. A curl pod on the Ubuntu PC resolved and reached nginx pods on the Mac VM by service name (http://nginx). Flannel's VXLAN overlay made the pod network transparent across physical machines and architectures. Workload migration. Draining the Mac node (kubectl drain k3s-worker) evicted all pods and rescheduled them to the Ubuntu PC. Uncordoning brought it back. This is the operational pattern for real cluster maintenance, invisible in single-node setups. Architecture-aware scheduling. nodeSelector constraints pin pods to specific architectures. Node affinity rules express soft preferences: "prefer arm64, but amd64 is acceptable." These matter for edge deployments and cost optimization with mixed-architecture cloud instances.

Remote Management

Two approaches for managing the cluster from the Mac: SSH alias (simple): wraps kubectl over SSH. Every command opens a connection, runs on the control plane, returns output. Fine for a home lab. SSH tunnel + kubeconfig (proper): forwards port 6443 locally, so kubectl runs natively on the Mac with tab completion and plugins. Requires understanding a k3s-specific TLS quirk: k3s uses separate CAs for server and client authentication, so the SSH tunnel is necessary to present the connection as local.

What This Taught

The flannel networking bug was worth more than the working cluster. It forced me to understand overlay networks at the interface level: which IP gets advertised, which interface carries VXLAN traffic, and why "node shows Ready" doesn't mean "networking works." Single-node Kubernetes teaches you the API. Multi-node Kubernetes teaches you the system.

Stack

k3s v1.34.5 (CNCF-certified lightweight Kubernetes)
Multipass (Canonical's Ubuntu VM manager for macOS)
Flannel (VXLAN overlay network, bundled with k3s)
containerd (container runtime, bundled with k3s)
Ubuntu 24.04 LTS (both control plane and worker VM)

The full step-by-step guide with every command, flag explanation, and troubleshooting section is in the project repository.