Run VPN inside Kubernetes

Imagine if VPN is ran right inside Kubernetes, with right configurations, theoretically you should be able to pretend like you are living inside it, which means all internal services will be resolvable and accessible

Here is an sample manifest of how we may self host VPN righ inside kubernetes:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: vpn
  labels:
    app: vpn
spec:
  selector:
    matchLabels:
      app: vpn
  template:
    metadata:
      labels:
        app: vpn
    spec:
      nodeSelector:
        kubernetes.io/os: linux
      volumes:
      - name: lib-modules
        hostPath:
          path: /lib/modules
          type: Directory
      containers:
      - name: vpn
        image: hwdsl2/ipsec-vpn-server
        imagePullPolicy: IfNotPresent
        env:
        - name: VPN_IPSEC_PSK
          value: xxxxxxxxxxxxx
        - name: VPN_ADDL_USERS
          value: user1 user2
        - name: VPN_ADDL_PASSWORDS
          value: pass1 pass2
        - name: VPN_USER
          value: user0
        - name: VPN_PASSWORD
          value: pass0
        - name: VPN_DNS_SRV1
          value: 10.0.0.10
        - name: VPN_L2TP_NET
          value: 10.1.0.0/8
        - name: VPN_L2TP_LOCAL
          value: 10.1.0.1
        - name: VPN_L2TP_POOL
          value: 10.1.1.1-10.1.1.254
        ports:
        - containerPort: 500
          protocol: UDP
        - containerPort: 4500
          protocol: UDP
        resources:
          limits:
            cpu: 500m
            memory: 500Mi
          requests:
            cpu: 100m
            memory: 64Mi
        securityContext:
          privileged: true
        volumeMounts:
        - name: lib-modules
          mountPath: /lib/modules
---
apiVersion: v1
kind: Service
metadata:
  name: vpn
spec:
  type: LoadBalancer
  selector:
    app: vpn
  ports:
    - name: port
      port: 500
      protocol: UDP
    - name: nat
      port: 4500
      protocol: UDP

Key things here are:

  • make sure to tune VPN_L2TP_NET so it will cover Kubernetes Pod CIDR
  • make sure to set correct VPN_DNS_SRV1 pointing to a CoreDNS
  • optionally tune CoreDNS to resolve ingress to internal address

And here you go, from my experiments after connecting to such VPN I was able to:

# resolve service, as myapp.mynamespace.svc.cluster.local pointing to private 10.0.0.0 addres
nslookup myapp

# talk to this service
curl http://myapp/

# resolve ingress (after optional changes to CoreDNS) pointing to ingress private IP address
host myapp.mac-blog.org.ua

# talk to this service via private network
curl http://myapp.mac-blog.org.ua/

# talk to any other service in kubernetes
nc -vz myapp-redis 6379

Note: macOS has a tricky DNS, I'm still not sure if I understand it to be fair, but the last commands that did helped out to resolve services were:

# networksetup -listnetworkserviceorder
# networksetup -listallnetworkservices

# # did not work
# networksetup -getsearchdomains vpn
# networksetup -setsearchdomains vpn mynamespace.svc.cluster.local

# # worked out
networksetup -getsearchdomains Wi-Fi
networksetup -setsearchdomains Wi-Fi mynamespace.svc.cluster.local

# check
dns-sd -q myapp

So suddenly it is like living inside Kubernetes, there is no need for endless kubectl port-forward everything just works out of the box

Alternative and popular Wireguard somehow did not work inside Azure, and there is OpenVPN alternative but both require additional clients to be installed, and this approach works out of the box with native software