Piraeus on Kubernetes

tags: tech

@nanibot 28/06/2025

I’ve been a big fan of Rook for simplifying storage on Kubernetes. Sure, CEPH is complex but the Rook operator makes it very easy to get a CEPH cluster up and running on Kubernetes. However, it’s common knowledge that CEPH is usually meant for large clusters and is resource intensive.

I’ve been wanting to try out Linstor for quite some time now. In this blog post, we’ll use the Piraeus Operator to deploy a Linstor cluster and use it to provision volumes on our Kubernetes cluster.

We’ll also ensure that TLS is configured for communication between the components.

DRBD kernel module

Linstor depends on the DRBD kernel module. You will need to ensure that the kernel module is loaded before you attempt to install the operator and configure a cluster.

On Talos, this is how you’d do it:

Use https://factory.talos.dev to get a Talos Linux image with the DRBD module built-in.
Use the following patches to load the kernel module(s).

	machine:
	  kernel:
	    modules:
	      - name: drbd
	        parameters:
	          - usermode_helper=disabled
	      - name: drbd_transport_tcp
	      - name: dm-thin-pool

Verify that the module(s) are loaded:

	talosctl --talosconfig talosconfig -n 192.168.16.12 get modules

Sample output:

	NODE            NAMESPACE   TYPE               ID                   VERSION
	192.168.16.12   runtime     KernelModuleSpec   dm-thin-pool         1
	192.168.16.12   runtime     KernelModuleSpec   drbd                 1
	192.168.16.12   runtime     KernelModuleSpec   drbd_transport_tcp   1

Installing the Operator

Follow the official instructions here: Link

Create the Linstor Cluster

Generate the required certificates using cert-manager (we’ll use these certificates to enable TLS communication between the various components)

	---
	apiVersion: cert-manager.io/v1
	kind: Issuer
	metadata:
	  name: ca-bootstrapper
	  namespace: piraeus-datastore
	spec:
	  selfSigned: { }
	---
	apiVersion: cert-manager.io/v1
	kind: Certificate
	metadata:
	  name: linstor-internal-ca
	  namespace: piraeus-datastore
	spec:
	  commonName: linstor-internal-ca
	  secretName: linstor-internal-ca
	  duration: 87600h
	  isCA: true
	  usages:
	    - signing
	    - key encipherment
	    - cert sign
	  issuerRef:
	    name: ca-bootstrapper
	    kind: Issuer
	---
	apiVersion: cert-manager.io/v1
	kind: Issuer
	metadata:
	  name: linstor-internal-ca
	  namespace: piraeus-datastore
	spec:
	  ca:
	    secretName: linstor-internal-ca
	---
	apiVersion: cert-manager.io/v1
	kind: Certificate
	metadata:
	  name: linstor-api-ca
	  namespace: piraeus-datastore
	spec:
	  commonName: linstor-api-ca
	  secretName: linstor-api-ca
	  duration: 87600h # 10 years
	  isCA: true
	  usages:
	    - signing
	    - key encipherment
	    - cert sign
	  issuerRef:
	    name: ca-bootstrapper
	    kind: Issuer
	---
	apiVersion: cert-manager.io/v1
	kind: Issuer
	metadata:
	  name: linstor-api-ca
	  namespace: piraeus-datastore
	spec:
	  ca:
	    secretName: linstor-api-ca

Create the Linstor cluster

	apiVersion: piraeus.io/v1
	kind: LinstorCluster
	metadata:
	  name: linstor-cluster
	spec:
	  apiTLS:
	    certManager:
	      name: linstor-api-ca
	      kind: Issuer
	  internalTLS:
	    certManager:
	      name: linstor-internal-ca
	      kind: Issuer
	  properties:
	    - name: DrbdOptions/Net/tls
	      value: "yes"
	  nodeAffinity:
	    nodeSelectorTerms:
	      - matchExpressions:
	          - key: node-role.kubernetes.io/control-plane
	            operator: DoesNotExist

Create the LinstorSatelliteConfiguration

	apiVersion: piraeus.io/v1
	kind: LinstorSatelliteConfiguration
	metadata:
	  name: linstor-satellite-configuration
	spec:
	  nodeAffinity:
	    nodeSelectorTerms:
	      - matchExpressions:
	          - key: node-role.kubernetes.io/control-plane
	            operator: DoesNotExist
	  internalTLS:
	    tlsHandshakeDaemon: true
	    certManager:
	      name: linstor-internal-ca
	      kind: Issuer
	  storagePools:
	    - name: volume-group-1
	      lvmThinPool:
	        volumeGroup: vg1
	      source:
	        hostDevices:
	          - /dev/vdb
	  podTemplate:
	    spec:
	      initContainers:
	        - name: drbd-shutdown-guard
	          $patch: delete
	        - name: drbd-module-loader
	          $patch: delete
	      volumes:
	        - name: run-systemd-system
	          $patch: delete
	        - name: run-drbd-shutdown-guard
	          $patch: delete
	        - name: systemd-bus-socket
	          $patch: delete
	        - name: lib-modules
	          $patch: delete
	        - name: usr-src
	          $patch: delete
	        - name: etc-lvm-backup
	          hostPath:
	            path: /var/etc/lvm/backup
	            type: DirectoryOrCreate
	        - name: etc-lvm-archive
	          hostPath:
	            path: /var/etc/lvm/archive
	            type: DirectoryOrCreate

Note: The podTemplate field is required only if you're running Talos

Create the storage class:

	apiVersion: storage.k8s.io/v1
	kind: StorageClass
	metadata:
	  name: piraeus-storage-replicated
	  annotations:
	    storageclass.kubernetes.io/is-default-class: "true"
	provisioner: linstor.csi.linbit.com
	allowVolumeExpansion: true
	volumeBindingMode: WaitForFirstConsumer
	parameters:
	  linstor.csi.linbit.com/storagePool: volume-group-1
	  linstor.csi.linbit.com/placementCount: "2"

Verify storage and TLS

	kubectl exec -n piraeus-datastore deploy/linstor-controller -- linstor storage-pool list

	kubectl exec -n piraeus-datastore deploy/linstor-controller -- linstor node list

	kubectl exec -n piraeus-datastore deploy/linstor-controller -- curl --key /etc/linstor/client/tls.key \
	--cert /etc/linstor/client/tls.crt --cacert /etc/linstor/client/ca.crt \
	https://linstor-controller.piraeus-datastore.svc:3371/v1/controller/version

	kubectl logs -l app.kubernetes.io/component=linstor-satellite -c ktls-utils