CEPH Rook clean cluster

Errors related

The admin wont mount, you can check logs of mgr container and see something like "not ready, certificat not created"

Delete your deployment

kubectl delete -f ./out_ombrella_chart.yaml

If you got some errors, clean from sqlite

sudo systemctl stop k3s
sudo sqlite3 /var/lib/rancher/k3s/server/db/state.db
DELETE FROM kine WHERE name like '%ceph%';
DELETE FROM kine WHERE name like '%rook%';
.exit
sudo systemctl start k3s

You also need to clean each node.

sudo systemctl stop k3s-agent
sudo rm -rf /var/lib/rook
sudo rm -rf /var/lib/ceph
sudo rm -rf /etc/ceph
sudo rm -rf /var/log/ceph

# clean mount
mount | grep ceph
sudo umount -lf /var/lib/rook/*
sudo umount -lf /var/lib/ceph/*

# clean drive
sudo wipefs -a /dev/nvme0n1
sudo sgdisk --zap-all /dev/nvme0n1
sudo dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=100 oflag=direct,dsync
sudo partprobe
# check
lsblk -f /dev/nvme0n1

#device mapper
sudo dmsetup ls
sudo dmsetup remove_all

sudo systemctl start k3s-agent

And now you can redeploy safely, starting by crds.