The Rook v1.18 release is out! v1.18 is another feature-filled release to improve storage for Kubernetes. Thanks again to the community for all the great support in this journey to deploy storage in production.
The statistics continue to show Rook is widely used in the community, with over 13K Github stars, and Slack members and X followers constantly increasing.
If your organization deploys Rook in production, we would love to hear about it. Please see the Adopters page to add your submission. As an upstream project, we don’t track our users, but we appreciate the transparency of those who are deploying Rook!
We have a lot of new features for the Ceph storage provider that we hope you’ll be excited about with the v1.18 release!
Ceph CSI Operator
The Ceph CSI operator is now the default and recommended component for configuring CSI drivers for RBD, CephFS, and NFS volumes. Until now, the CSI drivers have been automatically configured by Rook. Now the CSI operator has been factored out of Rook to run independently to manage the Ceph-CSI driver.
During the upgrade and throughout the v1.18.x releases, Rook will automatically convert any Rook CSI settings to the new CSI operator CRs. This transition is expected to be completely transparent. In the future v1.19 release, Rook will relinquish direct control of these settings so advanced users can have more flexibility when configuring the CSI drivers. At that time, we will have a guide on configuring these new Ceph CSI operator CRs directly.
For now, be aware of the following:
- During install, as mentioned in the Quickstart Guide, there is a new manifest to be created: csi-operator.yaml
- If installing with the helm chart, the Ceph CSI operator will automatically be installed by default with the new helm setting csi.rookUseCsiOperator in the rook-ceph chart.
The CSI operator has been in experimental mode since Rook v1.15 and we have confidence in its stability. That said, if you find any blocking issue, the previous CSI driver can be re-enabled by setting ROOK_USE_CSI_OPERATOR: false in operator.yaml or applying the helm setting csi.rookUseCsiOperator: false. In the event that you disable the csi operator, please open an issue to report the problem!
Ceph CSI 3.15
The v3.15 release of Ceph CSI has a range of features and improvements for the RBD, CephFS, and NFS drivers. This release is supported both by the Ceph CSI operator and Rook’s direct mode of configuration. In the next release (targeted for December), the Ceph CSI operator will be required to configure the CSI driver. As mentioned previously, we will provide a guide for customizing the CSI settings with that release.
CephX Key Rotation
This feature is *experimental*.
Ceph and Rook teams have been working together to implement key rotation capabilities for the CephX authentication keys used by Ceph daemons and clients. This release introduces a new, experimental API for CephX key rotation. Users will begin to see new cephx status items on some Rook resources in newly-deployed Rook clusters. Users can also find spec.security.cephx settings that allow initiating CephX key rotation for various Ceph components. Some caveats should be noted:
- Only Ceph versions v19.2.3 and higher have the capabilities Rook requires for key rotation.
- Rook’s Ceph admin key cannot yet be rotated. Rook expects admin key rotation to be enabled in a Rook v1.18 patch release after more extensive testing to ensure enterprise reliability.
- Ceph Monitor (mon) keys cannot be rotated yet in Ceph v19 due to Ceph architecture limitations. Rook and Ceph are working closely to ensure mon key rotation is achievable safely for enterprise deployments in a future Ceph release.
Full documentation for key rotation can be found here.
Node Topology Validation
OSD topology is configurable via node labels when the OSDs are being created. Ceph has several requirements to ensure the topology is valid when creating the CRUSH map. Until now, if the topology is invalid, a difficult-to-understand error was only found in the Rook operator log. To better ensure a valid topology, before creating OSDs, Rook will now validate the topology. If the topology is invalid, Rook will refuse to create new OSDs, and a detailed error will be found in the operator log. For example, here are some invalid topologies:
- A topology name must be unique across the cluster. For example, a rack and a zone cannot have the same name.
- The sub-topology of the same name cannot exist across different failure domains. For example, two different zones cannot contain the same rack.
If existing clusters detect an invalid topology, Rook will only log a warning and continue.
Resource IDs
Storage classes most commonly provision volumes from pools (RBD) and filesystems (CephFS). For greater separation with multi-tenancy, it is also possible to use a specific RADOS namespace (RBD) or subvolume group (CephFS). Creating a storage class based on those resources required an ID to be generated by Rook, which could then be parsed from the CR status in order to create the storage class. To make this flow simpler, now the clusterID can be declared on the related CR. See these documents to create a storage class based on these resources:
Mon failover
In various scenarios, a mon may be failed over automatically if the mon has fallen out of quorum. This has always waited for a timeout (10–20 minutes) to first give the mon a chance to come back into quorum. Now, if the assigned K8s node no longer exists, the mon will be failed over immediately instead of waiting for the timeout.
Versions
Ceph Tentacle v20
In the next month or two, we anticipate the Ceph team will be releasing the Ceph Tentacle (v20.2.0) release. Rook has already been testing against this release and is preparing for some new features available with this release.
As soon as the Ceph release is out, we will add official support to Rook. Stay tuned! In the meantime, Rook continues to test against Ceph Reef v18 and Ceph Squid v19.
Kubernetes v1.29 — v1.34
Kubernetes v1.29 is now the minimum version supported by Rook through the latest K8s release v1.34 (to be released any day now). Rook CI runs tests against these versions to ensure there are no issues as Kubernetes is updated. If you still require running an older K8s version, we haven’t done anything to prevent running Rook, we simply just do not have test validation on older versions.
Helm v3.13+
Previously, only the latest version of helm was tested and the docs stated only version 3.x of helm as a prerequisite. Now Rook supports the six most recent minor versions of helm along with their their patch updates. Specifically, Helm versions 3.13 and newer are now supported.
What’s Next?
As we continue the journey to develop reliable storage operators for Kubernetes, we look forward to your ongoing feedback. Only with the community is it possible to continue this fantastic momentum.
There are many different ways to get involved in the Rook project, whether as a user or developer. Please join us in helping the project continue to grow on its way beyond the v1.17 milestone!
<hr /><p>Rook v1.18 Storage Enhancements was originally published in Rook Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>