The Rook v1.17 release is out! v1.17 is another feature-filled release to improve storage for Kubernetes. Thanks again to the community for all the great support in this journey to deploy storage in production.
The statistics continue to show Rook is widely used in the community:
- 12.7K Github stars
- 341M Container downloads
- 7.3K Slack members
- 7.9K X followers
We have a lot of new features for the Ceph storage provider that we hope you’ll be excited about with the v1.17 release!
Object Stores
ObjectBucketClaims secure by default
Several ObjectBucketClaim options were added previously in Rook v1.16 that allowed more control over buckets. These controls allow users to self-serve their own S3 policies. Administrators may consider this flexibility a risk, depending on their environment. Rook now disables these options by default to ensure the safest off-the-shelf configurations.
To enable the full range of OBC configurations, the new setting `ROOK_OBC_ALLOW_ADDITIONAL_CONFIG_FIELDS` must be set to enable users to set all of these options. For more details, see the OBC additionalConfig documentation.
CephObjectStoreUser Credential Management
Rook has added first-class credential management to CephObjectStoreUser resources, allowing multiple credentials and declarative credential rotation. For more details, see Managing User S3 Credentials.
As a result, existing S3 users provisioned via CephObjectStoreUser resources no longer allow multiple credentials to exist on underlying S3 users, unless explicitly managed by Rook. Rook will purge all but one of the undeclared credentials. This could be a user observable regression for administrators who manually edited/rotated S3 user credentials for CephObjectStoreUsers, and affected users can make use of the new credential management feature as an alternative.
Bucket Owner
The name of a pre-existing Ceph RGW user account can be set as the bucket owner on an ObjectBucketClaim (OBC), rather than a unique RGW user being created for every bucket. A CephObjectStoreUser resource may be used to create the Ceph RGW user account which will be specified on the OBC. If the bucket owner is set on a bucket that already exists and is owned by a different user, the bucket will be re-linked to the specified user.
Kafka auth mechanism
Ceph recently added support for configuring which authentication mechanism to use for bucket notifications using Kafka topics. This can be configured by setting the mechanism field of the Kafka section of the bucket notification.
CSI
Ceph CSI 3.14
The Ceph CSI release has a number of features and improvements for RBD and CephFS volumes, volume snapshots, and many more areas. See the Ceph CSI 3.14 release notes for more details.
Ceph CSI Operator
The Ceph CSI operator is an active area of development and we have decided to keep the feature in experimental mode in this release. To ensure feature compatibility and have full confidence in the testing, we will wait until Rook v1.18 to enable it by default. This additional time to test will help ensure a smooth transition from Rook’s CSI driver orchestration to the CSI operator.
Until then, we encourage you to help us validate the CSI operator by enabling it in a test cluster with two simple steps:
- Install the CSI operator with csi-operator.yaml. This will create the necessary Ceph-CSI Operator CRDs and start the new operator.
- Enable Rook to use the CSI operator by setting ROOK_USE_CSI_OPERATOR: “true” in the rook-ceph-operator-config configmap
Rook will then automatically create the needed CRs to configure the Ceph CSI operator. Please let us know if you have any feedback on the new CSI configuration!
Mon Endpoints
External Mons
In some two-datacenter clusters, there is no option to start an arbiter mon in an independent K8s node to configure a proper stretch cluster. The external mons now allow a mon to be configured outside the Kubernetes cluster, while Rook manages everything else inside the cluster. For more details, see the External Mon documentation. This feature is in currently in experimental mode.
DNS Resolution for Mons
On occasion, the list of mons may change when there is a mon failover during a network or other outage. The CSI driver automatically handles this for the volumes inside the K8s cluster. Now dynamic DNS resolution allows clients outside the K8s cluster to resolve mon endpoints via DNS without requiring manual updates to the list of mon endpoints. This helps in scenarios such as virtual machine live migration. The Ceph client can connect to rook-ceph-active-mons.<namespace>.svc.cluster.local to dynamically resolve mon endpoints and receive automatic updates when mon IPs change. To configure this DNS resolution, see Tracking Mon Endpoints.
Ceph Config
Node specific node.conf overrides
The ceph.conf overrides can now be customized per-node. This may be helpful for some ceph.conf settings that need to be unique per node depending on the hardware. This can be configured by creating a node-specific configmap that will be loaded for all OSDs and OSD prepare jobs on that node, instead of the default settings that are loaded from the rook-config-override configmap.
The node-specific configmaps must have the label:
node.config.rook.io/osd
The configmaps must follow the naming convention:
rook-config-override-<hostname>
Versions
Kubernetes v1.28 — v1.33
Kubernetes v1.28 is now the minimum version supported by Rook through the latest K8s release v1.33 (to be released any day now). Rook CI runs tests against these versions to ensure there are no issues as Kubernetes is updated. If you still require running an older K8s version, we haven’t done anything to prevent running Rook, we simply just do not have test validation on older versions.
What’s Next?
As we continue the journey to develop reliable storage operators for Kubernetes, we look forward to your ongoing feedback. Only with the community is it possible to continue this fantastic momentum.
There are many different ways to get involved in the Rook project, whether as a user or developer. Please join us in helping the project continue to grow on its way beyond the v1.17 milestone!
<hr /><p>Rook v1.17 Storage Enhancements was originally published in Rook Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>