The Rook v1.16 release is out! v1.16 is another feature-filled release to improve storage for Kubernetes. Thanks again to the community for all the great support in this journey to deploy storage in production.
The statistics continue to show Rook community growth since the v1.15 release in August:
- 12.2K to 12.5K Github stars
- 333M to 337M Container downloads
- 6.8K to 7.1K Slack members
- 7.7K to 7.9K X followers
We have a lot of new features for the Ceph storage provider that we hope you’ll be excited about with the v1.16 release!
Object Stores
One of the themes the Rook team has been seeing for a while is an expanding set of advanced object store use cases. This Rook release focuses heavily on expanding object store options and flexibility.
Multiple Instances
Multiple object stores can now be configured to be backed by the same pools. There are various configurations that can be unique between the object stores:
- Hosting and gateway configurations on different ports and domains
- Different protocols to support S3, Swift, and/or Admin-Ops
- Resources and limits unique per RGW instance
This feature helps significantly increase the flexibility of the object stores. See the multi-instance documentation for more details on configuring these options.
S3 Bucket Policy
Object Bucket Claims can specify a bucket policy for the S3 bucket. The policy is applied with the additionalConfig.bucketPolicy field. The policy is specified as a JSON blob and supports all bucket policy that the Ceph RGW supports.
Arbitrary Object Store Configuration
Rook always seeks to maintain a well-designed and well-tested feature set. Maintaining this high quality at a pace that meets users has been a challenge. To help address a broad range of new features for CephObjectStore users wanting to push the limits of their object store, Rook has introduced two mechanisms for advanced configuration of Rook’s RGW daemons.
- Set arbitrary RGW config and commandline flag values
- Add arbitrary Pod volume mounts to the RGW pod with additionalVolumeMounts
The Rook team is excited to see what users are able to accomplish with the freedom afforded by these options. We also encourage users to share their uses with us and contribute suggestions that other users might benefit from.
RGW Operation Logging
By default, all RGW logs are included in the main RGW container logs. For enhanced observability of the operations, the RGW operations logs can be separated from the main RGW container logs. This is enabled by a sidecar with the opsLogSidecar setting found under the gateway settings.
Bulk Data Pool
The object store data pool is now marked as bulk: true by default. This flag enables the full set of PGs to be created immediately instead of growing over time as the PG autoscaler might determine. Since the data pool for an object store is expected to be large, PG placement will be better optimized with this flag.
Ceph CSI v3.13
The latest Ceph-CSI release has a number of features and improvements, including:
- Volume group snapshots for RBD volumes
- CephFS support for the omap data store in rados namespaces
OSDs Replacement
While many settings are configurable at runtime on OSDs with Ceph settings, some OSD settings are immutable. For example, encryption cannot be enabled or disabled on the OSD after it is created. The only way to change such settings is by purging the OSD, wiping the device, creating a new OSD in its place, and waiting for the data to backfill.
The OSDs can be replaced by updating the desired settings, then setting the value storage.migration.confirmation: yes-really-migrate-osds in the CephCluster CR. Rook will then automating replacing the OSDs one by one. See the OSD migration documentation for more details.
RADOS Namespace Mirroring
RADOS namespaces can be mirrored across clusters, similar to mirroring a pool or RBD image. Enable the RADOS namespace mirroring with the mirroring settings in the CephBlockPoolRadosNamespace CR. This feature is experimental as it requires Ceph Tentacle (v20), which is not planned for release until some time in 2025.
If the statusCheck is enabled on the parent CephBlockPool, the status of the RADOS namespace mirroring will be updated periodically in the status field of the CephBlockPoolRadosNamespace CR.
Versions
Ceph Quincy Support Removed
Ceph Quincy v17 has reached end of life and is no longer supported by the Ceph team. Rook has removed support for Quincy in this release. Clusters must be upgraded to at least Reef v18 before upgrading to Rook v1.16. Rook continues to support Reef v18 and Squid v19.
Kubernetes v1.27 - v1.32
Kubernetes v1.27 is now the minimum version supported by Rook through the latest K8s release v1.32, which means we run CI tests against v1.27 through v1.32. If you still require running an older K8s version we haven’t done anything to prevent running Rook, we simply just do not have test validation on older versions.
What’s Next?
As we continue the journey to develop reliable storage operators for Kubernetes, we look forward to your ongoing feedback. Only with the community is it possible to continue this fantastic momentum.
There are many different ways to get involved in the Rook project, whether as a user or developer. Please join us in helping the project continue to grow on its way beyond the v1.16 milestone!
<hr /><p>Rook v1.16 Storage Enhancements was originally published in Rook Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>