Migrating AOS persistent metadata to Flatbuffers with Zero Logic Changes

If you work on a large-scale distributed storage system long enough, you may eventually hit a wall where your data serialization format becomes a big CPU bottleneck.

For some time in the Nutanix Core Data path, our metadata was persistently stored and communicated using Google’s Protocol Buffers (Protobuf). Protobuf is mature, strongly typed, and excellent for many use cases. But, as our clusters scaled, metadata became more complex and, as we pushed millions of IOPS, profiling revealed a glaring issue: the CPU overhead and memory allocations required to serialize and deserialize these complex Protobufs were artificially capping our throughput. Every cycle spent unpacking metadata was a cycle stolen from serving user I/O.

The Problem

To understand why Protobuf choked our CPU, you have to visualize what deserialization actually does. Protobuf is beautifully compressed on disk. But, to read it, the CPU must parse every tag, allocate C++ objects on the heap, copy strings into new memory locations, and align pointers. A dense 14KB payload on disk explodes into a 45KB web of fragmented C++ objects in memory.

 Flatbuffers take the exact opposite approach. The data is written to disk already padded, aligned, and mapped with offsets. When we load it into memory, there is no decoding. We simply cast a pointer to the start of the byte array and read the data directly. We traded disk density for zero-copy memory access.

After careful study, we chose to move to Flatbuffers. By trading a larger payload footprint for massive CPU and memory efficiency, we could achieve a completely flat, near-zero deserialization time regardless of metadata complexity. For typical storage systems, where reads drastically outnumber writes – TPC-E, the most recent transactional TPC benchmark specification, has a ~9:1 R:W ratio. Therefore, eliminating this deserialization penalty in the read-path is absolutely critical.

Constraints

But migrating a live, hyper-scale storage system presents two massive, seemingly impossible constraints:

1. The Data Constraint: We had hundreds of gigabytes of existing metadata sitting persistently on disks in Protobuf format. A cluster-wide, blocking background job to rewrite all of this to Flatbuffers would destroy foreground performance and risk downtime.
2. The Developer Constraint: Flatbuffers are fundamentally different to construct than Protobufs. Asking our engineers to rewrite thousands of lines of complex state-machine logic just to populate Flatbuffers wasn’t feasible.

In this post, I’ll walk through how we engineered a transparent migration by building a system that was completely bilingual on disk – reading legacy Protobufs but writing new Flatbuffers – and how we modified the compiler itself to allow clients to adopt the new format simply by updating class names, with zero changes to their underlying code flow or business logic.

The Solution: Bridging the API Gap at Compile Time

To understand why a transparent wrapper was necessary, you have to look at how fundamentally different the construction paradigms are between Protobuf and Flatbuffers.

With Protobuf, the generated classes act like standard mutable objects. A developer can instantiate an object, pass it around different functions, and set fields in a completely arbitrary order. Flatbuffers, however, are built “bottom-up.” You must create child objects (like strings or nested tables) before you create the parent object that references them. Forcing our engineering teams to rewrite thousands of call sites to strictly follow a bottom-up construction order would have been a massive and arduous exercise.

We needed a drop-in replacement interface that mirrored the Protobuf API perfectly, but manually writing and maintaining these wrapper classes for thousands of changing schemas was out of the question.

Hacking the Compiler: Auto-Generating the Bilingual Interface

To overcome this hurdle, we modified the Flatbuffer compiler (flatc) itself to do the heavy lifting for us. We extended flatc so that during the build process, it ingests both the legacy .proto files and the new .fbs (Flatbuffer) schema files. Using both schemas, our modified compiler auto-generates a unified, bilingual interface class.

To the client code, this auto-generated interface exposes the exact same setter and getter methods they were already using (e.g., set_timestamp(), mutable_child_node()). The only code change required by the client teams was a simple search-and-replace to update the class namespace they were instantiating. The business logic and control flow remained 100% untouched.

Under the hood, this generated interface acts as a transparent translation layer. It handles the state accumulation required to bridge the gap between Protobuf’s random-order mutations and Flatbuffer’s strict bottom-up construction. When reading from the disk or the network, the interface layer dynamically routes access to either the legacy Protobuf object or the zero-copy Flatbuffer, completely transparent to the client.

Because this was all handled at compile time, developers simply continued writing schemas as usual. The build system automatically provided the bilingual classes that could read and write legacy Protobufs or Flatbuffers.

The “Lazy” On-Disk Migration

Solving the developer experience with an auto-generated compiler was only half the battle. The second, arguably more terrifying constraint was the sheer gravity of our existing data.

At Nutanix scale, our storage layer manages hundreds of gigabytes of persistent metadata. Historically, all of this was written to disk as Protocol Buffers. If we had attempted a traditional migration, forcing the cluster into a read-only state while a massive background job parsed and rewrote billions of metadata entries into Flatbuffers,we would have tanked foreground I/O performance and caused unacceptable downtime for our customers.

We needed a way to upgrade the airplane while it was flying. Enter our strategy for a “lazy,” mixed-state metadata migration.

The Bilingual Disk

Instead of forcing a mass conversion, we designed our storage layer to become completely bilingual at the storage layer. We updated our read/write paths so that the system could natively handle a disk where some metadata entities were legacy Protobufs, and others were newly minted Flatbuffers.

Here is how the lifecycle worked in practice:

• The Read Path: When our storage layer needed to read an entity’s metadata from disk, it inspected the payload header. If it detected a legacy Protobuf, our auto-generated interface dynamically parsed it into memory. If it detected a Flatbuffer, it achieved the zero-copy read directly. To the higher-level logic, both structures looked identical.
• The Write Path: The actual migration happened invisibly during standard I/O operations. If an entity was read as a Protobuf, modified by the system, and then flushed back to disk, the interface always serialized the updated state as a new Flatbuffer.

Hot Data Migrates Itself

The beauty of this architecture was its organic efficiency.

For hot data, where the metadata is actively being accessed and modified, naturally and rapidly converted itself to Flatbuffers simply by existing in the normal I/O path. For cold data (snapshots or archived metadata) sitting untouched remains safely stored as Protobufs with zero overhead. There was no need to waste precious CPU cycles or disk bandwidth rewriting cold data that wasn’t being actively queried.

With this, the cluster comfortably existed in a mixed state. We achieved a massive architectural shift with zero downtime, zero blocking background jobs, and a natural, phased transition that prioritized our most critical, latency-sensitive data first.

Under the Hood: Storage and Routing Mechanics

Abstracting the serialization layer of a live database sounds great in theory, but the actual implementation required solving complex memory and routing challenges. To make this project work, we had to build a system that dynamically routed operations to the correct backend engine.

The Magic Header Routing

How does a storage node instantly know if it’s reading a legacy Protobuf or a new Flatbuffer without taking a deserialization penalty?

We implemented a lightweight, unified MessageHeader struct that prepended every payload on disk. When our storage layer reads a payload, it simply peeks at the header’s magic bytes. If it sees 0xFEEEFEEE, the system routes the payload to the Protobuf parser. If it sees 0xACEDFEED, it bypasses parsing entirely and routes the payload to the zero-copy Flatbuffer engine. This routing happens in microseconds, completely transparent to the higher-level application logic.

The Tri-State Machine

Under the hood, our auto-generated interface classes didn’t just handle two states; they managed a strict tri-state machine represented by a unified BaseMessage class:

1. kProto: The payload was read as a legacy Protobuf and instantiated into memory.
2. kFlatbufferParsed: The payload was read as a new Flatbuffer. The interface class simply held a raw pointer directly to the underlying bytes (zero-copy), acting as a read-only view.
3. kFlatbuffer (The Staging State): The state used when a developer is actively mutating or constructing a new message from scratch.

The Developer Experience: Defeating the “Bottom-Up” Constraint

Flatbuffers are incredibly fast, but they demand a strict “bottom-up” construction. You cannot build a parent object until all of its children (strings, vectors, nested messages) have been created and their offsets collected.

In a system like Nutanix, an I/O operation is a complex, multi-stage state machine. We rarely have all the metadata available at the start of an operation. Instead, we populate pieces of the metadata incrementally as they are calculated or deduced across different stages of the pipeline.

Let’s look at a sanitized example from our internal test suite to see why native Flatbuffers would have broken this paradigm.

To set the stage, here is a simplified look at the data structures we are building. The schema involves a root message (Message1) containing scalar fields, vectors of strings, and nested child messages (Message2).

Here is how the legacy Protobuf definition looks:

message Message2Proto {
  repeated int32 rep_int = 1;
  optional int32 opt_int2 = 3;
  optional col.Color color = 5 [default = Red];
}

message Message_1Proto {
  optional int32 opt_int = 1;
  repeated string rep_string = 2;
  repeated int32 rep_int = 3;
  optional Message2Proto opt_msg = 5;
  repeated Message2Proto rep_msg = 6;
}

And here is the corresponding Flatbuffer definition we needed to migrate to:

table Message2 {
  rep_int : [int] (id: 0);
  opt_int2 : int (id: 2);
  color : _nutanix._test._col.Color = Red (id: 4);
}

table Message1 {
  opt_int : int (id: 0);
  rep_string : [string] (id: 1);
  rep_int : [int] (id: 2);
  opt_msg : Message2 (id: 4);
  rep_msg : [Message2] (id: 5);
}
root_type Message1;

The Protobuf Way 

With Protobuf, the construction is highly intuitive and perfectly suited for a state machine. You can pass a mutable object around and set fields or append to lists as the data becomes available, in any random order:

// Standard Protobuf Construction: Piecemeal & Intuitive
Message_1Proto m1_p;
// We can set scalar fields natively
m1_p.set_opt_int(48);

// We can append to repeated fields randomly
m1_p.add_rep_int(4);
m1_p.add_rep_string("Hello");

// We can instantiate nested messages and set their fields piecemeal
auto m2 = m1_p.mutable_opt_msg();
m2->set_opt_int2(444);

// Now set a field in the parent message.
m1_p.add_rep_int(8);
// And another one in the child.
m2->set_color(col::Color::Green);

The Native Flatbuffer Way 

If we had forced our feature teams to use the native Flatbuffer C++ API, that exact same logical flow would have been destroyed.

Because of the bottom-up requirement, developers cannot incrementally populate a parent message. They would be forced to manually manage vectors of offsets, temporarily cache calculated strings, and ensure every nested object (like Message2) is completely finalized before they even attempt to construct the parent (Message1):

// Native Flatbuffer Construction: Bottom-Up & Rigid
flatbuffers::FlatBufferBuilder builder;
builder.ForceDefaults(true);

// 1. Child elements MUST be created and their offsets stored manually
std::vector<flatbuffers::Offset<flatbuffers::String>> str_offsets;
str_offsets.push_back(builder.CreateString("Hello"));
std::vector<int32_t> rep_int = {4, 8};

// 2. Build the nested message using direct offsets
auto m2o = CreateMessage2Direct(builder, nullptr, nullptr, 444, nullptr, 
                                _nutanix::_test::_col::Color::Green);

// 3. Finally, you can build the parent message using the manual offsets
auto m1o = CreateMessage1Direct(builder, 48, &str_offsets, &rep_int, nullptr, m2o);

FinishMessage1Buffer(builder, m1o);

Forcing this paradigm onto a complex, piecemeal state machine would require passing dozens of flatbuffers::Offset objects between functions, resulting in a massive, unreadable refactor across the entire storage layer codebase.

The Solution (The Drop-In Replacement)

We aimed to completely shield our engineers from this complexity. Using our auto-generated interface, developers wrote code that looked and behaved almost identically to the legacy Protobuf implementation.

Under the hood, our BaseMessage interface buffered the piecemeal state additions into memory efficient data-structures. Any elements which could be directly written to the underlying flatbuffer were pass-through. For complex data types (strings, vectors, nested messages), data was buffered so that the developer was free to mutate the message in whatever chaotic, random order their business logic dictated.

Here is how the exact same message is constructed using the interface class:

// We instantiate the auto-generated bilingual interface class with type 
// kFlatbuffer to indicate that the underlying format is a flatbuffer.
TestMessage1 m1(arena, nutanix::MessageType::kFlatbuffer);

// The API looks and acts exactly like Protobuf!
m1.set_opt_int(48);
m1.add_rep_int(4);
m1.add_rep_string("Hello");

// Nested messages are handled transparently
auto m2 = m1.mutable_opt_msg();
m2->set_opt_int2(444);

m1.add_rep_int(8);
m2->set_color(col::Color::Green);

// The actual strict, bottom-up Flatbuffer packing happens here.
auto buffer = m1.Serialize();

Only when the state machine completed its run and the system called Serialize() did we traverse the staging area and execute the complex Flatbuffer packing logic.

Zero logic changes. Zero manual offset management. 100% of the Flatbuffer performance.

To make this illusion as performant as possible, the wrapper uses a hybrid construction model. Fixed-size scalars and plain old data (POD) types are passed efficiently, but dynamic types like vectors and strings, which Flatbuffers require to be built out-of-order, are intercepted. When a developer mutates a string, it gets safely parked in an in-memory staging area. Only when Serialize() is called does the wrapper flush the staging area into the final Flatbuffer binary, perfectly aligning the offsets.

Results and Takeaways: Trading Bytes for CPU Cycles

Benchmarks revealed the classical systems trade-off: we were trading network and disk footprint for massive CPU and memory efficiency.

Using representative scaled metadata payloads, here is what the relative performance revealed:

The Massive Wins: CPU and Memory

The primary goal of the migration was to eliminate the “serialization tax,” and the Flatbuffer architecture delivered spectacular reductions in overhead:

• Zero-Cost Deserialization: As metadata entries grew in complexity, Protobuf deserialization time scaled linearly and steeply. In contrast, Flatbuffer deserialization time remained a completely flat line near zero, regardless of the entry size.
• Cumulative Speedup: Thanks to completely bypassing the parsing phase, the total API call time for batch lookups (where multiple metadata entries are looked up together) plummeted. At larger payload scales, Flatbuffers achieved a roughly 60% reduction in total lookup latency compared to Protobufs.  This is purely due to CPU savings.
• Reduced Memory Footprint: Unpacked Protobuf objects consume significant heap space. Flatbuffers proved to be much more memory-efficient when loaded into RAM, yielding a ~40% reduction in memory consumption for the same data. This in turn, allows us to cache more metadata.

The Trade-Off: Payload Size and Network RPCs

Flatbuffers achieve their zero-copy speed by storing data on disk and over the wire exactly as it is represented in memory, complete with padding and offsets.

• The Disk/Wire Footprint: Protobuf is vastly superior at packing data densely. Our benchmarks showed that Flatbuffer payloads were roughly 100% larger (double the size) on the wire compared to Protobuf.
• Network Transfer Times: Because Flatbuffer payloads are physically larger, the actual time spent pushing the bytes over the network during a batch lookup RPC call saw a ~30% increase. However, the massive CPU time we saved by completely bypassing deserialization heavily outweighed this network penalty.

Additional Considerations & Tooling

Migrating a core serialization protocol across a massive codebase is notoriously difficult, but the success of the project came down to our investment in tooling and fallback mechanisms.

There are a few critical considerations that made this a truly seamless transition:

1. A Truly Zero-Touch Compiler Pipeline: We didn’t just write wrapper classes by hand; we modified the flatc compiler itself. The compiler takes the existing .proto metadata definitions as its input. During the build process, it automatically generates the corresponding .fbs file, the native Flatbuffer C++ code, and the bilingual Interface class. It is a completely automated pipeline.
2. Graceful Handling of Legacy Metadata: We designed the system around “Magic Headers” , but terabytes of existing metadata on disk were written years before these headers existed. To handle this, our parsing logic includes a safe fallback. If the parser encounters an unrecognizable header (or no header at all), it gracefully falls back to treating the raw buffer as a legacy Protobuf. The system natively digests the old data and automatically attaches the new magic headers the next time that data is modified and written back to disk.
3. A One-Line Code Change: Because the compiler auto-generates an interface that perfectly mimics the Protobuf setters and getters, and because the C++ staging area handles the complex Flatbuffer offset math, the feature developers had virtually no work to do. The only requirement for a feature team to adopt this massive architectural shift was a simple Find-and-Replace to update the class name being instantiated in their code.

Conclusion

This project proved that in modern hyper-converged infrastructure, CPU cycles are often far more precious than payload bytes.

By taking the time to build a compiler-generated wrapper, a bilingual tri-state machine, and a defensive read-path, we migrated our core data path to zero-copy Flatbuffers with zero downtime. Most importantly, we protected our developer experience. Teams adopted a massive architectural shift simply by changing the class namespaces they instantiated, allowing product velocity to remain completely uninterrupted.

With rigorous systems engineering, and a little bit of compiler magic, you really can rebuild the airplane while it’s flying.