At Ably we recently shipped AI Transport, our drop-in transport layer for streaming LLM output over Ably channels, with all the resumability, multi-device continuity, and handover guarantees that implies.
One of the tricky things about token streaming from a pub/sub point of view is that while a model progressively emits fragments of text, those fragments logically all belong to a single message. You want to deliver them to the user live as they’re produced, but for many other purposes you want