A model streams tokens, but a user reads a message. Store the tokens instead of the message, and every client that joins mid-response has to rebuild the response.
Need help?
Contact usA model streams tokens, but a user reads a message. Store the tokens instead of the message, and every client that joins mid-response has to rebuild the response.