Hailo-10H / HailoRT 5.1.1 — KV-cache state retention across repeated InferModel requests

Title: Hailo-10H / HailoRT 5.1.1 — KV-cache state retention across repeated InferModel requests Hello Hailo Community, We’re investigating numerical reproducibility when reusing a VDevice for sequential transformer inference on a Raspberry Pi 5 with a Hailo-10H accelerator. Our environment uses HailoRT and firmware 5.1.1, with a custom transformer HEF containing separate prefill and token-by-token inference groups and a 128-position KV cache. We’ve observed the following: Repeated identical requests using a shared VDevice can produce different numerical outputs, even when the input buffers, at