How AI is applied across API Evangelist and APIs.io. Read my AI disclosure →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

Oxidizing Cross-Encoders

calendar_today September 25, 2026 person info@qdrant.tech (Andrey Vasnetsov) domain qdrant

We rewrote cross-encoder inference in Rust, benchmarked it against Python, and on a small reranking model it came out only 1.1x faster at the median. That is not a sign of a slow Rust implementation. The Python libraries we compared against, fastembed and sentence-transformers , hand the same work to the same C++ engine, onnxruntime , so all three spend almost all of their time in the same compiled code.

open_in_new Read original post