AIFIRST Pushing the speed-cost frontier for Qwen3-TTS: sub-50 ms time to first audio
The brief
Nari Labs describes five inference optimizations applied to Qwen3-TTS on a single NVIDIA H100 SXM, reaching 10 requests per second and sub-50 ms p95 time to first audio.
Key points
- Throughput is about 630 characters per second at full load, at roughly $2 per 1 million characters.
- The post benchmarks the result against vLLM-Omni, SGLang-Omni, and VoxServe, and against the commercial services ElevenLabs and Cartesia, claiming lower latency at higher request volumes.
Sources
- HNnari-labs.com