A high-throughput and memory-efficient inference and serving engine for LLMs
其他人工智能更新于 2026-11-02
其他人工智能Apache-2.0
Repository dossier
vllm-project/vllm-omni
A framework for efficient model inference with omni-modality models
Brief / README 摘要
查看原文Easy, fast, and cheap omni-modality model serving for everyone [2026/10] We released 0.32.0, aligned with vLLM 0.32.0, featuring optional Ray-based multi-node inference, session-aware full-duplex serving, server VAD for turn-based realtime requests, and expanded image, music, and speech generation with YuE2-3B, Qwen-Image-2.1, Kandinsky6, MOSS-Image, and TaoMate-H3. See the release notes for compatibility changes and installation instructions. [2026/10] We released the vLLM-Omni Technical Report, describing a unified serving runtime for text, audio, image, video, and action generation. [2026/0…
Topics / 标签
01
浏览全部相关推荐
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
其他人工智能更新于 2026-11-02
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface. The fastest local inference engine in the world.
其他人工智能更新于 2026-11-02
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
其他人工智能更新于 2026-10-30
Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows.
其他人工智能更新于 2026-10-31