Search. Pull. Build.

The Triton Inference Server provides an optimized cloud and edge inferencing solution with vllm backend

vLLM is a high-throughput and memory-efficient inference engine for Large Language Models (LLMs). It provides an OpenAI-compatible API server for production LLM deployments with GPU acceleration.