Search. Pull. Build.

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

The Triton backend for the PyTorch TorchScript models.

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

The Triton Inference Server provides an optimized cloud and edge inferencing solution with vllm backend

vLLM-Omni extends vLLM to serve omni-modality models, covering text, image, video, and audio, through an OpenAI-compatible API with GPU acceleration.

vLLM is a high-throughput and memory-efficient inference engine for Large Language Models (LLMs). It provides an OpenAI-compatible API server for production LLM deployments with GPU acceleration.