/
DirectorySecurity AdvisoriesPricing
Sign inRequest a trial
Directory
tritonserver-vllm-backend-fips logoFIPS

tritonserver-vllm-backend-fips

Last changed

Request a free trial

Contact our team to test out this image for free. Please also indicate any other images you would like to evaluate.

Request trial
Tags
Overview
Comparison
Provenance
Specifications
SBOM
Vulnerabilities
Advisories

Chainguard Container for tritonserver-vllm-backend-fips

The Triton Inference Server provides an optimized cloud and edge inferencing solution with vllm backend

Chainguard Containers are regularly-updated, secure-by-default container images.

Download this Container Image

For those with access, this container image is available on cgr.dev:

docker pull cgr.dev/ORGANIZATION/tritonserver-vllm-backend-fips:latest

Be sure to replace the ORGANIZATION placeholder with the name used for your organization's private repository within the Chainguard Registry.

Getting started

The Triton Server with vLLM backend container provides GPU-accelerated large language model inference through NVIDIA's Triton Inference Server with the vLLM backend for optimized performance.

Basic vLLM Model Serving

Set up a simple vLLM model repository and serve the facebook/opt-125m model:

# Create model repository structure
mkdir -p model_repository/vllm_model/1

# Download model configuration
wget -P model_repository/vllm_model/1 https://raw.githubusercontent.com/triton-inference-server/vllm_backend/main/samples/model_repository/vllm_model/1/model.json
wget -P model_repository/vllm_model https://raw.githubusercontent.com/triton-inference-server/vllm_backend/main/samples/model_repository/vllm_model/config.pbtxt

# Start Triton server with vLLM backend
docker run --gpus all -d \
  --name triton-vllm \
  -p 8000:8000 -p 8001:8001 -p 8002:8002 \
  --shm-size=1G --ulimit memlock=-1 --ulimit stack=67108864 \
  -v $(pwd):/workspace -w /workspace \
  cgr.dev/ORGANIZATION/tritonserver-vllm-backend-fips:latest \
  --model-repository ./model_repository

The server will take 2-5 minutes to initialize as it downloads and loads the model.

Health Check and Model Status

Check if the server and model are ready:

# Check server health
curl http://localhost:8000/v2/health/ready

# Check model status
curl http://localhost:8000/v2/models/vllm_model/ready

# Get model metadata
curl http://localhost:8000/v2/models/vllm_model

gRPC Client Inference

Test text generation using the gRPC interface:

# Download sample client and prompts
wget https://raw.githubusercontent.com/triton-inference-server/vllm_backend/main/samples/client.py
wget https://raw.githubusercontent.com/triton-inference-server/vllm_backend/main/samples/prompts.txt

# Run client (requires tritonclient[grpc])
docker run --rm --net=host \
  -v $(pwd):/workspace -w /workspace \
  --entrypoint python3 \
  python:3.12-slim \
  -c "pip install tritonclient[grpc] && python3 client.py -u localhost:8001"

Refer to the vLLM documentation for detailed configuration options.

What are Chainguard Containers?

Chainguard Containers are minimal container images that are secure by default.

In many cases, the Chainguard Containers tagged as :latest contain only an open-source application and its runtime dependencies. These minimal container images typically do not contain a shell or package manager. Chainguard Containers are built with Wolfi, our Linux undistro designed to produce container images that meet the requirements of a more secure software supply chain.

The main features of Chainguard Containers include:

For cases where you need container images with shells and package managers to build or debug, most Chainguard Containers come paired with a -dev variant.

Although the -dev container image variants have similar security features as their more minimal versions, they feature additional software that is typically not necessary in production environments. We recommend using multi-stage builds to leverage the -dev variants, copying application artifacts into a final minimal container that offers a reduced attack surface that won’t allow package installations or logins.

Learn More

To better understand how to work with Chainguard Containers, please visit Chainguard Academy and Chainguard Courses.

In addition to Containers, Chainguard offers VMs and Libraries. Contact Chainguard to access additional products.

Trademarks

This software listing is packaged by Chainguard. The trademarks set forth in this offering are owned by their respective companies, and use of them does not imply any affiliation, sponsorship, or endorsement by such companies.

Licenses

Chainguard container images contain software packages that are direct or transitive dependencies. The following licenses were found in the "latest" tag of this image:

  • Apache-2.0

  • BSD-1-Clause

  • BSD-2-Clause

  • BSD-3-Clause

  • BSD-4-Clause-UC

  • CC-BY-4.0

  • CC-PDDC

For a complete list of licenses, please refer to this Image's SBOM.

Software license agreement

Compliance

This is a FIPS validated image for FedRAMP compliance.

This image is STIG hardened and scanned against the DISA General Purpose Operating System SRG with reports available.

Learn more about STIGsGet started with STIGs

Related images
tritonserver-vllm-backend logo
tritonserver-vllm-backend

Category
featured
FIPS
STIG
AI

Safe Source for Open Source™
Contact us
© 2025 Chainguard. All Rights Reserved.
Private PolicyTerms of Use

Products

Chainguard ContainersChainguard LibrariesChainguard VMs