​
DirectorySecurity Advisories
Sign In
Directory
kserve-storage-initializer logo

kserve-storage-initializer

Last changed

Sign In for Updates

Get notified of upcoming product changes, critical vulnerability notifications and patches and more.

Sign In
Versions
Overview
Provenance
Specifications
SBOM
Vulnerabilities
Advisories

Chainguard Image for kserve

Chainguard Images are regularly-updated, minimal container images with low-to-zero CVEs.

Download this Image

This image is available on cgr.dev:

docker pull cgr.dev/ORGANIZATION/kserve:latest

Be sure to replace the ORGANIZATION placeholder with the name used for your organization's private repository within the Chainguard registry.

Usage

KServe enables the scalable deployment of machine learning models in Kubernetes. It provides mechanisms for model versioning, monitoring, and autoscaling, and is built on top of Kubernetes' extensibility features, such as custom resource definitions (CRDs) and operators.

Components

This deployment includes the following critical KServe components:

  • Agent: Responsible for dynamically pulling models from storage (e.g., Google Cloud Storage or S3) and loading them into the model server.
  • KServe Controller: Manages the lifecycle of ML models and their associated services in Kubernetes.
  • ModelMesh Controller: Manages model serving infrastructure, including dynamic model loading and inference.
  • ModelMesh Runtime Adapter: Communicates between ModelMesh and models.
  • REST Proxy: Acts as a proxy layer for model serving, supporting multiple model types.
  • Router: Manages routing inference requests to the appropriate model version or model instance. It handles traffic distribution, enabling versioned model serving and A/B testing.
  • Storage Initializer: Downloads models from storage systems and prepares them for inference.

Installation

You can install KServe components using Helm charts. Below are instructions for deploying KServe with Chainguard images.

1. Install KServe CRDs

KServe relies on Custom Resource Definitions (CRDs) to manage model serving resources.

helm install kserve-crd oci://ghcr.io/kserve/charts/kserve-crd --version v0.13.1

2. Deploy KServe Components with Custom Images

To deploy KServe with Chainguard images, use the following Helm commands:

helm install kserve oci://ghcr.io/kserve/charts/kserve --version v0.13.1 \
  --set kserve.agent.image.repository=cgr.dev/chainguard/kserve-agent \
  --set kserve.agent.image.tag=latest
  --set kserve.router.image.repository=cgr.dev/chainguard/kserve-router \
  --set kserve.router.image.tag=latest
  --set kserve.controller.deploymentMode="RawDeployment" \
  --set kserve.controller.image.repository=cgr.dev/chainguard/kserve-controller \
  --set kserve.controller.image.tag=latest \
  --set kserve.modelmesh.enabled=true \
  --set kserve.modelmesh.controller.image.repository=cgr.dev/chainguard/kserve-modelmesh-controller \
  --set kserve.modelmesh.controller.image.tag=latest \
  --set kserve.modelmesh.config.restProxyImage=cgr.dev/chainguard/kserve-rest-proxy \
  --set kserve.modelmesh.config.restProxyImageTag=latest \
  --set kserve.storage.image.repository=cgr.dev/chainguard/kserve-storage-initializer \
  --set kserve.storage.image.tag=latest
  --set kserve.modelmesh.config.modelmeshImage=cgr.dev/chainguard/kserve-modelmesh \
  --set kserve.modelmesh.config.modelmeshImageTag=latest \
  --set kserve.modelmesh.config.modelmeshRuntimeAdapterImage=cgr.dev/chainguard/kserve-modelmesh-runtime-adapter \
  --set kserve.modelmesh.config.modelmeshRuntimeAdapterImageTag=latest \

For detailed installation configurations, refer to the KServe Helm chart values.yaml.

Inference Testing

Inference tests are critical to validating KServe's model serving capabilities. After deploying KServe, you can run inference tests to confirm that the deployed models are working as expected. For more info

1. Apply a Sample Inference Service

The following example creates an inference service that uses the iris-sklearn model:

kubectl apply -f /tests/inference-service.yaml

This service will download the model using the storage-initializer, and KServe will expose the model for inference requests.

2. Test Inference Using Port Forwarding

Once the inference service is running, you can send inference requests to validate the model's predictions:

kubectl port-forward -n kserve-test svc/iris-sklearn-predictor 8080:80 &

Use the following command to send a test prediction request:

curl -s -X POST http://localhost:8080/v1/models/iris-sklearn:predict \
  -H "Content-Type: application/json" \
  -d '{"instances": [[5.1, 3.5, 1.4, 0.2]]}'

You should receive a response like this:

{"predictions":[0]}

This confirms that the inference service is functioning correctly.

Contact Support

If you have a Zendesk account (typically set up for you by your Customer Success Manager) you can reach out to Chainguard's Customer Success team through our Zendesk portal.

What are Chainguard Images?

Chainguard Images are a collection of container images designed for security and minimalism.

Many Chainguard Images are distroless; they contain only an open-source application and its runtime dependencies. These images do not even contain a shell or package manager. Chainguard Images are built with Wolfi, our Linux undistro designed to produce container images that meet the requirements of a secure software supply chain.

The main features of Chainguard Images include:

-dev Variants

As mentioned previously, Chainguard’s distroless Images have no shell or package manager by default. This is great for security, but sometimes you need these things, especially in builder images. For those cases, most (but not all) Chainguard Images come paired with a -dev variant which does include a shell and package manager.

Although the -dev image variants have similar security features as their distroless versions, such as complete SBOMs and signatures, they feature additional software that is typically not necessary in production environments. The general recommendation is to use the -dev variants only to build the application and then copy all application artifacts into a distroless image, which will result in a final container image that has a minimal attack surface and won’t allow package installations or logins.

That being said, it’s worth noting that -dev variants of Chainguard Images are completely fine to run in production environments. After all, the -dev variants are still more secure than many popular container images based on fully-featured operating systems such as Debian and Ubuntu since they carry less software, follow a more frequent patch cadence, and offer attestations for what they include.

Learn More

To better understand how to work with Chainguard Images, we encourage you to visit Chainguard Academy, our documentation and education platform.

Licenses

Chainguard Images contain software packages that are direct or transitive dependencies. The following licenses were found in the "latest" version of this image:

  • Apache-2.0

  • GCC-exception-3.1

  • GPL-2.0-or-later

  • GPL-3.0-or-later

  • LGPL-2.1-or-later

  • MIT

  • MPL-2.0

For a complete list of licenses, please refer to this Image's SBOM.

Software license agreement

Category
AI
application

Safe Source for Open Sourceâ„¢
Media KitContact Us
© 2024 Chainguard. All Rights Reserved.
Private PolicyTerms of Use

Product

Chainguard Images