Chainguard Container for airflow-core

Apache Airflow offers a platform to author, schedule, and monitor workflows programmatically. This image is a minimal, slimmed-down version of the official Apache Airflow with only core components.

Chainguard Containers are regularly-updated, secure-by-default container images.

Download this Container Image

For those with access, this container image is available on cgr.dev:

docker pull cgr.dev/ORGANIZATION/airflow-core:latest

Be sure to replace the ORGANIZATION placeholder with the name used for your organization's private repository within the Chainguard Registry.

Compatibility Notes

Chainguard's airflow-core image provides minimal set of dependencies and core components of Airflow, intended for use as a base image for other images. Scripts for installing databases have been excluded as they depend on aptitude. Databases provided in Wolfi can still be installed via apk on -dev tagged images by use of a Dockerfile.

Getting Started

This image is optimized for size and only contains the core Airflow components. It provides a minimal set of dependencies and is intended for use as a base image for other images.

There are a few known issues using Airflow core as a drop-in replacement within the Airflow Helm chart.

While the official documentation does not provide a clear way to deploy the image, here's an example using a Kubernetes manifest:

cat <<EOF > airflow-deployment.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: airflow
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: airflow-dags-pv
  namespace: airflow
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "./dags"  # Ensure this path exists on your host
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: airflow-dags-pvc
  namespace: airflow
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: airflow-webserver
  namespace: airflow
spec:
  replicas: 1
  selector:
    matchLabels:
      component: webserver
  template:
    metadata:
      labels:
        component: webserver
    spec:
      volumes:
        - name: airflow-dags
          persistentVolumeClaim:
            claimName: airflow-dags-pvc
        - name: airflow-db
          emptyDir: {}
      initContainers:
        - name: init-db
          image: cgr.dev/ORGANIZATION/airflow-core:latest
          command: ["bash", "-c", "airflow db init || airflow upgradedb"]
          env:
            - name: AIRFLOW__CORE__EXECUTOR
              value: SequentialExecutor
            - name: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN
              value: sqlite:////airflow-db/airflow.db
          volumeMounts:
            - name: airflow-db
              mountPath: /airflow-db
      containers:
        - name: webserver
          image: cgr.dev/ORGANIZATION/airflow-core:latest
          args: ["webserver"]
          env:
            - name: AIRFLOW__CORE__EXECUTOR
              value: SequentialExecutor
            - name: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN
              value: sqlite:////airflow-db/airflow.db
          ports:
            - containerPort: 8080
          volumeMounts:
            - name: airflow-dags
              mountPath: /opt/airflow/dags
            - name: airflow-db
              mountPath: /airflow-db
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: airflow-scheduler
  namespace: airflow
spec:
  replicas: 1
  selector:
    matchLabels:
      component: scheduler
  template:
    metadata:
      labels:
        component: scheduler
    spec:
      volumes:
        - name: airflow-dags
          persistentVolumeClaim:
            claimName: airflow-dags-pvc
        - name: airflow-db
          emptyDir: {}
      initContainers:
        - name: init-db
          image: cgr.dev/ORGANIZATION/airflow-core:latest
          command: ["bash", "-c", "airflow db init || airflow upgradedb"]
          env:
            - name: AIRFLOW__CORE__EXECUTOR
              value: SequentialExecutor
            - name: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN
              value: sqlite:////airflow-db/airflow.db
          volumeMounts:
            - name: airflow-db
              mountPath: /airflow-db
      containers:
        - name: scheduler
          image: cgr.dev/ORGANIZATION/airflow-core:latest
          args: ["scheduler"]
          env:
            - name: AIRFLOW__CORE__EXECUTOR
              value: SequentialExecutor
            - name: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN
              value: sqlite:////airflow-db/airflow.db
          volumeMounts:
            - name: airflow-dags
              mountPath: /opt/airflow/dags
            - name: airflow-db
              mountPath: /airflow-db
---
apiVersion: v1
kind: Service
metadata:
  name: airflow-webserver
  namespace: airflow
spec:
  selector:
    component: webserver
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
EOF

After deploying the image, we can experiment with it further by creating DAG files and copying them to Airflow Webserver/Worker Pods.

Documentation and Resources

You can learn more about Airflow via the official documentation.

What are Chainguard Containers?

Chainguard Containers are minimal container images that are secure by default.

In many cases, the Chainguard Containers tagged as :latest contain only an open-source application and its runtime dependencies. These minimal container images typically do not contain a shell or package manager. Chainguard Containers are built with Wolfi, our Linux undistro designed to produce container images that meet the requirements of a more secure software supply chain.

The main features of Chainguard Containers include:

• Minimal design, without unnecessary software bloat
• Daily builds to ensure container images are up-to-date with available security patches
• High quality build-time SBOMs attesting to the provenance of all artifacts within the image
• Verifiable signatures provided by Sigstore
• Reproducible builds with Cosign and apko (read more about reproducibility)

For cases where you need container images with shells and package managers to build or debug, most Chainguard Containers come paired with a -dev variant.

Although the -dev container image variants have similar security features as their more minimal versions, they feature additional software that is typically not necessary in production environments. We recommend using multi-stage builds to leverage the -dev variants, copying application artifacts into a final minimal container that offers a reduced attack surface that won’t allow package installations or logins.

Learn More

To better understand how to work with Chainguard Containers, please visit Chainguard Academy and Chainguard Courses.

In addition to Containers, Chainguard offers VMs and Libraries. Contact Chainguard to access additional products.

Trademarks

This software listing is packaged by Chainguard. The trademarks set forth in this offering are owned by their respective companies, and use of them does not imply any affiliation, sponsorship, or endorsement by such companies.