Chainguard Container for spark-fips

Spark provides high-level APIs in Scala, Java, Python, and R, and an optimized engine that supports general computation graphs for data analysis.

Chainguard Containers are regularly-updated, secure-by-default container images.

Download this Container Image

For those with access, this container image is available on cgr.dev:

docker pull cgr.dev/ORGANIZATION/spark-fips:latest

Be sure to replace the ORGANIZATION placeholder with the name used for your organization's private repository within the Chainguard Registry.

Compatibility Notes

While Chainguard's Spark FIPS image is comparable to the official Spark image on Docker Hub, Chainguard's image is also FIPS 140-3 compliant. Chainguard's image includes only the minimum set of dependencies needed to run Spark.

FIPS Support

This image contains Bouncy Castle crypto libraries for FIPS.

The FIPS certified version of Bouncy Castle (CMVP [#4743]) is compliant with the FIPS 140-3 standard when used in accordance with the [Bouncy Castle Security Policy].

This image also ships with a validated redistribution of the OpenSSL's FIPS provider module. For more on FIPS support in Chainguard Images, consult the guide on FIPS-enabled Chainguard Images on Chainguard Academy.

Getting Started

Creating a KeyStore

Before getting up and running with Spark FIPS, you'll need to create a BCFKS KeyStore:

docker run -v $(pwd):/tmp/keystore --entrypoint keytool cgr.dev/ORGANIZATION/spark-fips:TAG \
  -v -keystore /tmp/keystore/keystore.bcfks \
  -storetype bcfks \
  -providername BCFIPS \
  -alias "localhost" \
  -genkeypair -sigalg SHA512withRSA -keyalg RSA \
  -dname CN="localhost" \
  -storepass "<YOUR TLS KEYSTORE PASSWORD>" \
  -keypass "<YOUR TLS KEY PASSWORD, can be the same>"

You can now use keytool to view the KeyStore:

docker run -v $(pwd):/tmp/keystore --entrypoint keytool cgr.dev/ORGANIZATION/spark-fips:TAG \
  -v -keystore /tmp/keystore/keystore.bcfks \
  -list \
  -storepass "<YOUR TLS KEYSTORE PASSWORD>"

After the KeyStore has been generated, Spark will need to be configured to use it. As an example, you may set the properties below:

spark.ssl.keyStorePassword=<YOUR TLS KEYSTORE PASSWORD>
spark.ssl.keyStoreType=BCFKS
spark.ssl.keyStore=/usr/lib/spark/conf/keystore.bcfks

Please consult the official documentation for configuring Spark for guidance on what properties are best for your environment.

Creating a TrustStore

To create a TrustStore and import and trust an existing CA certificate you can also use keytool:

docker run -v $(pwd):/tmp/keystore --entrypoint keytool cgr.dev/ORGANIZATION/spark-fips:TAG \
  -v -keystore /tmp/keystore/truststore.bckfs \
  -storetype bcfks \
  -providername BCFIPS \
  -import -file /tmp/keystore/MyCA.crt \
  -storepass "<YOUR TRUSTSTORE PASSWORD>" \
  -trustcacerts \
  -noprompt

Configuring Spark for use with your TrustStore can be done in your config file as with the KeyStore above. For example, the properties below may be set:

spark.ssl.trustStorePassword=<YOUR TRUSTSTORE PASSWORD>
spark.ssl.trustStoreType=BCFKS
spark.ssl.trustStore=/usr/lib/spark/conf/truststore.bcfks

If you'd like to use a TrustStore type other than BCFKS, you'll need to set the environment variables below when running Spark:

docker run \
  -e JAVA_TRUSTSTORE_OPTIONS="-Djavax.net.ssl.trustStoreType=<TRUSTSTORE TYPE>" \
  -e JDK_JAVA_OPTIONS="--add-exports=java.base/sun.security.internal.spec=ALL-UNNAMED --add-exports=java.base/sun.security.provider=ALL-UNNAMED -Djavax.net.ssl.trustStoreType=<TRUSTSTORE TYPE>" \
  cgr.dev/ORGANIZATION/spark-fips:TAG \
  ...

Deploying Spark

Start by creating a network for your Spark nodes:

docker network create spark

Then we can deploy Spark FIPS with Docker:

  docker run -it \
    --net spark \
    -v /path/to/spark-default.conf:/usr/lib/spark/conf/spark-default.conf \
    -v /path/to/keystore.bcfks:/keystore.bcfks:/usr/lib/spark/conf/keystore.bcfks \
    -v /path/to/truststore.bcfks:/usr/lib/spark/conf/truststore.bcfks \
    cgr.dev/ORGANIZATION/spark-fips:TAG \
    spark-shell

This will throw you into Spark's interactive shell.

We can submit a Spark application using another node via spark://localhost:4440:

  docker run \
    --net spark \
    -v /path/to/<YOUR APPLICATION>:/usr/lib/spark/work-dir/<YOUR APPLICATION>
    -v /path/to/spark-default.conf:/usr/lib/spark-default.conf \
    -v /path/to/keystore.bcfks:/usr/lib/spark/conf/keystore.bcfks \
    -v /path/to/truststore.bcfks:/usr/lib/spark/conf/truststore.bcfks \
    cgr.dev/ORGANIZATION/spark-fips:TAG \
    spark-submit <YOUR APPLICATION> --master spark://localhost:4440

You should now be up and running with Spark FIPS!

Documentation and Resources

• Configuring Spark
• Spark Documentation
• Spark GitHub Repository
• Spark Quick Start Tutorial

What are Chainguard Containers?

Chainguard's free tier of Starter container images are built with Wolfi, our minimal Linux undistro.

All other Chainguard Containers are built with Chainguard OS, Chainguard's minimal Linux operating system designed to produce container images that meet the requirements of a more secure software supply chain.

The main features of Chainguard Containers include:

• Minimal design, without unnecessary software bloat
• Daily builds to ensure container images are up-to-date with available security patches
• High quality build-time SBOMs attesting to the provenance of all artifacts within the image
• Verifiable signatures provided by Sigstore
• Reproducible builds with Cosign and apko (read more about reproducibility)

For cases where you need container images with shells and package managers to build or debug, most Chainguard Containers come paired with a development, or -dev, variant.

In all other cases, including Chainguard Containers tagged as :latest or with a specific version number, the container images include only an open-source application and its runtime dependencies. These minimal container images typically do not contain a shell or package manager.

Although the -dev container image variants have similar security features as their more minimal versions, they include additional software that is typically not necessary in production environments. We recommend using multi-stage builds to copy artifacts from the -dev variant into a more minimal production image.

Need additional packages?

To improve security, Chainguard Containers include only essential dependencies. Need more packages? Chainguard customers can use Custom Assembly to add packages, either through the Console, chainctl, or API.

To use Custom Assembly in the Chainguard Console: navigate to the image you'd like to customize in your Organization's list of images, and click on the Customize image button at the top of the page.

Learn More

Refer to our Chainguard Containers documentation on Chainguard Academy. Chainguard also offers VMs and Libraries — contact us for access.

Trademarks

This software listing is packaged by Chainguard. The trademarks set forth in this offering are owned by their respective companies, and use of them does not imply any affiliation, sponsorship, or endorsement by such companies.