Introduction

In the world of modern software development and deployment, containerization has become a cornerstone technology. Containers offer efficiency, consistency, and scalability, allowing applications to run reliably across different computing environments. However, along with these benefits comes the responsibility of ensuring the security of containerized applications. One critical aspect of this is securing the base images from which containers are built.

What is Jenkins?

Before delving into the specifics of securing automated container-based images, let's first understand what Jenkins is. Jenkins is an open-source automation server that helps automate the non-human part of the software development process. It enables developers to automatically build, test, and deploy code changes, streamlining the development pipeline.
Read more about Jenkins here: https://www.jenkins.io/
(You are a smart engineer, you know what Jenkins is!! ... I suppose 😅)

Secure Automated Container Base Image Hardening

Definition and Importance

Container base image hardening refers to the process of enhancing the security of the base image used to create containers. It involves configuring the image to remove unnecessary components, reducing its attack surface, and applying security best practices. This step is crucial because vulnerabilities in the base image can propagate to every container instance created from it.

Techniques for Hardening Container Images

Several techniques can be employed to harden container images:

• Image Minimization: Remove unnecessary packages and files to reduce the attack surface.

• Patch Management: Ensure that base images are up to date with the latest security patches.

• Least Privilege: Run containers with the least privileges necessary to perform their tasks.

• Image Signing: Digitally sign images to verify their integrity and authenticity.

Architecture

The first part of the process is pretty straight forward. Here, we are taking input, the docker image, and doing a few things with it.

• We are identifying the underlying OS

• Understanding the initial state of the image (Pre-Scanning)

• Applying Hardening (building with right Dockerfile)

• Check the new state of the image (Post Scan)

• Signing the Image with Cosign

• Pusing the image to your repository.

The pipeline will look somewhat like this:

The second part is where things get a little interesting.
Before we jump in, there are few things we need to consider:

Is the Registry you are using:

• Have Vulnerability scanning capabilities?

• How can we set periodic scanning and fetch those results?

For example, Harbor has vulnerability scanning feature and you can fetch results with its easy to use API set, thanks to the comprehensive API doc, whereas Sonatype Nexus, it does not have inbuild vulnerability scanning feature and the APIs are a bit tricky to use.

You can write a wrapper using python that will do all the heavy lifting, from triggering the scans to fetching the results or you can leverage external paid softwares like Prisma Cloud to onboard your repo and scan your images or write script with python to fetch and pull all the images and scan them with you scanners and store results.

This entire automation is comprise of a group of small pipelines. Let’s discuss about them one by one.

Repo Scanner Pipeline

Repo Scanning pipeline as the name suggest will trigger the periodic scanning of our images in the repository if we have scanning functionality built-in the repo. Else, get the list of all the images, scan them one by one and store their json report in an external storage.

This pipeline will run via cronjob everyday.

Vulnerability watcher Pipeline

Vulnerability watcher pipeline will fetch those scan reports form storage and process the data in order to make a list of images with High/Critical vulnerability present with status as fixable. And, for each image from the Vulnerable_image_list, trigger the Image Hardening Pipeline.

Image Hardening Pipeline

The Image Hardening pipeline is the heart of this entire process. This is where Container image hardening flow will begin (refer the architecture above). Here we will identify the underlying OS, fetch the relevant Dockerfile from code repo, build an image, scan it again to make sure no High/Critical are present with status fixable, and finally tag the image, sign it and push it to the Repo.

Conclusion

This is a critical problem faced by many organisation, big or small, and this is just one way of approaching the problem. One more way to approach this problem is Distroless and might be the future of this entire race against vulnerabilities (Stay tuned for a technical blog around Distroless).

I will write another blog where we will jump deeper in the technicalities and how to implement each pipeline and make this entire project possible.

Untill then, keep reading!
It's me …
The Modern Witcher ⚔