Pipeline Vulnerability Scanner

You are building a new cloud native application that is hosted in a container. You want to use container images that are obtained from existing public registries in order to take advantage of the work of others in the Docker community. However, you also want to ensure that container images with known vulnerabilities are not deployed to the image registry from the CI/CD Pipeline.

How do you prevent Docker image with known vulnerabilities from being uploaded into your image registry in the first place?

There are several issues that are brought up by the use of an image registry. First of all, you don’t want your Public Image Registry or Private Image Registry to contain vulnerable images; you want issues to be sorted out before the image is deployed to the registry. You don’t want to place the responsibility entirely on the developer; you want them on focusing on writing code. Likewise, you don’t want developers to be forced to go through a lengthy and arduous security review.

Therefore,

Ensure that your CI/CD pipeline has a vulnerability scanner included in one or more of the stages in your pipeline. The scanner will scan your container image for any known vulnerabilities and stop the deployment and report the issue if any issues are found. The static vulnerability scan should check public vulnerability databases such as CVE at a minimum.

The types of scanners that we are referring to are often referred to as static scanners in that what they do is examine the configuration of a Docker image in looking for known vulnerabilities to operating system, language runtime or middleware. This is different from an active scanner, which examines the activity of a running docker image for known malware behavior. An example of the overall build process (a potential model for a Docker build pipeline) including a pipeline vulnerability scan is shown in Figure 2: Docker Build Process.

Figure 2: Docker Build Process

Let’s look at Figure 2: Docker Build Process in detail. At the beginning of the process, you have a number of different input triggers that can result in the need to update an image – in all cases, what this amounts to is changing the dockerfile to introduce a configuration change, or updating the application code executing within the dockerfile that will be built as part of the CI/CID process. When you do so, this acts as a trigger to the CI/CD build pipeline – in which one step in the pipeline will be a vulnerability scan. That scan must complete successfully before the image can be published to a temporary registry in preparation for running dynamic scans (see Birthing Pool).

However, it’s not enough to simply include a vulnerability scan as part of a DevOps pipeline, you also need to alert your DevOps, Site Reliability Engineering or run teams of any vulnerability scans as and when they occur. Especially as vulnerability databases are constantly updated, it may be true that an image that has passed an earlier scan and already been deployed to production may fail a later scan in an earlier stage against an updated database.

You want your images to be scanned regularly; not just when code changes, in case a new vulnerability is discovered post-release. Therefore it is important to have triggers such as notifications from your security team of new vulnerabilities that can begin a new pipeline run. Likewise, you don’t want to purely dependent on the vulnerability scanner of your image registry.

Open-Source scanners such as Clair can provide such functionality and can be easily integrated with CI/CD pipeline tools such as Jenkins. There are multiple other examples of such tools, such as the open source Docker Bench Security and the commercial IBM Vulnerability Advisor.

Using Vulnerability Scanners is another well-established practice in the Docker development community, cited specifically as a best practice in 7 Threats and Sumologic.

Using a Pipeline Vulnerability Scanner integrated into your CI/CD pipeline increases the overall security of your solution by ensuring that your image is secured from known vulnerabilities and prevents vulnerable images being uploaded into your Public Image Registry or Private Image Registry in the first place.

You should also have a policy that prevents any unsecured images being deployed into production. Only allowing teams to only pull from an Approved Image Repository is one way of doing this.