Inability, Limited Knowledge, and Errors: Challenges with Reproducible Builds

Securing the container build process from malicious tampering is critical to protecting national and proprietary interests. Having reproducible container builds increases the trustworthiness of the build artifacts by providing consistent outputs and predictable behaviors.

However, there is limited awareness of container reproducible-build practices. Furthermore, even if software engineers are aware of reproducibility, builds themselves are rarely reproducible because elements of build environments often rely on external, nondeterministic factors (e.g., timestamps, filesystem file ordering, and unique ID generation).

Additionally, when builds are reproduced, failures often occur when (1) a reproducibility issue exists in a build environment (e.g., an unpinned package dependency) and (2) an external factor changes in a way that exercises that issue (e.g., an update to that package occurring between builds). The failure results in the creation of two container images that do not match (i.e., contain different files) from builds using the same container build environment (e.g., a specific Dockerfile).

A further limitation for the success of reproducible builds is that existing tools are limited in their ability to find and fix common issues. For example, Dockerfiles can be used to identify some reproducibility issues, such as nonexistent version pinning; however, they are not designed for reproducibility and so do not identify other issues (e.g., volatile input sources or timestamps).

Container images are increasingly being used as the main method for software deployment, so ensuring the reproducibility of container images, among others, is becoming a critical step in protecting the software supply chain as a key aspect of overall build reproducibility.

Kevin A. Pitstick

Software Engineering Institute