Build servers are one of the main targets at security conferences, and for good reason – they are a centralized authority holding credentials for everything. This talk traces every trust decision from a developer’s pristine laptop to the production artifact, focusing specifically on how things can be tampered with during the build process.
Starting from the laptop, development is increasingly about assembling libraries rather than writing code. The TLS chain we depend on has gaps: Node.js defaults to wide cipher suites, curl does not check certificate revocation, DNSSEC is not used by NPM or GitHub. The homebrew install command pipes Ruby scripts from the internet into your shell. Node.js at least provides GPG verification, but the process is manual and clunky. Docker Desktop could not even be checksum-verified outside the Apple App Store when I researched this.
Library trust goes beyond vulnerability scanning. We evaluate competence, reliability, sincerity, and care – the four dimensions from the Thin Book of Trust. Package managers execute post-install scripts that can do anything. NPM once showed advertising during install, a friendly reminder that these scripts have full system access. Proxy-level scanning through tools like JFrog X-Ray can catch vulnerabilities before packages reach your laptop.
CI systems present a particular challenge. The Volkswagen project – detecting when code runs on CI and making tests pass regardless – illustrates that build systems can lie. The Bitcoin core team addresses this through multi-build consensus: multiple people compiling on different systems, comparing checksums. Reproducible builds are extremely hard – even running Node twice on the same laptop produces different binaries. Debian has invested years of effort. Bazel’s hermetic builds specify the entire toolchain as dependencies. Distroless Docker images reduce containers to single binaries. In-toto records and signs every build step.
The extended Q&A surfaced important points: using short-lived keys that expire after each build instead of long-lived credentials on shared executors; the fallacy that things inside the firewall are secure; Jenkins plugins that can decrypt secrets from other teams; the Netflix paved-road approach versus security training checklists; and the critical need for security people to educate rather than police. We are at the point with security where CI/CD was years ago – it seems impossible now, but awareness is growing and practices will mature.
Watch on YouTube – available on the jedi4ever channel
This summary was generated using AI based on the auto-generated transcript.