Skip to content

How secure is your build server - Cloud Native Security Day Edition

talks 2 min read

The journey from laptop to production artifact is paved with trust decisions we rarely examine. A new developer gets a laptop and immediately trusts the operating system, the hardware, and the internet. Installing tools with curl-pipe-bash is still common practice, even though it should not be. At minimum, verify checksums from multiple sources, use intermediaries like app stores that have their own verification chain, and keep your crypto libraries and ciphers up to date.

Dependencies are where things get interesting. We evaluate libraries through the lens of trust – competence, reliability, sincerity, and care – but these are fundamentally assessments of the people behind the code. Typosquatting attacks exploit our carelessness with package names. Post-install scripts can execute anything. Even download links in package.json can use HTTP instead of HTTPS. Tools like Snyk check for known vulnerabilities, but this applies to all code: frontend, infrastructure-as-code, Helm charts, configuration files. Data-as-code matters too – checking for passwords in config files.

Pushing code to version control introduces another trust layer. Is your Git binary secure? Are you verifying the SSH fingerprints of the server you are pushing to? Signing commits helps others verify provenance. Multi-signing with multiple people prevents a single compromised laptop from poisoning the chain. On the CI side, SaaS build systems introduce their own trust questions – disk scrubbing, container isolation, audit visibility. The Volkswagen project (which makes CI tests pass regardless of actual results) is a reminder that build systems can lie.

Reproducible builds are the gold standard but are deceptively hard. Time stamps, compiler flags, platform differences – Debian has invested enormous effort and is still not fully there. Bazel takes a different approach by specifying the entire toolchain as dependencies, enabling hermetic builds. The Bitcoin core developers use multi-build consensus: multiple people compile on different systems and compare checksums. The Update Framework (TUF) provides principles for limiting authority, requiring consensus, fast key revocation, and accountability.

All this verification has a cost. At some point you have to go with real trust because you cannot keep up. The vulnerability budget concept – tracking how many patches you are behind within defined bounds – provides a practical framework. John Allspaw’s question remains: how long can your system survive without human intervention? Building a secure server is no different from trusting a person. Transparency, communication, and care are as important as the technical controls.

Watch on YouTube – available on the jedi4ever channel

This summary was generated using AI based on the auto-generated transcript.

Navigate with