Week 12 - AI Security & AI Red Teaming on AI Platform Engineering Handbook

AI Governance & Compliance

Mon, 01 Jan 0001 00:00:00 +0000

AI Governance & Compliance

The Big Picture First

Everything we’ve covered so far in Week 12 has been about technical security — attacks, defenses, testing. This final section shifts to a different but closely related layer: the organizational, ethical, and legal frameworks that sit above all that technical work. Here’s the useful distinction to hold onto throughout this whole explanation: security is about can something go wrong technically (can this system be hacked, manipulated, or misused), while governance and compliance are about should — is this system being built and used in a way that’s genuinely responsible, fair, and legally permitted in the first place. A perfectly secure AI system that’s used to unfairly discriminate against job applicants, or that quietly violates someone’s privacy rights, still represents a serious, genuine failure — just a different kind than a hacked one. This section is about making sure the “should” question gets just as much serious, deliberate attention as the “can” question does.

AI Red Teaming & Defensive Techniques

Mon, 01 Jan 0001 00:00:00 +0000

AI Red Teaming & Defensive Techniques

The Big Picture First

The previous section walked through the various ways an AI system can actually be attacked. This section flips that same picture around, covering how you actually go about deliberately finding those weaknesses yourself, before a genuine real attacker ever does, and how you actually go about building real, concrete defenses against the specific threats we already covered. There’s a genuinely useful mental model worth holding onto throughout this whole explanation: red teaming is fundamentally about finding the problems, while the various specific defensive techniques covered afterward are all fundamentally about fixing or actually preventing them. Together, these two halves form a genuinely continuous, ongoing cycle — you test, you find real weaknesses, you fix them, and then you genuinely test again, since new weaknesses can always keep emerging over time, whether through actual model updates, new features being added, or genuinely new attack techniques being actively discovered elsewhere out in the world. Let’s work through it all.

AI Security Fundamentals

Mon, 01 Jan 0001 00:00:00 +0000

AI Security Fundamentals

The Big Picture First

We’ve now spent ten weeks’ worth of material talking about how to build, operate, and govern AI systems well. This section shifts to thinking like an attacker for a moment — because to genuinely secure an AI system, you have to understand how someone might actually try to break it, manipulate it, or misuse it. AI security is a genuinely distinct field from traditional cybersecurity, even though it borrows heavily from it, precisely because language models introduce fundamentally new categories of vulnerability that simply didn’t exist in traditional software. A regular piece of software either has a bug in its code, or it doesn’t — the code is fixed, and its behavior is knowable in advance. A language model processes natural language instructions as its actual input, and it doesn’t have a clean, built-in way of reliably telling apart “a legitimate instruction from my actual developer” from “text that just happens to look like an instruction, sitting inside something a user or an external document said.” That single fact is genuinely the root of a huge number of the specific security problems we’re about to cover. Let’s build this up piece by piece.