Lightboard series - Secure your AI-powered applications with Cloudflare

Hello. My name is Homer. I work at Cloudflare in technical marketing on everything application security and AI related. And in this session, I'm going to cover how you can secure your AI powered applications with Cloudflare. Let's go ahead and get started. So here on the left, you can see a client is making a request to a AI powered application. Now, this request is going to get routed to Cloudflare, which is acting as a proxy. This is where all our application performance and security services are running. Services like DNS, which by the way happens to be the fastest DNS in the world. CDN, load balancing, and other application performance services. This is also where we secure and protect your web applications and API endpoints with Cloudflare's application security services like WAF, rate limiting, API security with API Shield, bot management for malicious bots, client-side security with Page Shield. We want to be able to detect and mitigate for those malicious scripts, right? And now AI security with AI security for apps. And AI security for apps adds that AI specific context to WAF. So it's the same operational model you're already used to, but now you're creating these WAF security policies with this AI context to secure your AI applications. And you may be asking, okay, well, where do these these applications need to sit, right? Well, you can use Cloudflare's serverless development platform and deploy these applications on Cloudflare. You can also deploy them on third party. These could This could be like third-party cloud providers. These applications can even be on prem, right? So we're really agnostic to where the application sits. Now, what about the models that are powering these applications? Well, again, you can use Workers AI, which is LLMs deployed across all of Cloudflare's edge locations, so you can use Cloudflare here. You can also use third party again. These would be folks like OpenAI or Anthropic. And you can even use models deployed on prem. So at the end of the day, we want to be able to secure these applications and models regardless of where they sit. Now, with AI security for apps, there's really three main components. The first one being LLM discovery. So as that request comes in, using LLM specific heuristics and all the intelligence we have, we do LLM endpoint discovery. So here we're doing LLM discovery, right? Not only that, with all these endpoints that we identify, we go ahead and label them. We label them with the Cloudflare LLM label. Now, you may be asking yourself, well, what's the significance of this, right? Well, a few things here. First, it allows you to easily identify all the LLM endpoints, right? You can filter on this Cloudflare LLM label and jump right to the analytics for the specific LLM endpoints that you're interested in, right? The other thing is, it allows us to easily create and apply these security policies. So you don't even have to tell us where to apply a security policy that you're creating. We're automatically going to apply it to all the endpoints we've identified and labeled with the Cloudflare LLM label. Now, of course, we allow you to be more granular and specific if you want, so you could do something like, hey, only apply this policy to all the requests with a specific host name, right? But right out of the box, we make it very easy to secure your AI applications. Okay. The second component here is visibility into all LLM activity and risk. So as those requests comes in, right? As those requests come in, we're able to do prompt detection. And we can even extract the prompt. We can do prompt logging, which provides you even more details and visibility into exactly what's going on. And we take all this intelligence, right? All the request header information, the results of all the AI threat detections, and we give you deep analytics here, right? That you can view in the dashboards. We even bubble up all the AI specific uh suspicious activity in our security analytics. So we give you those alerts so you can jump right to the things that may be the most important. Now, the third component here is protect and mitigate, right? We want to detect and protect against AI specific threats. Now, a few use cases here. First one is PII exposure, We don't want our model learning off of PII, Which can later also result in PII exfiltration. So we want to block it right up front. Okay? This would be stuff like social security number, credit card number, and so on. We don't want our model model trained on that. Second, harmful content. So we definitely don't want our model to be trained on any harmful content, which can later also be used to to respond to user requests, right? We don't want our model returning guidance on something that could be harmful or illegal activity that we can even be liable for. So we also want to block that up front. And finally, prompt injection and jailbreak. prompt injection is where the user is trying to trick the model to get it to do something it wasn't intended to do. So we want to be able to detect and mitigate for that. Now, how all this works is, remember, AI security for apps is adding that AI specific context to WAF. So it's the same operational model where you're creating these WAF policies, but now with this AI context Now, as that request comes in, remember, we've already identified your LLM endpoints. So as traffic comes in to those LLMs or to those LLM endpoints, we can intercept that traffic, And we send it to an LLM. And we have specific LLMs for each of these threats. And this is all done in parallel. So as we add more detections and mitigations, right? And models, there's not going to be increased latency. It's all happening in parallel, okay? Now, for things like PII exposure, harmful content, obviously the model, if it detects that, we're going to you know, it's going to return that response and the user can take the appropriate action. Once it sees, hey, we've detected this is PII, this is harmful, the user with the WAF policy can set it as a light, right? Now, with PII exposure and harmful content, right? We also have categories. So you can be more granular. If you don't want to block on all PII or all harmful content, for example, with PII, you can maybe maybe you're interested in specific categories like email address, credit card number, social security number. So you can select the category uh and put that in the policy. Same with If you're interested in maybe specific categories, maybe violent crime, non-violent crime, uh proprietary or intellectual property, right? You can use those categories in your policies so you don't block on everything. Okay? So here we have categories. We can detect everything and set a rule around that, or we can set a policy around a specific category as well. Now, prompt injection is a little bit different. For for every request that comes in, right? To that LLM endpoint, we give it what's known as an injection score. Right? And if the injection score is less than 20, that signifies an attack. So, you can create a policy saying, "Hey, if injection score is less than 20, go ahead and do take this action, whether it is logging, blocking, and so on." Right? So, hopefully this gives you some idea of how you can use Cloudflare to secure your AI-powered applications. If you want more information, make sure you check out the AI security perhaps reference architecture.