Build Dynamic AI Routes with Cloudflare AI Gateway

All right, here's a very super common SAS use case. You got a free plan and a pro flam. Free users get a tiny budget, slower limits, and a cheaper model, whereas prousers get more budget, higher limits, and a much more premium model. Uh, sounds simple, right? But this is honestly a week's worth of effort. Imagine building this in a traditional way. Uh, you need a routing service that knows every plan, every model, every role. Maybe you're already a setup for per user rate limits. And you also need to implement caching for repeated requests. There's token counting pricing math for each provider. And if you're doing this, you also need to run background jobs to keep the prices and metadata up to date. Uh and there's like a lot more things involved uh like retries, fallbacks, health checks too, and basically a lot of things for just one AI response. And we could obviously do is spend a lot of time writing boiler plate or we could just use Cloudflare's AI gateway. We have something called a dynamic route which does all of this in just 5 minutes. Let me show you how this is AI gateways dynamic routing. You literally drag and drop blocks to build this whole flow. Just head over to the Cloudflare dashboard, click on AI gateway, and once you create a new one, you can just head over to the dynamic routing tab, and then just add a new route. Let's give this a name, my trillion dollar SAS. And once you hit create, you'll be presented with this flow builder and add an if condition. Click on this X. Then you can click on this little circle button. Then I want to add an if condition and split traffic. Let's do if else. Once you click that, click on this block. And then you can actually add conditions. Uh let's say I want to do if plan equals free. You can actually choose from a lot of operators too. If you want something greater than, lesser than, lesser than, equal to, etc. I want to stick with the free plan. Then I want to split traffic accordingly. Then let's do if this is true, I want to add some rate limits and head over to limit. Then I can do a rate limit pool. Click on the block again. Then what I do is basically add an identifier which will help me rate limit per user. Let's say I want to do email and I want to do 10 requests per day. Then let's do another limit. Let's add some budgeting. I don't want my free users to use more than $10 a day. Let's do that again on the email identifier. Let's do $10, which is a lot actually. And I also want my free users to get a lower quality model. So, let's just head over here again, click on the little circle button, then click on add model, click on the block, and you can see a lot of providers um which Cloudflare's AI gateway gives you. Uh let's do GPT 4.0 O mini 4 o mini. Uh once you right type it, click on the plus button and then there where there we have a free plans conditions and limits and the model we want them. Uh let's also do if the plan is something other than free. Okay, let's do this. Okay, let's add some higher limits to them. Let's say I want them to have a,000 requests per day. And we'll also use the email as the identifier. Let's also add some budgets. Again, let's click on the circle limit budget. Let's give them a $100 setting. Again, a lot. Then if all of these conditions pass, let's give them a much premium model. Click on add model. Let's do provider token AI. And let's say I want to give them chat GPT 5.1 latest. And yeah, there we have it. We have now created a dynamic route which would have taken at least a week or at least a couple of days to build. So there you have it. Just hit this endpoint and all of these conditions will just work flawlessly. To showcase the dynamic route we just built in realtime use, I actually built this little simulator which is a VIP coding website builder which shows you the difference paid users versus a free free users get uh with just the URL we just built. So let's say we want to run this uh prompt uh create a website for software developer and it's the same prompt uh and I'm actually sending the metadata which I just added in the dynamic route. So there's the email which is identifier and you can see the requests it has uh for free users and the request for the paid ones. So let's just hit this. And while this runs, let me show you the And you can see that it's just one route. None of the primitives or the utilities I was talking about is in here. It's just walking through this metadata that I have passed. And then I just hit the completion endpoint from OpenAI. And let's see. Okay, there we have our free users website. And looks like the paid one is still running. And there we have it. It took nearly like 3 minutes, but yeah, it's finally ready. And you can see the difference the paid user got versus the free user which is because we used a much premium model. So yeah, this is the exactly the use case or the problem we were trying to solve and you can see how cloud AI gateway can help you build this. Thank you.