Use the Agents SDK to listen to concert posters

absolutely love when you have a problem like personally and you can solve it for yourself with code. Like I I think that this happens to be one of those killer ideas and if you want it, it's yours. Just tell them that they should like and subscribe to this educational developer channel. So here's the thing. I love seeing live music, but there's so many bands that are coming through town, especially here in Portland where I live. There's so many posters and I started just taking photos of them and then I'd open my photo apps when I got home, right? and I' I'd go find the bands and I'd listen to them when I remembered, but oftentimes I don't and I don't end up seeing music. So, I wanted to change that. In episode two of AI Avenue, we dive deep into computer vision and I realized how good these state-of-the-art models are getting at OCR or optical character recognition. So, I was like, could it read a poster? And it can. Check this out. So, uh, the app's called Band-Aid, uh, because it's aiding you with bands, and it's a dad joke. Uh, so, so I've been testing with some stuff. I wanted to just show you. It's a work in progress because there are some problems, uh, and you might not be thinking about them, and I I actually did because sometimes you'll take a a a picture and the picture will be sideways. Uh, it can still read it. Actually, it's actually really good at reading it, but you want it to rotate. So, I I will I've been experimenting with some stuff, and I'll show you here uh what I mean by that, but I've been trying to like can can I use OCR on the text to rotate it, and it turns out it's not the greatest. And maybe we'll see. Uh and it's it's definitely non-deterministic. Like, that's something to to think about here. So, it takes the poster, it reads the poster, and because there's different bands on the poster, I should be able to find those bands on Spotify. And guess what? You can't. Oh, so here it is. So, so this came across, right? And like what I've been finding too is like I wish I was at this concert. I wish I was at this Nirvana Mudhoney concert. That would have been amazing. So if I click into here, you'll see that it's going it's found that it happened in Portland, uh where I'm at here in 1992, uh which uh would have would have been fun to be at this concert. Uh and it pulled the date and you'll see that it just says Saturday, August 22nd. And it did that because it pulled the date. It knew that that happened in 1992, right? Uh, and it didn't it didn't know because of Nirvana. It didn't look anything up. And I tried to rotate it, but you'll see that it's upside down. So, you might have to turn your head like that to see, but but uh it it read this. It read this and it it found out who was playing. Now, I was able to go and I used browser rendering to go and research the band. So, now I can learn more about the band. But what I really love is uh I used uh my I I made it so that I could log myself into Spotify, right? This is for me. So, I've logged in uh with my Spotify account and I can go here and I can press this button, right? And I can press this button and it's going to go and it's going to create a Spotify playlist. And it's really actually pretty quick. And there it just did it. So, here's the Portland 92. And you'll see that my my my photo is upside down. And it takes the top songs from it, right? It takes the the top songs of what what uh those bands were that were there, right? These are this is that concert. So, I can feel what that concert was like just from a concert poster. And I did it. I love this. I love this. And if it doesn't find people, most most people have the thing on Spotify is what I'm finding. So, I started thinking about it a little bit more, right? Like, oh my gosh, I have this whole agents framework. I have all this stuff. Are you ready to see like I have never built such a big uh and I'll be honest, I went a little bit overboard. I don't know if you've seen this before, but if you go to your your worker uh page, so this is my framework, right? So, I'm I have my metrics page when I come in. There's this bindings tab. And if you haven't seen this bindings tab before, it's amazing. But look at this. I'm having a hard time fitting them all on the screen here. Uh uh so I I used Workers AI. So I used Workers AI to summarize what came back from um uh Spotify, right? I wanted to talk about those artists and I wanted to summarize. I wanted to get very specific about like what kind of live performance do they put because I I appreciate that when I go see a show. So that's again, this is my app and I wanted it for me. Uh, and so, uh, I I used browser rendering to do that. I went I I used browser rendering. I I went and, uh, hit Spotify in a in a sane way. I went and opened up the page after I I found them. There's a summary that you could get to on the page, and I use browser rendering to get that. The image normalizer is a thing that I tried. I tried to do this. I try I'm trying to get that rotate, right? And I I don't know if anybody's there yet. Um, if anybody knows about that model, if you could let me know in the in the comments. I want to take a a a picture and say like, is this rotated the right way? Because the images uh library that we have, if you haven't played with this before, this is awesome. Uh I should do a whole video on this itself. You can take the image and you can shrink it, right? because I needed to shrink it for my Spotify thing and I also needed to rotate it in case the poster was off because it turns out that you you end up or posters are sideways when you take the picture and you kind of want to get it straight and like how do I how do I do that uh with AI was something where I was I was just playing around seeing how that works. Uh but what's really nice about this workflow is I can pass the agent into it and uh inside that workflow I can access the agent I can call the RPC methods on it. I'll I'll show you that here in a second. Um, I was also playing with this idea of like what if I had these agents that like had an orchestrator. So like I I orchestrate uh across because I wanted to make a poster and if I wanted people to share that poster, I wanted Spotify users to know and subscribe to the poster. So So what I kind of ended up doing a lot of agents. So I made a there's a poster itself as an agent. Uh and we'll talk about why that is. And then there's a Spotify user who's also an agent because I wanted to track it uh users that listen to the poster and I wanted to track in there. Now, if you haven't seen the agents SDK before, it has its own little SQL light database in there. So, I'm I'm making use of that uh and and it's expanding outwards. And again, this is for me and some of my friends, like if I wanted to let them into this this app. This isn't for uh uh uh consumption, but it could it could scale. And I'm feeling that as I build with it. And I love this. I love this so much. Uh I'm using R2 bucket for the posters, right? So, so, uh, that's the object storage, right? So, if I upload, um, I I do an upload, right? So, first of all, there's an uploads thing, and if anything that goes into that folder, I have a little queue there, and it goes and it transforms it and gets it into the right size. Uh, because you could do that when you you have an upload, right? So, I put an upload in there. Um, yeah. And so, the the the poster itself is pulling uh the information off. I'm doing the one uh query here in the poster agent. I'm I'm I'm doing that. And let's take a look. Let's take a look at that. Enough of this talking about these bindings. Let's just take a look at some of this code real quick. Direct, right? So, so this is a ZOD object and and you describe in here what you want to have happen. So, this is the event schema, the name of the venue where the event's happening. That's why it was able to pull that, the name of the city where it's happening. Also being able to like logically pull that out. Um, the date and the time uh when it's happening in ISO 9601 format. So, you might have saw that I on my on my Band-Aid poster. It had the right format. It was able to take that and look at this. determine the year based on the day of the week and the date if it's not provided. I didn't even know you could do that and I just tried it and it worked. Uh, and I was so happy about that because now I know that it happened in 1992. Uh, right. So, uh, have all the concert dates not yet happened or is this in the past? Because you might have a tour poster where there's multiple dates, right? So, maybe all of them haven't passed yet. So, anyway, that was just a that's a little edge case there, but I was able to do it, which is super cool. Um, and I want the name of the tour if it exists. I want to have all the band names, right? And I I want to do a slug, a suggested URL safe slug for this event based on the headlining band location here. It does that. It can build the actual safe slug uh uh for that, which is I I I love that. And I use that for the name of the image and I use that for um uh the URL. And then if we go down to where we use it, we say extract information from this concert poster. This is the current date and I'm using GPT5. It came out. It's a lot better than what I was playing with. Uh, and I give it that I give it that schema, right? I passed that schema into it. And, uh, I say this is what I want to have come out. I want to have that that poster text format come. And it works. And we saw that. And then, uh, I have a bunch of, uh, things set up in here. I have these workflows, right? So, uh, let's look at the Spotify researcher, right? So, you summarize musical artists. I'm using our Llama 333 here uh, to get information out of this. I get a summarized description. And you know when when you come into one of these workflows, this is what I was talking about. When you come into the workflow, it has this run method and I can go get the agent, right? I can get the agent by name and then I can use that agent, right? Whenever I want to use that agent, I can call methods on it, right? So I can say poster agent get band names and for each of the band names, I'm going to go and find the Spotify artist. So this workflow goes and it finds all of the Spotify artists and if it fails or if I hit any sort of uh problem there, it will retry. So, I I'm almost certain that I'm going to go and get this. And so, it goes through, it does um all the top tracks and it's pulling all the stuff and then it goes and it tries to get it does the browser rendering stuff, right? So, so I'm able to get from the the API using the Spotify API and then I'm able to get a little bit more uh from the HTML on the actual page that's provided uh uh from that. So, if you haven't tried browser render before, that's a great place to go and try and there's there's probably newer ways to do this. I I have wrote this way a while ago. We have newer things on browser rendering now which I haven't gone to update it yet. Uh and maybe that's a good thing too. Is that something that you you want to you want to uh learn about? Anyway, super exciting. Uh the other thing that I really wanted to show off is that there's a schedule, right? So so we have the ability again these agents. This is the Spotify user agent. I can write uh SQL tables, right? So my Spotify user uh agent has has these tables and I can go and I can go and uh make sure that when somebody listens to it. So this is this is getting the the I can get the recent track URIs I can go on a schedule and say hey did you listen to the the songs and if so you should go attend this thing. So I want to I want to use the orchestrator to say hey this person's interested in this concert and I want to send that to them. I I want to be able to to use that information. I'm still playing with what this algorithm is because I don't get all of the tracks at once. And so I kind of have to choose when do I check is this person a heavy Spotify listener and I'm going to change the amount of time that the person listens to because I can only get back 50 of the most recent tracks. But if 50 of the most recent tracks are all week long, that's one thing. But if it's uh every hour if somebody's listened to a lot of songs, a lot of punk songs, I guess in that case to get 50 in an hour, uh they they would uh it changes that. So, so the agent kind of grows with you as a and it's specific to the user, which is nice, right? Because that's that's what this is. It's a personalized user agent there that is working with this this poster that multiple people could listen to. So, it's this really neat uh structure that I've really been enjoying. I'll share the code with you. It is a work in progress. Uh but I I wanted to get it out there. I wanted you to see this because I'm I'm super excited about I also wanted to inspire you to like go go build stuff, right? And so, like, what I plan to do is I I plan to make the agent get to know the Spotify user, which is me. And in this case, over time, start doing more proactive recommendations like suggesting that that concert's coming soon and I should listen to it because I haven't yet. And look, I know I say this all the time, but what a bonkers time it is to be a builder. I'll keep you posted on this project cuz see posted this band poster. But seriously, let me know if you'd like to learn more about what I did here. I really think that this is a product. If you've got the time, take it. Run with this. Thanks so much for hanging out and we'll see you real soon.