A deep dive into hooks | Workflow SDK

Hey, thanks so much for joining us today. I'm Pranay from the workflow team here at Vercel. All right, I'm Nate, also from the workflow team. Nate, you and I have been on the workflow team from the very beginning and of all the things that we've shipped in the last 6 months, I think hooks and webhooks are one of my favorite features and that's exactly what we're here to talk about today. Hooks and webhooks are also my favorite feature. They're incredibly powerful and I'll show you a few demos to explain why. So, the first demo is something we're probably all familiar with, magic links. A magic link is a login form, you type in your email, receive an email in your inbox and when you click that link, you're logged into the service. Yeah, in fact, if I remember correctly, with Vercel, actually even before it was called Vercel, when it was still called Zeit, magic links were the only way that you authenticated and we kind of built this whole system ourselves at the time, too. That's right and I still have PTSD. Uh, because without workflow, implementing such a system is a lot more complicated than it seems like at face value. Uh, logic ends up being scattered across multiple files. You need to involve uh, a database to keep track of your state. Yeah. Um, and it it gets it gets messy quick. Yeah, I've already been thinking about sort of how I would structure this and and what database I'd use cuz this feels like a feels like a common problem that I've I've built stuff like this before. So, yeah, love to see what it looks like. Yeah, so just to to demonstrate what I'm talking about, uh, the the the pain points that I'm talking about, I started by implementing kind of a quote-unquote traditional without workflow version of a magic link login. Um, and so there's, you know, there's three endpoints involved. Uh, first one is when the login form is submitted and it needs to generate session and store that session in a database such as Redis, um you need to implement a TTL and let you know, can't let data be hanging around forever. You need to expire it. Um and then send the email. This thing could fail. Uh and then your login doesn't work and that's a frustrating experience. Right. And then you have to go back and like have a cron job or an or an intern clean up your database. I might have been the intern at the time. So then there's a second endpoint. This is what the one that happens and this is when the user clicks the the link in their email. Uh and this needs to basically query the database, restore the the state that was constructed in the first endpoint. And we're already getting to really kind of spaghetti code. Code code is being Is it Is kind of what when I was when I was trying to imagine what this would look like, this code looks so familiar and it's kind of how I would structure it, too. So. We see that this is getting complicated quickly even though it's a it's a very simple thing, a very simple concept. So let's look at how you would implement this feature in Workflow. The magic link implementation using Workflow SDK looks something like this. We can see we have our function. It has our use workflow directive, which means this is our workflow function. And the very first thing we do is we call the create webhook function, which comes from the workflow package. Um and we are also using the respond with manual option in this case, which means that our workflow function is going to be responsible for writing or sending the response to the HTTP request that uh triggered triggers this webhook. Okay. And this is so you can like do a redirect or something afterwards when they log in. Yeah. Yeah, so if there's some information in the our workflow function that we need in order to know what to what kind of response to send. Similar to the first endpoint, we are send the login email. This is a U step function. So, if something like this fails, workflow SDK has automatic retries. The the durability aspect is already providing some benefit over the traditional approach here. Yeah, so send login email is a step, and if it's doing one thing, and if that email fails, you you retry the you retry the send email with the same like URL that you created already for the webhook. And if we look here, this is uh this is a very interesting pattern. We're using promise race with a sleep 5 minutes. This is possible because this webhook object implements a promise. So, to await the request of this webhook, you just await webhook. Or in this case, you're doing it with the race. And this is cool cuz I kind of would have expect I kind of would have expected this webhook feature to have a timeout or some sort of option as another argument, but I almost like that it's so much cleaner now that to do a timeout, you just model it as doing a race with the webhook and a sleep. This feels like I can do a lot more with it. I could maybe race two different webhooks with each other. There's not a lot that you can do when you have a couple of arguments in a function, but the fact that it's just a promise and I can do a promise.race against sleep, maybe against another step. I love this pattern. This looks like my mind is just racing and all the things I can build with it. Right, and that's that's the beautiful thing about the primitives that workflow SDK offers. Everything is exposed as a promise. So, standard JavaScript patterns like await promise.race just work. And then another thing to point out here is there's no Redis here. There's no database in in the in the traditional example, we were using Redis TTL in order to implement this timeout, and in this case, we're using the workflow sleep primitive. And also no intern who has to go clean up a messy database afterwards. So. That's the best part. And so you can see that the work workflow responds to the webhook request by redirecting to the login success page and then retrieves information about your user to return to the client that initiated the login page. And that's our entire workflow. That's our Magic Link implementation is 50 lines of code. This is so This is so incredible to see. Can we see this in action? Yeah, so here is our Magic Link demo. I'll just enter my email. And So, our workflow got initiated there and sent the email and there's a webhook just waiting. And in fact, our workflow is suspended right now, so there's zero compute being consumed while waiting for the human to click the link in the email. Cool. And what does this look like on Vercel? Can I see the Can I see a run that's that's like hanging around? Yeah, so we did get the email and before I click that, let's take a look at the observability. I know I'm just like jumping around, but I love that we're looking at this. Okay, so we can see that our run is here and started 40 seconds ago. If we take a look, we have our standard observability features that Workflow provides. We can see the inputs to our workflow run. You see my email address that I typed into the login form. Um and interestingly, we can see our hook here is just waiting. And you said there's no compute running right now. It's like this This is the observability, but there's not There's nothing actually sitting in a waiting for me to click that hook. That's right. So, the hook is waiting and the sleep is sleeping and both of those things are not actually do not actually involve any compute. But we can see that our hook And if you remember, both of these are racing in an in a promise not race. So, one of these needs to finish first for the workflow to continue. So, if I go ahead and click that link, Okay, so we can see that I was redirected to the login success page, which was one of the steps in our workflow logic. And if I jump back to the login form, Right. And then back on the dashboard, it should be that should be complete as well. That's right. Right. So, our workflow did complete. And you can see that the timer just stops as well once the hook's won. Cool. Yeah, so we were able to implement magic link with around 50 lines of code. No, that's that's really neat. I it's so cool to see how like you know, if if if I we just had to like draw a whiteboard diagram of how magic link works and explain it to someone, the the steps the steps that you have in code is exactly how you would map it out. But now that's that's what the final code is, too. There wasn't an additional database in between. There wasn't like multiple API routes. It was just the code you showed me like read so cleanly, right? Yeah, and then I think that is the most powerful aspect of workflow SDK is the way that it allows you to structure your application logic to flow logically as opposed to stretching itself to fit the infrastructure. Right. Also, I love the idea with web hooks here naming it naming it web hook cuz it gives me a whole different way of thinking about web hooks. It's like it's just this ephemeral URL that I can create and to spend on. Actually, maybe this is a good this is a good segue cuz I think about you know, we build a lot of these agents at Vercel. We we build and go to my Slack agents and GitHub agents. And we often subscribe to web hooks from GitHub or Slack, right? Every single time there's a new comment in a PR and we want to kick off a Vercel agent. We want to do this based off of an event that GitHub sends these web hooks, right? Can we use workflow web hooks to actually subscribe to events from GitHub, for example? Okay, so for something like a webhook dispatched from Slack or GitHub, uh the typically you have to go into the dashboard manually and configure a static rear callback URL. Right. You can't create these like one-off. I can't give it a one-off URL the same way I can do with an email here. Right. Right. So, the create webhook feature is a little bit more high-level in that sense, where it provides a randomly generated unique webhook URL uh that maps back to one specific workflow run. In the case of our GitHub for Slack webhook route, that might map to any number of workflow runs. Right. You have to preconfigure something that that you have multiple pull requests, but they all go to the same endpoint. So, in order to implement that with Workflow SDK, we're going to drop down a level. We're going to use the more low-level hook primitive. And I have just a demo to show that to you. Let's take a look. Okay, so this is the Storytime bot. This is the very first application that I wrote with Workflow SDK uh a little over a year ago. And how it works is you just type the Storytime {slash} command, and we're going to see this thread get created. Each thread is represented by an individual workflow run. And so, when we uh expand the thread, we see that an LLM started this story for us, and uh you or me or whoever is in this channel is able to continue the story. And the LLM will help us get it to its its final conclusion. Okay, so Luna has a magical seed, and what happens next? She plants the seed. Okay, and we see some activity happening here. What happens next? Magical thing. Our story is finished, and we have a a final story, and there's going to be a a little image generated as well. But, we'll jump to that back to that. I'm actually really curious already, cuz I noticed that I was expecting one webhook request, but there's actually there was two at least two requests there cuz you had two messages. So, I'm really curious to see what this looks like in code. Okay, so this is the workflow function for our story time bot. So, we can see that uh it takes in the channel ID, which is the story time bot channel. It has some configuration options that you can pass. Uh but interestingly, we can see that there's this messages array, uh which if you're familiar with AISDK, this is the data format that you store your AI conversation in. And in a typical Slack bot application like we're like like we've created here, you would normally store this kind of thing in a database, and on each iteration or each webhook event, each message typed into the into the thread, you would restore your state, you would look up the conversation from a database. And that's not what's going on here. This is just an array in your function. Yeah, actually it's funny cuz I I chuckled earlier cuz I saw the intro and you had this comment that said, "Look, Ma, no queues or KV." And like there's no there's no import here for a database. You're just importing workflow. And like back to the back to the last message thing, this this almost gets missed, but you really you really just have a variable here called final story that presumably over time, you know, we're going to push messages to the this array, I imagine, and final story is going to show up as a string here. But there's no way that this there's no database that this has to go to. So, it's almost like let is your database here. Yeah, let is your database is a a great term that we should We're going to We're going to coin that. I may have stolen that from you, but [laughter] The interesting thing here, and the thing we're here to talk about, is the hook feature. So, we can see that we're we're creating this hook here, and what's different from the webhook example that we saw with the magic link is that in this case we're providing a token, which is a string that includes identifier information that is unique unique to this workflow run. Uh the TS is the thread ID. So, this string uh is is the token that uniquely identifies this workflow run. when we look at the code for the webhook route, we'll see that the event payload that Slack sends contains all the information that we need to deterministically recreate this identifier. And that's the magic of how the webhook maps back to an individual workflow run. Yes, I was curious when I saw webhook because you're creating new URLs for the magic looking example, but you know, cuz we've built these Slack bots before, you can't just have a you can't give it a new URL on every single thread. So, the way this I understand the way you're doing it here is you have an API endpoint that's already hooked up to Slack. But every single time you get a message there, you're you're basically calculating the same token on the resumption side. So, your workflow can basically wait for this this token, and you can construct the same token from a message payload to resume this this workflow run. Exactly. Yeah, so the Slack bot was configured once in a manual click around in the dashboard of Slack uh fashion, and you need to statically define a webhook callback URL. So, that's why the lower-level hook primitive works better in this case because we can dynamically recreate the token. So, just uh to quickly look, this is the webhook route. And there's not a whole lot going on here, actually. Uh the main thing is how we recreate the token from data passed by Slack, and then we call the resume function, and this resumes the workflow run unique to that. So, I actually imagine that that's really cool and I guess I I actually imagined that with webhooks what you're doing is it's kind of the same thing. Is webhook basically just doing a random token and then you have an HTTP endpoint that just resolves the same random token? Yeah, well, so the difference with the webhook feature is that that is a you don't need to define that API route in your code. Uh the workflow SDK actually implements a default route for the webhook feature for you. But yeah, so other than that it's a randomly generated token unique to one specific workflow run. But in this case we have our hook with the token and back to something that you mentioned just a moment ago, this hook can receive data multiple times. This is different from the magic link example which only needed to be triggered one time. In this case we want the hook to fire for each unique message uh that somebody types into the Slack thread. Uh to do that you use the for await uh syntax in JavaScript which is common with uh async iterators, but in this case we're receiving multiple event payloads from the Slack webhook uh using our hook. This is this is so cool. I've I've like I've never found a great use case for like I love async iterators and I love generators and I even did a talk about this a long time ago. But they've always been like good for demos and I haven't been able to find a good way to use it. Here it looks to and this reads to me like you just have a loop. But instead of looping on some fixed set of items they're looping on some like a time a time stamp because you're using for await and then doing it on the hook. You're actually in Here's a loop that maps exactly like here everything inside the loop maps to one user message. Right? And that's like a nice way to think about this with new user message will cause another iteration of this loop and it just queues up and keeps going. The beautiful part about this is that for each iteration of this loop while we're waiting for the user to type in the next message, there's absolutely no compute being consumed. The workflow is completely suspended. And the next message in the thread could arrive in minutes or days or maybe even never, and that's totally fine. So this is probably like Slack threads in that same channel with Slack bot where I could go back now and they're just still a run just sitting and waiting for a couple of weeks if no one responded. Oh, yeah. That's really cool. And so to touch back on that messages array that we mentioned earlier, now we're modifying the array. We're pushing the the new user message, and that's that's our database modification because our messages array is just a local variable. That's so cool. And then I can yeah, you see you doing more promise or alls to parallelize more steps in between and other This reads so cleanly for every single loop or every single message on on Slack. I like how I This is exactly how I would model this if you were trying to build this at a hackathon or something. It's kind of like here's how I would write down what happens in every single message. Yeah, so in the the promise.all model, these are just regular use step functions, and the idea is to run them in parallel. And so something like adding a reaction to the Slack message is just to give more immediate feedback to the user that, you know, something is happening. Uh but at the same time, we want to initiate the LLM to help move the move the story generation process along. I'd actually be so interested whenever to to see what the observability looks like as well when we when we get around to it cuz I can see I can imagine those sort of spans starting at the same time and making it super obvious. All right, so we do have the observability for our story time. It is completed, so we'll have to go back and check on that image. But we can see our hook. Uh and what's interesting here is that in this case, we have two hook received events. So that maps to the two messages that I typed into our Slack thread. And the observability allows us to see the individual data that was provided to the hook. Oh, that's really cool. So, that's every It's basically the Slack payloads, and you didn't have to log this additionally. That the Slack payloads just show up as events that I can go back and and inspect uh on the dashboard. Right. And then we can see that each time the hook payload was received that continues our workflow execution, and our steps continue, and then we finally have our result of generating our storyboard image, which looks like there. So, that's Story Time Bot. That's That's really cool. And And then seeing webhooks both, you know, generated from magic links, and now seeing you use the lower-level primitive with hooks, and also do them in a loop so you could actually do multiple events. That's really cool. I feel like um I feel like the the model like really clicks for how I how I do human operations with uh webhooks. Is there anything else I can use hooks for? Yeah, definitely. But, the last demo that I have planned for you here is a very similar pattern uh to responding to a Slack webhook. Um but, in this case, we're going to be using a webhook uh as a way of kind of handing off execution of our of our application code uh and then waiting for some compute to finish elsewhere while our workflow suspends and waits. Uh and then using that webhook URL to call back into our webhook, and then we can finish doing whatever we need to do with our application logic. For this example, uh we'll use Vercel Sandbox, and we'll do some long-running compute operation uh like using FFmpeg to do a conversion of a file. So, this is our FFmpeg conversion workflow. Uh one of the first things that happens is we create a Vercel Sandbox. What's interesting about this is actually that the Sandbox NPM package exposes functions that underneath the hood have use step in them. So, actually this this operation is a step. So, you heard right. You can ship an NPM package. Sandbox Sandbox is basically just shipping an NPM package that has use step, the directive, inside this function. So, when you import it and use it inside a workflow, it mean it automatically makes Sandbox a step without you having to write any of that code. Doesn't mean you can still use Sandbox.create outside of workflow. Like, what happens when you call this without a workflow? If you realize the directive is just a string, and if you were just going to execute this with without the workflow compiler, that string doesn't do anything. So, this just works. Adding use step in your in your NPM packages just works fine without Workflow SDK. And then once you use that function inside of Workflow SDK, you get the added benefits, the durability benefits, out of the box. Okay, so the Sandbox uh just does some typical things. Uh installs FFmpeg, because that's not available by default. Downloads the URL of a file that we're going to specify. And each of these runs are just steps as well, right now? Yeah, so these uh run an individual command in the Sandbox. Um and that those are those are steps. We'll be able to see them in the observability. Um and then we're we're back to calling create webhook, which you might remember from the magic link demo. Uh but in this case, we're just going to pass that webhook URL into our bash script that we're going to run in the Sandbox. What's going on here is we're going to run FFmpeg and convert the file uh to the format that we request in the UI. And then when that's done, the bash script is going to run a curl against our callback URL from the webhook. And when that curl request happens, our workflow logic resumes. Okay, I got you. So, that's cool. I can see already I'm skipping a little bit ahead, but I noticed that there's an end on this on this run. So, you're actually writing the script, running this in the background because then FFmpeg step like this could take a lot longer than you know, you don't want some you don't want a step that's just sitting around and waiting for it. Right, right. So, this this line right here starts our FFmpeg conversion script in the background and then our workflow function suspends and then we wait for the webhook to be resumed. And and I see the you see the promise race again with the with the one-hour sleep. That's such a cool pattern. Right. And so, this time, you know, our our FFmpeg conversion process might take a long time. It could be a a very large media file. So, we're specifying a one-hour timeout in this case and that's totally fine. In workflow, you can sleep for essentially indefinite amount of time and again, there's zero compute running while we're waiting for this webhook to be resumed. And can we see this Can we see this run? Do we have a demo? We do. It's a little bit of a silly example. Yeah, no, I recognize the Big Bunny example immediately. This That's from Blender. Yeah, I remember looking at these videos when when learning Blender a long time ago. Oh, wow. I'm jealous. So, we got our media file URL pasted in. Uh in this case, we'll just uh extract the audio layer from it. So, once we click the button, that initiates a workflow and we should be able to go over to our observability. Oh, there it is. Yeah, so we can see our sandbox create and that returns our sandbox instance. It's pretty cool. And this is cuz sandboxes cuz everything in workflow has to be serializable, but like you said, sandboxes implement the serialization, so they're actually serializable as well in the shop in workflow. Right, yeah. So, the the Vercel sandbox npm package has a sandbox class and that class implements the workflow serialization uh functions and so it just works in our And And so any package can do this, right? It's not just sandbox. There's nothing special about sandbox that like any class that wants to work inside workflow could implement the same symbols and have you step directives. Yep, that's right. So we can see that our hook uh ended up being called back in 20 seconds this time, a little bit faster of a conversion cuz kind of a small file, but that could have been any amount of time. We can see that after our sandbox got created and initialized our hook got created and we passed that into the sandbox to initiate our FFmpeg command and when that finished we received a payload from our sandbox. Oh, and this is the curl This is the curl that happened inside the bash script earlier. So it's running the command and then Lewis using curl on a sandbox to complete the webhook. Right. So our our our sandbox finished the work it was doing so it's handing control back to our workflow. Done. I mean so the way the way I'm thinking about this now is with steps in workflow, you know, you run a step, it runs a code in the background, and then continues the workflow. But web like hook and webhook both feel like they were lower level. I can just create a token or a URL and wait for anything. That could be a human magic link, it could be an email, it could be a sandbox, it could be any sort of computer, whatever that has to happen and have my workflow just like pause with all of its state till that event happen. Like it almost feels like it's lower level than step itself. Yeah, the way I think about it is webhook and hook are a way of passing in external payloads into your workflow. The way I think about this is like step is a way that a workflow can suspend and then wait for some sort of, you know, compute to finish and then we resume. But hook and and webhook both actually feel like they're even lower level cuz you're just making a token or a URL that you could send like in this case a sandbox, but you could send anywhere. Could be a human, it could be an email, it could be some sort of it could be another workflow for example. And whenever that completes, your parent workflow basically wakes up and and resumes right where it left off. Right? So it's somehow even lower level than step. It's a way to just suspend your workflow for any sort of external action. Yeah. The the way I like to think about it is hook is a way of suspending your workflow and waiting for some external payload to be passed back into your workflow, which is very powerful. This is really cool. I know we're out of time today, but with those demos, you kind of validated again to me why a hook is my favorite feature in workflow and I'm so excited to keep building with it. Awesome. Yeah, I'm glad you enjoyed.