Astro's NEW take on content (live content collections)

Content collections have been core to Astro for a long time now, but there's something new coming called live content collections and that's what I want to look at today. You ready? Let's go. [music] Hey, what's up? My name is Chris and welcome to Coding in Public. Okay, so last week we looked at content collections as a whole, like from a macro perspective. Today I want to look at live content collections which give you the same API but you get the data live instead of at build time. There's also some nice cache helps and a bunch more, but you have to build a little bit more yourself. And that's what we're going to look at today. Let's start by looking at this overview I showed off last week if you didn't see that. Basically, this is how content collections currently exist. You define everything in a content.config.ts file where you define each individual collection that has to have a loader and can optionally have a schema to define the shape of the data. Now, you do get some helps with the loaders. For instance, if you have local files here, they have two built-in functions, glob or file. These are both loaders that you can use without any extra configuration. You can also write your own custom inline one or for more control over how everything is stored, you can use the object loader. Now, this is all build-time data. It stores it all at buildtime and then whether you're using SSG or SSR, it pulls from that same store. Then you get collection or get entry to actually show the data here. And again, the benefit is type safety through your entire stack. So that's how it currently exists. Remember this structure because it's going to look very familiar in a second, like right now. All right, look at this. So we now have a live.config.ts file where we use define live collection instead of define collection. But notice it still has a loader and an optional schema as well. So same basic structure. You do need to use the object loader. Remember this is the one that gave you more control over how the data was displayed. Well, you do need to do this in part because you have to actually define manually what the collection should be. In other words, what's returned from the collection and what a single entry should be. So you're doing this live and you have to actually handle any errors. So it's really nice on the front end, but it does take a little bit of work when you're actually writing it. Now on the front end, there is one difference here and that is you need to make sure you have an SSR adapter and for the particular route, if your whole site isn't, you need to make sure you serverside render that particular route, which I've done here. Then you'll use get live collection or get live entry. So same basic idea and you just display the post right here. Now, there's more we can do with caching, which we'll talk about in a second. But let's go back to the error handling first. When you're using a live loader, lots of things can fail. You can have network issues, API errors, validation problems. So, if any of these things happen, you need to be able to handle them in the actual loader itself. This is why we have a load collection and load entry. You have to define what happens in each of those instances. Now, Astro is smart about it and it will actually automatically return some errors. So for instance, if you pass in some kind of ID for a specific entry and it's not found, it knows that and it can say, "Hey, this isn't found," it will return this type of error. If it's a validation error, let's say you pass in a schema, and you say, "Hey, this is what the shape should be like." And if it doesn't match that shape, it will return this validation error. If there's a cache hint error, which we'll look at caching in a second, in other words, there's something invalid in the way that you've done the cache the caching, then it will return this error. And then there's also a live collection error for anything else. You can also define your own custom error types if you want to, but that's probably more for like if you're, you know, writing this for your product and you want lots of people to use this uh this loader. So, I'll let you check out the docs for that. So, all that to say that you can actually get specific errors here. We'll look at all this in code in a second. And then the other real note here is caching. If you don't want everything to be always live, you can actually provide some cache hints and then use them to set headers when you're rendering. Then you can tell the browser, hey, set these headers. And then you can invalidate specific entries or entire collections if you want to when you're ready to update that. So you'll have to invalidate that cache manually. And I think there's more to come from the Astro team on this eventually. Okay. So hopefully that gives you a sense of kind of what we're looking at. It's very similar. The details right here can be a little confusing. So let's look at that next. Okay. So over this way you can see I've got products and live products. And the real benefit here to kind of see the difference is because these products are using a object loader. the remote data, but it's just using content collection. So, this is at buildtime. If I were to change any of that data, let's open that up right here. And I say, let's call this, I don't know, uh, wireless headphones, uh, two. All right, new and improved. You'll see that the live one updates. This one does not, right? Because it's just from the cache store. If I want this to update again, I'll have to restart my dev server, which grabs all that, stores it once again. So this is the real benefit of live content collections is you actually get all the same API but again you get it in this kind of live environment. All right. So how are they defined? Live.config.cs same kind of thing. And notice we've got a loader. It looks very similar to the loader we used when I showed off kind of how to write a loader last week. But you can see here that I'm passing in a URL. This happens to be to my local dev server that's kind of mimicking what this would look like. And then I have a schema optional here as well where I do some kind of manipulation on the data. All right. And then I just export the collection. So simple enough, right? Well, here's where some of the details come in. This live uh products loader. Let's open this a little bit bigger. You'll notice that we have a name and instead of a load function which you used on a normal object loader, we have two items we have to provide load collection and load entry. Notice we've got this nice helper here where we can pass in the product. And then you definitely want to pass in some kind of filter. um interface and also a collection filter. Otherwise, you'll get no type safety on kind of the front end. This allows you to filter out certain items and like for the entry. That's really important because that's how you actually pick off the individual item right here. Okay. So, let's come back up top here and let's look at this. So, for the load collection, this should look similar to what we did before. We're going to grab all the data, then we grab the products. The difference here is what we're going to return. You return entries and you can optionally return some cache hints. So for the entries, what we need this to be is an array of items. Here I'm picking this off the filter if it happens to exist. Then I use the filter and I just have one which is is it in stock or not. And so you can define each of these separately if you want to and handle that filtering out yourself. Uh otherwise what I'm going to do is loop over all the items and they have to have an ID and data. Now you can also have cache hint. There's also another helper here where you can actually have something rendered and this would be like rendering markdown for instance uh to HTML or something like that. So these two though are required. The ID and then the actual data you get back. Now the data is just the structure you'd expect. This is the kind of the new section here, this cache hint. So we'll look at that in a second, but for now just notice that we're going to handle any errors ourself. So that way we get back data or errors automatically. So we always have that handled for us and you're kind of forced into that in this structure. Now for load entry looks very similar. We're going to grab all the items uh just because that's the way I have the server set up. I think actually with JSON server you can pick off an individual item. So I could probably handle this differently but we're going to then find a single item and we're going to make sure that it matches whatever filter we pass in. Then if it's not found we throw this error product not found. Otherwise, we return again ID data and then cash in again is optional, but I'm going to show you that in a second. Then again, we handle any errors. So for these cashins, either on the individual items or in this loop of the individual items, it looks the exact same. What we're doing is we're going to tag it with product with the number whatever ID it happens to be product one, product two, and then category as well. This will be the entire category like electronics or whatever it happens to be in the data structure. Now when we do this, we can actually on the front end add these to the headers and that way we can invalidate that specific tag if we want to. Now I won't show you all that right now just to note that you can actually set set this off. And then we also have this last modified date which is just the day that the data has been updated. So I'm setting this on each individual item. Then I'm setting a tag for the entire product category. So anything in the collection should also have this tag of products. Now what this means is if I invalidate products as a whole that whole tag then every product will be you know grabbed again live fetched live when I hit that route. Okay, same kind of thing down here again product whatever that product ID happens to be and then the category as well would be the category wireless electronics whatever we also have this last modified date. So hopefully that makes sense kind of how that works the options you have available to you. Now on the front end let's go ahead and first of all just look at the collection. If we grab the live collection, then notice we grabbed live products and I've passed in my custom filter. Now, this is slightly different to how you do it in a content collection because again, I custom defined this, which means I can come in here and get type safety because I know the exact items. I can say, hey, show me if this is in stock and I can set this to true if I can type and then I just get those items. Now notice here I can say if there are cache hint tags then I want to just go ahead and set this cache tag and I'll set it on the this route which I think happens to be used in my index route. So I'll show you all that in a second when we look at the individual items. But notice then I just map over it like normal. Uh it is a little bit different in that I have entries cache hint and I also have an error. So I probably want to handle this error here as well. These are the three things you get back from live content collections. So, it's not data here, it's entries. And then on the entries, you actually loop over those. And each of those have their own ID and then their own data as well. Hopefully, that makes sense. Now, notice I'm just linking to the live product. So, let's look at those. Here are the individual items. Again, this page needs to be pre-render false. Same thing for this index route as well. Um, because we're doing this live, right? And then you'll see that I just do some basic error checking to make sure it's there. And I get the same things except it's entry instead of entries. I get error and then cache hint. If I have provided that I do need to have a filter on these to make sure I can filter off the individual item and in this case I called the page slug. So I'm passing that in as the ID but it's the same thing. It's the ID of the actual entry. Down here I've got the tag. So I'm setting it for this particular item. I could also add in here if I wanted to like the product tag itself I guess because technically this should be refreshed as well if the entire product is refreshed. And then I've got a last modified that I'm setting as well. Then, as you'd expect, I just show off the data right here and I have type safety all throughout the stack again right here. Now, by forcing me to handle the errors, I can be confident on the front end that I know what's going on, right? So, if there is an error, I can redirect to 404 and say there's an issue. I obviously could grab specific errors and do different things based on that error. But you get the idea. I'm forced to handle the errors, and that's the important thing. So, I want to come over here and let's look at this live product here. And I just want to look at the cached headers we get. So, if I come over here, uh let's uh not disable cache and I'm going to come in here and we'll grab this here. Let's look at the actual headers we get. So, notice I have these cache tags and it combines them all. Product one and category electronics. I also get the date and then I also get the last modified date. Now, these are items I can use to invalidate cache if I want to. You can do this through your CDN or however else you handle your caching. So, these are the building blocks to build something really robust that allows you to have caching control. But obviously right now it's a little complex. Now at VC Comp this year, Matt Kaine, one of the core maintainers of Astro, talked about a feature caching strategy that will make this a lot easier. So I'm hoping that comes out soon because obviously setting all these manually and handling it all yourself can be a little complex if you're not used to doing this thing, which I'm certainly not. So it's been a lot of learning for me as well. Well, let me know if you have any other questions about live content collections. Obviously, this is more of a preview of what is to come rather than us building this out from scratch. I'll include the code down below though, so if you want to play with it, you're welcome to do that. I hope you're excited about the future of content collections with live content collections. Once it goes to production with V6, I'll make sure to do a video actually walking you through the entire process of building one yourself and we'll kind of do that together live. But today, I just wanted to give you an overview of what's coming. Well, thanks so much for watching. I'll catch you in the next one. Happy cutting.