Laravel AI Chatbot Trained on Your Data: Example Project

Hello guys, when talking about Laravel and AI, we don't need to just generate Laravel code with AI, but also the other way around. Use AI features in Laravel projects. So in this video, I will showcase you how we implemented a very simplified version of this platform with AI features from Upwork. Not the full project, but basically these two parts. Someone uploads the document, for example, travel policy document. In this case, it may be txt, it may be PDF. Then it is split into text chunks. And then we have chatbot Q&A. So from this full description, I asked Chad GPT to simplified it to just the basic management of file upload and AI processing. And then I asked my colleague Modestas to implement that in a demo project. How would that look with Laravel and PHP tools without any external over complications? So the minimal set of what we need to learn to implement something like that with our current Laravel knowledge and in this video I will show you the results. So this is the project based on livewire starter kit. We have a document uploaded which is split into chunk and then we can ask a question how much per night in for example Chicago and then it would find the answer and reply in a few seconds something like this. So what we will use open AI API for transforming into embeddings but we will call that API with regular Laravel HTTP client. Then starter kit is live wire. So we will use livewire for dynamic elements on the page. And then we will use a lot of PHP services to transform the original text document which looks like this into chunks embeddings and then vector search. Again, this is a very simplified version with one goal to show you the concept of how that process works in general. So, I'm thinking maybe to dive deeper into that topic of using AI like chatbots or general AI in Laravel projects. So, in the comments, let me know which subtopics do you want me to dive into. I probably will create a specific course on using AI stuff in Laravel. So even this project may be much more complex with different tools and I will mention them as alternatives at the end of this video. So basically let's discuss what we can create. What do you actually need to learn? How do you feel the market? Do you have any demands from clients or just job descriptions or somewhere so I would know where to put my effort in diving deeper into that topic. Before showing you the code, I will show you two diagrams so you would understand the concept of how that works. So when someone uploads the file, it is extracted into text, then into chunks. So for example, 2,00 symbols and then 2,000 symbols and then so on and then transformed into embeddings which are then used in vector search. So whoever asks the question, the question is also transformed into embeddings and then those two embeddings are compared in vector search to find the right answer similar text and then we will also use openAI API to transform that search result into human friendly answer. And this is a bit more technical diagram of the same thing but basically you can see how many services we will have PHP services and then where we will use open AI API. So someone submits the question then we have validation of that question and then we ask that question we embed that into embeddings again from here we call open AI API and get the embeddings back and then we perform the search of those embeddings against the uploaded document chunks. So get all the chunks and then perform the search return relevant chunks and then build the message for open AAI API. We generate the prompt to help us generate human friendly answer which is then returned again into the browser for user. So it may look quite complicated but this is a pretty typical chatbot scenario in a very simplified environment. So I'm missing a lot of details like extra validation on each step. There may be parameters like which model to use for embeddings and for text result. How long should be the chunk and so on. So there are a lot of moving parts here. Also probably for each step we can use different tools and I will mention them alternatives at the end of this video. But again in this video the goal is the overview. If you want me to dive deeper, comment down below and I will create probably a course on Laravel Daily and a summary with more things here on YouTube. And also if you want to get the source of this project that I'm showing you, I've uploaded that on Laravel daily project examples. This is available for premium members. So this is one more reason to subscribe to premium membership. It's not only courses, it's a lot of project examples. You can see them in the list including that travel policy chatbot. Okay, enough intro. Now let's dive into how it all works and what we can learn. The first step is easy to upload the file and put it into storage. So let's call the file ABC and let's upload the policy for now in text format txt for simplicity. Upload policy and it is uploaded in the code. is just a simple Laravel controller policy controller with store with just file store and create database record for policy just with file path. For now we didn't do any processing and this file is in storage app private policies folder as something encoded file name.txt and we have a new record in the database with this file path. So this is the easy part but by default this file isn't started to be processed. We just upload. We may upload multiple files. But when we click start processing, then the actual processing starts. And what actually happens with that button? We have policies show. And we have livewire component here. Since we have livewire starter kit and we have flux various elements like flux button and flux badge all from free flux UI version. And in here we have start processing. And this is powered by livewire component policy detail. And start processing looks like this. Start processing. We have process policy pipeline. And let's go inside and see what happens. So there's a job being dispatched to extract policy text. Our first step of processing is to read the text from the file. In our case, it will be text file, but it may be PDF, it may be doc, it may be whatever. For each extraction there should be different processors and as you can see here we have extractor text extractor service which is automatically resolved in the service like this extract method and then we have match by policy mime type if it's PDF if it's office document or just txt. So extract from txt is just file get contents. So after that extract policy text job we should have policy update of the database with extracted text and then it fires the next job chunk policy text job. So chunking is the next phase. Let's see what's inside. And chunking has a separate test chunker service. So we had extractor service. Now we have chunker service which basically chunks the full document text into chunks of 2,000 symbols with overlap of 200 symbols before and after to not lose important context information. So this is just working with the text with paragraphs and so on. And as a result we have array of chunks. And then that array of chunks is placed in the local database policy chunk create for each of the chunks. And then after it's done this job calls another job embed policy chunks job. And this is what calls embeddings from open AI. We'll get to that in a minute but for now let me demonstrate how it actually works. Start processing. Document is being processed. And of course we need to launch the queue. So we do PHP artisan Q work and now we have extract chunk and embed for that policy and this is the result. So we have document details from the database the preview and then chunks also from local database. So if we refresh our database table, we should see extracted text here with a lot of text here and then some summary, meta, lines, word count and also it shows character count and embedding model. You don't see that in this text area. Maybe you can see that at the bottom right. I will drag that here. So yeah, this is what is in the meta database column. Now how does it work with those embeddings? So embed policy chunks job it has embedding service. So for each of the things we have service and this is where open AI comes into play. We choose the model for embeddings. It's not like GPT model or cloud model is just specifically for embeddings. For that we need to configure our open AI API key and make the Laravel HTTP client request to embeddings. This is just an HTTP client request from Laravel without any external packages. But of course, there are packages to do that. But in this project, I'm trying to show the simplified version. So you would understand the concepts. In later videos, we can experiment a lot with various other alternatives for pretty much every step here. So again to recap from the live wire component or it could be laral controller, we start the process policy pipeline service. It may be just pipeline class not in app services but this is a personal preference which calls the extract policy text job which after it's finished calls the chunk policy text job which in itself after it's finished chunking calls the embed policy chunks job. So three things that we need to do with the text document also in the database. Now we have policy chunks which are these. So, we have policy ID, our doc number six ID, and these are the chunks, roughly 2,000 symbols each, and these are the embeddings created by OpenAI and posted into our local database. So, such numbers, embeddings are the foundation of how LLMs work and other AI related technologies. Each word and phrase and other text is transformed into vectors, so-called vectors, which are basically just a lot of numbers. In the structure of that database table, that embedding is a JSON. And this is what I probably didn't emphasize, the part of the code where those embeddings are updated is here. So, we create the embeddings with OpenAI API service and then store them locally. And now we can start asking questions to our policy. So for example, let's ask for something about I don't know, let's zoom that in and let's for example refer some city and ask what is the travel policy for that city. How much per night for Atlanta for example in pretty free human language form. Let's ask analyzing and let's see what it returned. So this is the answer from chunk specifically and this is the human language answer which is corresponding to this specific chunk this one chunk two. Now how does that part work and also by the way we save all the questions just for future reference in the database we have policy questions as well with data. These were my previous tests but this is question answer and citations from specific chunk in this case that ask question form is another livewire component inside of that policy detail. Live wire is great for such dynamic refreshing of things. And this is the text area for the question with wire model and when it is submitted flux button wire target ask which is an ask question livewire component. This is a classbased component even with livewire 4. So this is ask. First we have validation with validation rules just like this. In fact, it should be much more strict probably for real AI production usage. But anyway, this is the simplistic example again kind of like foundation so you would understand the concepts. And then we have QA service question and answer service with ask method. What does it do? We have policy as a parameter. So the original doc and then the question and then we have these steps. First we need to embed the question. This is where another time embedding service from OpenAI API takes place. So the policy chunks are already embedded are already numbers JSON. We need to embed the question as well and then those JSONs are compared with each other. So the actual search of similar data, similar words, similar phrases is happening in the embeddings under the hood. So we need to transform the question into embeddings. And this is where that search happens. Vector search service. So we pass the embedding. We pass the policy that has embeddings inside. And let's see what that vector search service is. So we get the chunks. And then we have map for each kind of of each chunk. And then we're looking for similarity. This is called coin similarity. So in this case we're doing that kind of manually. This is how algorithm works under the hood. But actually there are tools and packages and vector search mechanisms which do that without manually writing those PHP functions. But so you would understand the algorithm. We have similarity for each of the chunks and then we sort those chunks by similarity. And also we have a filter for minimum similarity for that chunk even to be returned and that min similarity is zero but you can play around with that and make it bigger. So these are search results basically chunks. Then from those results we need to build the answer the human language answer and here we have rag prompt service another service we need to build the messages from policy the question and search results. Let's take a look here we're building messages for open AI. So we will ask open AI to prepare the text the human language text for us. So we're building the system prompt from the policy. We're building the user message. This is the prompt for the user message. This above should be the prompt the system prompt. You are helpful assistant and so on. So we prepare that prompt for OpenAI messages. And then we have answer generation service generate which again calls OpenAI API with our API key. And in this case we're calling chat completions. basically calling the chatbot chat GPT if you will with model and our messages and it returns our answer in the human ready format and also on top of that we also build citations for those citations we also work with chunking again I can show you that as well so we specify the number of the chunk in this case and then for built citations it's just returning the text of which site citation it was from which chunk from which page. So basically building this chunk to citation source and this comes from open AI API and yeah then we store the policy question in the database return the answer which is shown on the front end and now after this system overview at the end I promised you to talk about alternatives. So first there are a lot of things that you can change inside of the same code. So for example for external dependencies we're using these models. So embedding three small and also GPT4 mini for text results also we're using libraries like PDF and word. I haven't demonstrated them in this video for text is just native PHP but you can play around with those and change those. And also there are a lot of parameters what you can change for processing the documents. So the size of the chunk, the size of overlap between the chunks and so on. So this is just inside of that project with the same PHP services and PHP tools. But also on top of that, you may choose other tools for all of that experience. For example, Taylor Artwell recently tweeted that for upcoming AI SDK in Laravel, he added Postgress vector stuff into the framework with similarity search tool. So you may switch from MySQL in my case to posgress and use that feature. I'm going to test it myself and shoot a separate video about that as well as AI SDK. will be at Laracon India where Taylor will present it and probably will launch public beta as he announced on Twitter sometime in February. So yeah, I guess we can then create a chatbot similar chatbot with AIS SDK as firstparty Laravel. And then finally, I've listed here alternatives for other stuff in my Obsidian. So I asked Chad GPT and Claude and made some research. So these are things that we may talk about in the future. So you may swap vector indexes for the thing that I mentioned posgrql. Then also vector databases there are a lot of separate solutions to a managed vector db. So you would store embeddings there instead of local database. Also for embedding providers instead of openi you may use others. And finally, of course, generating the answer may be much more complicated with more validation with more logic to provide the most accurate answer possible. So you may play around even with this code on GitHub and see where it may be failing. Again, the full code is on Laravel Daily in project example section, including in the docs a few sample documents to process. So yeah, what do you guys think about this whole topic? Should I dive deeper into one of these or other AI use cases in Laravel? So, we would add new skills, AI skills on top of web framework that we use and love. Let's discuss all of that in the comments below. That's it for this time and see you guys in other