This AI SEO System Builds and Updates Revenue Pages Automatically

Tricky Raffin, welcome back to the podcast. You wrote, "The key to any automatic SEO system has to be autotopic and keyword research that has depth to it and automatic clustering. My keyword universe has keywords flowing in via five different sources automatically. We turn the scraped content into a vector database and my keyword agent looks at the data and generates seed keywords. We use Google Search Console data. We scrape forums and scrape and analyze product/service data and everything flows into the keyword universe. Data gets populated and automatically clustered into topic groups. Then keyword groups get mapped to pages or turned into new pages or removed if they are not relevant enough. All automatically incredible stuff really. But nothing of this works without the AI brain that sits at the top and profiles the business. So we have this is super cool that profiles the business. So we have a deep understanding of what they want to sell, who their audience is and who they want their audience to be in the future. And so that is what you are showing us on this episode of the show. Thank you again for coming back on. Yeah, thank you for having me. Looking forward to going over this here. Right. Oh yeah. So yeah, um I last time we spoke, I started building this system because I saw a lot of these automated SEO tools. And what they basically do is they turn, you know, a content gap between your site and and a competitor into like a 30-day content calendar is just on the blog side. But I wanted to build something that's maybe also takes into account like what keywords you want to rank for, what topics you're missing and more importantly like what what is the page type, you know, you want to have landing pages, you want to have tool pages, uh service, not just blog posts. Everyone's just doing doing this for blog posts. Yeah, exactly. So, I just sat down and started building that and and then this is what I came up with and I think it's pretty pretty awesome. But it starts here with the like with the brand DNA and um we basically take like it has like eight or nine sections and the agent I can talk to it here but it basically profiles the business who your who is your like who's your um who's your buyers who's your customers what do you want to what do you want to sell what products do you want to have and it start populate your identity like your your voice and tone It takes all of your service pages and builds out this section. So, this is specifically great for e-commerce. It like takes all of your products and just builds them in here. Uh, you know, brand term, site structure. It just has a lot of things that you can build out. And this allows us to have a section called keyword universe where it just takes all of this data and starts discovering keywords and adding it into this section here. Right? So, wow. And we can talk about um automated SEO because like when you're using chatbt or claw to do SEO, you're asking it questions. Uh you're you're kind of missing a lot of the automated things that are important and that is you know building a data pipeline. So that allows you to actually call you know with API calls call href in a way you're you're like your error handling you're then storing that data and because you might run into problems and things like that. So what what I wanted to build was something that stores the data and allows you to crawl and and you know just build a universe around your keywords. So everything that we see here is just fed in automatically basically and we can see um here's the source it this one hasn't been been added on but we basically have AIC it finds that via the brand DNA and if it finds a topic that it thinks is relevant it scores that you know from zero to 100 and that way we can keep you know the the keyword universe uh keep it you relevant to the business. You know, free local SEO tools is not relevant to my business. Free SEO tools versus paid might not be and it gives a reason here. So, we can then if we wanted to, I can go in and mark it as as relevant. But this keeps the universe, you know, uh manageable. Okay. This method of marketing is so effective, I had to make sure it wasn't against Google's rules before I kept doing it. It's a form of SEO I call compact keywords. Whereas most SEO focuses on putting up articles to answer questions how, what, when, compact keywords focuses on putting up dozens of pages that sell to searchers who are actually looking to buy. These pages rank on Google and convert so much better than normal that when I discovered this years ago, I couldn't believe this was allowed. It's less work, too. The average compact keywords page is only 415 words. Compact Keywords is a 13-hour deep course on getting sales with SEO. A customer recently said, "Each lesson is dense with information. You're giving years worth of experience boiled down into 15 to 30 minute lessons with no filler or fluff. I feel like I'm gaining a new superpower. Compact Keywords is about setting up an SEO funnel that brings you sales for years and years and years. It works with AI. It's less work than traditional SEO and it makes way more money. You can get it now at compactkeywords.com. Back to the podcast. What do you think? This is awesome. Um I'm I'm curious how long it took you to make this and how how did you make it? What did you make it? Uh it's taking like four four months something like that or pretty relentless. And yeah, four months just like day and night pretty pretty intense. Gosh. Yeah. And but yeah, what it does basically we built a a pipeline. So the first thing is discovery. So it goes and finds the relevant keywords. The second part is it runs the the relevant score and the relevant keyword get the status retained and then you have excluded. the ones that haven't run there, they're they're just uh it's called candidate and and then it checks the SER it checks the volume and when we have done that it start clustering them automatically and you start building out the keyword topic clusters here right so and then you can see what keywords belong to digital marketing agency for example um because I don't know if a lot of people understand why we're clustering the keywords so you might have a lot of keywords but they might belong to the same page. So, Google's going to show the same page when you search for these different keywords and instead of like looking at all of these different keywords, you just analyze the head term and all of the rest of the keywords just sit there as member keywords basically, right? So, it starts clustering and it creates these two because we ran into a few issues when we're autoclustering. It is like you create a cluster and then you get more keywords and then you want to keep the cluster open. So we we created something called singleton or or you know it's still open. We can get more keywords and that can get populated. So this one has only one keyword. So we can actually go in and expand if we wanted to and it will find more keywords to that topic and you know it just keeps the system dynamic and yeah. Can you share um some results you've gotten with this so far? Yeah. So, we have uh I can I I can share the specific I can show you off camera spec specific site. Yeah. Yeah. But you could if you could talk about it. Yeah. You don't have to say the site. Yeah. We have a pretty good keyword for a very competitive uh topic in the UK to number one. And uh it's just basically a best article and a lot of supporting content that that helps us rank for that. and it it's bringing in a lot of sales for that business and it's a high ticket one so it's like average sale is like 8K so it's pretty good and yeah it's and that was that was automatic or how much of that implementation was done by you it's uh so the system is built in a way where you set things up and there is you need a review state so I have a re review state here if I want to I can run this automatically as well Then so let me talk about this. So it it creates the clusters. It finds for example Shopify SEO it found that we don't have a page for it. So it creates a page. So it it that gets sent to something called command center where we have this playbooks. Uh all opportunities like you have new keywords like this is a a topic it found. I can then play that. It gets wow. Yeah. And then it gets sent down to mission controller. have agents that actually like write out the pages and and Oh, cool. Yeah. So, yeah. So, like once this is ready, it goes into neat review, right? We want to have uh we want to have our review team take everything that that it creates review. Yeah. And it just goes over and says like, "Yeah, these signals are correct. The content is good. Let's change this, add this, and then you get it uploaded basically." And yeah, so it's it's it has human involvement definitely. You don't want to run it automatically, but you can if you wanted to like just connect it to your store and just can autopop populate all the the topics and keywords and but I think you need to have like a step in there that's that's human. So you're not you're not doing scaled AI content with this. No, no, no. It's it's I mean like agents and LLMs are very good at at you know uh finding signals, right? you're we're taking all this data into the system from Google search console. We're taking, you know, all of this keyword data, performance data, and we're turning uh our pages into embedded vectors. So, we're we're vectorizing the the content, and all of that just helps us find signals. Okay, the agents are good at interpreting the signals and deciding what needs to be done. But we create the playbooks. So we say like this playbook should run when this happens. So when you have a topic that's in the top 20 is very good to analyze the page find uh information gain uh look at missing entities and things like that. So it goes and does that and that then you can go and review that right that's that's the strength of the system is it like gives you the the data to analyze. Yeah. What I what I thought was also really cool was the focus on uh commercial pages on on pages that were meant to sell. Yes. Which you talked about, not just not just blog posts. No, like I mean it just takes it just basically finds a lot of keywords and it then analyzes the intent. So that's very important. when you have tools that just look at Google search console data for example, it kind of forgets that you need to analyze and store the intent and and it's good to track it over time because it can change and that's one of the signals you can have in the system is that you know your your your intent changed so you need to review the content or like switch page and things like that. Um yeah, I think it's pretty pretty awesome to be able to build this just with AI. you use cloud to to make this? Uh yeah, cloud and codeex. Awesome. Um do you have like a library? So you're doing this for a brand? Like let's say you set this up for a brand. So what what does that look like in terms of how you how you learn everything about the brand? Do you have like a library of images? So if you wanted to make like a product landing page for example, like how does all of that what does that that whole workflow look like? Yeah. So the the the landing page builder is not completely ready, but we were using uh basically chatbt to to generate images and try and get the brand you know copy of their their landing pages. So and then it's just human work to actually fill out the content and add the images. But we're building the landing page like builder out so we can have it a little bit more automated. Um but yeah, would you would you consider actually because like um when I when I make alt text for articles, you know, probably a lot of people have way more sophisticated workflows. I just drag I put the article into chatbt. I say I I have like specific file names for images. Uh, and then I say write the alt text for these images um using the file names and the article as context and it is like a very good job. So would you do something would you ever think about doing something where you have the images and then you have like every you have everything that you will that the system has learned about the business uh and then actually writes really thorough alt text and and tags tags for images that would make it really easy to insert them into relevant sales pages or articles. Yeah, definitely. Uh we do that with like Shopify has some good apps that do this. You just create these these uh templates and it runs it on all your files and just renames the the files and and the you know add the right tag that's very descriptive that works really well. And we're building something similar. We have the image and media library. So everything that we create for that brand, it gets stored here and basically the the file names and everything as well. And then we use that in the the blog articles. So when we have e-commerce clients uh their product images and things like that, we we feed that into the image module and then we can rename and and do things like that. Definitely. Cool. And can you um can you show like how h like how detailed this goes into understanding a business? So we can look at the so it for you I recommend talking to a AI agent about your business when you're doing something like the but I'm doing so brand I can I can basically talk to I can I can tell questions um and it will figure things out better than me. Okay. So it will research the the profile of my my buyers and things like that. So you can talk to it and it will then fill out these sections much better than I would be able to. So you have and then when you're because Claude is very good at taking brand identity and voice and tone and breaking them down, you can actually do this like this as well. Um so you have your brand story. Yeah. uh positioning, target audience, what you sell. Then you have like trust signals and proof themes. And it just starts filling all of these out. And you know, also have things like proof like what are your case studies? It it just takes everything that you can find and builds it into the project. And when I'm talking to the agent about this, it also fills out the project brain. So this is uh a layer that's you know when you're talking to it and you say oh this isn't correct or we want to talk about this topic or it it will fill that out as a knowledge. So it gives it a confidence score. Um and things like this is an issue, this is working, this is research, this is a preference, this is an actual strategy that I'm going to apply, this is insight, and it will just fill this out. And when I'm talking to the agents that are within the system, for example, if I go to the keyword universe, I can open this up and we can say something like I need more keywords about Yeah. something like keywords from href about AGNC SEO and it will you know help me find those keywords look at different you know it will look at the brand DNA and try and find more keywords that are missing from the unit. Wow. Wow. Yeah. Here we go. So, discovery summary fetss from href sue agency like it will just not everything is is perfect especially when I'm fetching from href. It's it's better when I just have the AI workflow go but we now have 79 new keywords added in here that we can then analyze. So, I can go here and we can sort it like this is a Yeah. So and then it will just go and find the volume check the relevance and you know it just loops through increase the these clusters and then you can analyze like is this cluster relevant to my business or not and you can you know um build the page optimize and you can also run things like striking distance um find information gain and things like that so I think it's very good but how are you finding how are you finding information gain um yeah when we have when we have keywords that sit in the top 20. We create something called I can just show you here. Hold on. It creates I'm also I'm also I'd like to hear about how you scrape forums. Uh and then yeah, I mean you talked about how you analyze the product service data, but scraping forums is really interesting too. Yeah. So that's more about like finding like pain points within the articles. So it will take scrape the forums and obviously add that to keyword universe and you can do the same with things like uh AI SEO when you're trying to find like prompts and things like that it it gets added as a topic inside a cluster. So we have uh yeah for example here let's take a look at so we run this vitamins. Yeah. So we run this as a it's it's ranking like 13 or 14 or something and it will take and analyze the top 10. It will find gaps basically like what topics are not covered, what topics are like engineering. Uh it will just run all of these different things and find and create this research brief. So, and we'll just tell it like this pa solid structural bones product picks and then just it goes into what's missing like no buying guide section how to do section is missing no FAQ section and it will run through all of these things and let me just go back here and then it will fetch SEO intel rules so basically find entities that and then it will update the content from a a clean source. It scrapes it and then creates a new brief new article with all of this information and yeah, it it creates a pretty good article I think from that. Yeah, that's sick. Um, so this is Yeah, I wanted to see the screen. So this is so you so what what does it start with? If you if you scroll to the left, if you go to the left Yeah. So you have the backlog. There we go. And so this is so this is basically to-do. Why doesn't it just run on its own? Uh so these are items that are we don't have a workflow for them. I was just playing around. So they're just stuck. And when I create manual tasks for my team, it will go into backlog. And then someone will just basically take them and they will go into progress. And if I want to create just hold on. If I want to create a switch task I can just create an article from just writing out something just best SEO agency that will go let's just have a buying guide uh that will get added here as as cued right and then how long will it take to to do all the research and to write the to write the content? Yeah, it can take like from 20 minutes. Oh man. Yeah, up to like an hour. Depends on how how much content is cued in the in the system. Um the this agent here, he just basically the the project manager, he'll send it into progress and you can see now uh resources starting. So states is the resources you're working on task 1639 and that's basically these are what we call stateless su like stateless agents they don't have a uh like open claw is stateful agent like he has memory um context he has like a mission but these are just agents that wake up do a task and then they go back to sleep so it's a basically a pipeline that runs Can we see Can we go through a um one of the needs review ones or is that like too sensitive? These are just everything here is just tests data. I'm just Oh, great. Yeah. So, could So, like I I want to know like what it looks like when you're reviewing one of these pieces and then what um what makes it into the next stage and then what gets blocked. Yeah. So we have obviously different workflows for different uh projects like this one is just very very uh normal. It just writes it sends to review and I can just open this article up in the editor and it has a editor that shows the content and I can tag it with a team member that's going to go in and review it. Wow. And yeah and then we can just hit approve or we can uh send it for further review like we can do like I mean I can do a a comment on it and say like I'm not happy with this. Yeah. So they will basically go and and fix it add images and and it is just goes through the pipeline very fast and you can just switch uh you can switch topics here on the left if you wanted to. How much time do you think this is saving your team? Yeah, a lot. Like it's insane. Um, but everything like having everything in one place like you have all of the HF data, you have all of the keywords, you have all the clustering, you have all of the content, you have all the Google search console, you have everything in one place and images and it just saves so much time to be able to push all of this content through the system very fast and you don't have to think about like, oh, I have to go and do spend two hours doing like all of this research. Agents are really good at doing research. They're doing very good at finding gaps. They're very good at, you know, finding information gain, uh, finding missing entities. And so your work is actually to take that and just, you know, make sure it's good. You don't have to go and spend all of this time doing the research yourself. And yeah, but I can go back here. Yeah, if it's not frozen because Yeah, I'm pushing a lot of updates, so it tends to But yeah, so what are you what are you wait what are you trying to show us? I was going to go back to mission control. There we go. Yeah, I was going to show you the So we have this one is now building out. I can go into deliverables and we can see uh this is just a brief of what the resource is is looking at. So they're looking at all of these different pages is uh is is analyzing the content. It's analyzing the entities. it's looking at and giving the next uh the writing agent all of this information when they write the article. So So you had to design all of these workflows yourself. Yeah. So yeah, basically from scratch and not all of them are ready, but we have like 12 workflows that are running through pretty good. Um yeah. And then can you talk about the writing workflows? This is something that I think trips a lot of people up and they turn out really terrible AI content. So what makes your AI content better? First of all, we have a review stage. So we're not just having it write it where we're actually putting it into the hands of a human. Um number two, we we have a a pipeline. So it goes and takes all of this research and we then when we are not happy with the content we go in and we update the system and we end up with content that we are happy with for that specific page type basically. So uh in the beginning I was not happy with the buying guides. Uh they had a lot of AI things in them. So we're iterating we're talking to uh the system telling them like how we want it to done and so it updates the workflows but also specifically for each project we have the the brain you know that I showed before where it's like a preference it stores that so this project does not want this specific type of of you know writing or or structure and then they listen to that and Uh, so this is internal for your team only. Would you ever open up something like this for other people to use? Yeah, I might in the future. I don't know if if there's enough demand or something and like isn't the the system isn't uh I wouldn't say it's ready just yet. There's a lot of things that I need to fix before I would open it up. But uh I would definitely be open to allowing a few people I trust to play around so they can give me feedback. Uh but you know actually another interesting question is for people who want to make uh for SEOs and marketers who want to make something like this themselves. What are what were the biggest roadblocks when you started or the things that you spent the most time on iterating and the the biggest lessons that you had making this? Yeah. So the first thing was just to learn how to actually write code with with agents. I don't know how like I've never learned how to code. is not one of the things I know how to do. And I was playing around with with open claw, you know, claw claw uh the the open claw agent, you know that? Yeah. Okay. So, it's an agent basically that you install on your computer that has like it has memory, has context, and you can give it like a mission and it's like very proactive. It will use your computer, will browse like it didn't delete all your emails or anything? No. So, I bought a new computer. I set up a new email and everything. I did. Okay. Yeah. Great. You know what you're doing. Yeah. Yeah. I didn't have it messing around in my my work computer, but what it like it started building tools. I was like blown away what it was was creating. And uh the first thing it created was actually this mission control. And it like started building out sub agents. I was like, okay, this is pretty incredible. So, but I figured out that it isn't actually very good at writing good code. like it will not uh review the code like things started to break basically. So I started building out a system where I write the code but I get the ideas and things from like talking to the agents and they helped me just organize stay on top of things. So error handling uh is like critical you if you want to go and and and wipe code something like this you have to have a system where it's just everything that you upload goes through a phase where it's just is gated. some of it needs to go back like and fix this. So that was the first thing I had to learn. H I created this platform like four times before learning that and it just broke down every time I tried to like scale something or like add a feature. So that was like the biggest thing was to learn how to do that and yeah I recommend learning that first. Then it was actually right like creating the the content pipeline was like very very difficult to get everything to flow from one stage to the next. So it doesn't break. Um you know you have an agent that does this. It hands work over to this agent but this agent has to use the entities in the right way. Sometimes the the scraped entities are not correct and that agent has to realize that. So it's just a little bit back and forth. Um so we have spent a lot of time on this you know the content system and then is the the all the keywords and clustering has been very difficult to get right because when you're auto mapping you're auto creating topics you have to have a way for the agents to understand you know like these are the pages that you have these are the intents um and like get everything right there was very difficult but you know we we were very close to having that pretty uh pretty automatically right now. Yeah. And you you also said so um you mentioned that the keyword universe has keywords flowing in via five different sources automatically. So it's hrefs. It's it's what like forums, Google search console data. Yeah. Uh it is. So we create the first thing we do is we when we add a project is we create and feed all of the the the topics. So it scrapes your store or or your your your site and it then feeds all of the search console data into this place. So and that means you can try out uh errors, you can look at performance data, but you can also have fixed topics on a page. So this page is this topic. This page is this topic. This is the one we created before. Um yeah. So it it takes everything that's on these pages and it creates a vector database. Okay. So vector database is it takes the content and turns it into numerical values. So instead of having words, you just have numbers and that's uh like LLMs are very good at interpreting that data and it can then guess what the topic is very easily. So uh it will take my current page and it will guess the seat. How how hard how hard was it to turn to have something that turns this into a vector database like for you to do that? It was very easy like uh um the coding agent can do that quite easily. It's it's not difficult when you have all of the other uh like pipelines in place. You have all of the like you have created your data warehouse. You have created all the the pipelines. is very easy to that function was very easy. Um yeah, so it just basically will use a cosine similarity function to like find what it thinks should that page is about and it will see that into the universe and that is very good when you have products service pages because you will get uh you know you'll get these commercial intent keywords into the universe that helps you then you know seed out pages that are about that and yeah so that's that would be be be one uh the number let me go back to the universe. So number two is just the agents looking at the keywords that you have that have high relevant score and if I click this it will go and add more but the agent will do this on a like a cron uh like on a schedule it will just go and add more keywords that are relevant and that will then go through the same pipeline. check the relevance, check check the SER, check the volume and then cluster it and then yeah, if you scrape forums, we we use href data for SEO and yeah, basically everything that we can to get more volume into the the keyword universe. So we can have like a a broad keyword database that we then narrow down with with the relevant score and the the clustering. And so you you said this has saved your team a ton of time. Has it also made any sort of difference on results? Yeah, it definitely has. Um, we can now very quickly get these, like I showed you before, the striking distance report, you know, when you have a report like that on a page, you can very quickly go and update it and it will rank much faster into the top 10 when you go and apply these changes. They're very good, these changes. Um, you know, I also just all of these tools now they love like fresh content and just by being able to go in and quickly update content, you have a lot of fresh content going and and we have seen that across multiple project that that's really really um helps the sites get more traffic quite fast. I think let's go. This is this is great. Thank you for showing this to us. Uh are before we go, are there any other any other awesome things that you should show us? I I think uh everything I showed you today is is pretty pretty cool and that's the the you know this is central things of this tool. I'm also building out things like um you know how I have a domain finder. I don't want to show it but it will basically find we'll use your brand DNA to find available domains for your business if you want to buy them. uh so you can find like brandable domains and every time I get an idea I'll just build it and add it to the tool basically. So um yeah then we building out things for the agency to track work and things like that that I don't think is relevant right now but yeah I think I think this is the the main thing is the keyword universe and the the command center and mission control to like have everything just flow through. Thank you so much Trickvy. Um, all of your socials and your contact will be in the description for this episode like it was last time. And uh, dude, you are you are great. And you're welcome back on any time to to show what you're working on. This is so this is so sick. Thank you for having me. Love being here. Uh, great to to show this. And if people have questions about how to get started building things or or SEO, just go ahead. You can find me on Twitter. This is episode 1,027 of the Edward Show. 1,027 days in a row doing this podcast. No days missed. If you watch us on YouTube, thank you so much for watching. You probably didn't listen on Spotify or Apple Podcast because this was a screen shareheavy episode, but if you did, thank you so much for listening and I will talk to you again tomorrow. Bye now.