Claude SEO: Make Your Own FREE SERP Scraper (steal this)

Today, I'm going to be answering the question whether you can do free work with Claude code and also Claude code SEO. I'm going to show you how to make a scraper. I'm going to try and only use the free plan of Claude code, but I may have to upgrade if the usage is like nothing. I'm going to be using Sonic 4.6, which you might think is completely crazy, but to be honest with you, I don't I'd rather use Sonic 4.6 for this. It's not complicated task, and I believe we can do it. So, if you don't know, I'm not going to use any usage, but Claude code is capable of doing web searches for you. Oh, damn, they removed Claude code from the free plan. I did not know that. Okay, so we'll choose the chat. I don't care. We don't have to use Claude code. Okay, so let me just think about how to do this. So, if you didn't know, if you do like a web search for like a keyword that you're trying to rank for, right? So, let's just say hot tub stays island. Can you do a search for this online and give me the top five ranked sources for this keyword. So, remember, this is totally free, new account. Let's just see how this goes. I'm curious to see what kind of usage you get. It's very slow, I have to say. So, look, we can get all of these five things right here. These are um you can actually see that these are not the best um SERP for my business because I am not like a blog. These are a lot of these are blogs or you know, Trip Advisor, um Airbnbs, and then this is actually just Airbnb itself. So, uh this is pretty interesting to note. So, this is like the query, like what what is the intent behind the query? So, Google shows these five, right? Let's just search hot tub stays island quickly on Google. Yeah, you can see pretty much these are the five that it came out with this being the one that it didn't come out with. This is a very very interesting one right here. Uh this is one that we could we could take a look at and you know, maybe get their content. So, how does this work? I want to be able to scrape all of the pages that are relevant to my business and see how they are ranking. So, like this page here, this is a page that we could technically um you know, use the SEO of, right? So, that comes to our second piece of the puzzle, which is actually going to be Bright Data. The way this works is this can scrape the actual HTML of a web page, so we can get their exact statistics, right? There is a link in the description of the video. Thank you so much to Bright Data for sponsoring the channel. You're really really helping me out allowing me to make the content that I want to make. So, shout out to Bright Data. So, I'm just going to press get started for free. I'm going to do a new account. So, we'll just do a new account with the same email as the new Cloud account. So, you can see here it says free tier activated. So, perfect. So, we get $5 free for 30 days if we add a payment method. You will not be charged. But, we do already have the Web Unlocker API and it looks like we have 5,000 free credits. So, all I want to do is I do want to use the Web Unlocker API. So, I should just be able to come here and then continue and then just make a new Web Unlocker. Continue. So, it does look like I need a payment method. Okay, so we basically I should be able to copy this right here and this should now work. Right, we have the bearer down here, which is our API token. So, I'm just going to say make a scraper using Bright Data. By the way, guys, there's a link in the description for Bright Data. Please use my link. It shows them that we are a good channel to sponsor. If you do want to support the channel, that is the best way to do that. So, I'm just going to put this, and then I'm just going to put this, right? And then I'm going to say, "Use the scraper to scrape relevant websites. So, not like Airbnb or the Irish road trip or other large businesses, but small businesses that are actually offering the service directly, like, I don't know. What was that one that we had here? This one here, like this Uh do an example. And then generate a report of the SEO of the top-ranking pages of websites that actually offer the service. Okay? Okay, so it's asked some very basic questions right here. Again, this is pretty nice just to be able to use for free. I wonder what the limits are here, because Sonic 4.6 is not that bad of a model, like, literally. I I didn't know that you could do this for free, guys. This is crazy. It is kind of annoying that you can't use Claude code. That's But for 20 bucks a month, definitely worth it, I would say. Guys, I actually didn't know that this was possible, because obviously I've got a paid account. Okay, so it looks like I have to turn this on here, allow network egress. Try it again now. This should work now. Um So, yeah, this is super interesting. If If you've never used Claude before, like, this is completely free. It looks like it can run code. Okay, so that didn't work. Okay, so kind of guerrilla warfaring this a little bit, right? I'm doing this um I'm doing this in a very specific way. I'm just going to say, "Please please the actual API. Yeah, it's okay. I'll re- all of it after. Don't tell Claude I'm not actually going to re- all of it, boys. Ah, come on. God's sake. Upgrade to keep chatting. No way. 5-hour message limit. Crazy. So, we did technically get some useful information out of this completely for free. So, like I can kill this now. Um I would need to set your API key, which I can do pretty easily. So, let's just do that. Let's try this and then go here. And then copy this, put that here. This is quite fun, actually. And then we go terminal on the Mac and then we do this and we paste. That should unlock the There we go. Bang. Nice. So, now we can't send the rest of this into Claude, unfortunately, because I have hit my 5-hour limit. I don't think that was too bad, honestly, considering I didn't pay them a penny. That is not too bad. I would probably recommend doing the 20 bucks a month one. I might just do that. Um but like just so you guys know, you can do a lot of this for free, right? So, like I'll let me let me just show you how you can finish this for free, right? We go to private window here. We go to ChatGPT. And then say, um "Extract all the SEO from this HTML. Um meta title, meta description, title, everything." Right? And then just paste all that. Let's see if this works. Should work. There we go. The title appears to be truncated. That's fine. That does happen. Um Yeah, also meta description is truncated. We can see all of the other stuff here. So, the H1, H2s, H3s, breadcrumbs, structured data, etc. So, we can really start to get an idea of how these guys are ranking, right? Now, I would almost certainly recommend upgrading to Claude Code Pro, right? I might What I might do is I'll continue this video uh tomorrow, most likely, right? Where I will upgrade my plan. But, for now, I just wanted to see what was actually possible completely for free. So, I haven't paid for anything. I'm using Bright Data's free tokens, right? Which they give us a pretty good amount of free tokens, all things considered. Let's see how many tokens I actually used. So, if I refresh this, what's this on now? There's no way it's on 4,999, right? It didn't even go down. Weird. Okay, I'm not sure why that would be the case. I definitely used it just there because it gave me the unlock. That's pretty interesting. Yeah, you can see there's an event here. What the hell? Why did that not cost me anything? Weird. Weird. Weird. Weird. So, yeah, basically, guys, all you do is you just do each of these one by one, right? Get the info, and then, you know, you put it back into either Claude Code, in this case, if you've got the pro plan, the $20 plan. I am tempted to just quickly get the $20 plan just to continue this video. Um I don't know for sure what I'm going to do. I'm just uh showing you guys what's possible, right? So, this is again, this is the HTML of another page that is ranking top of Google for your search term, right? And then, if you think about it, you can very easily, with Claude Code, which is what I was going to show you today, make a really, really good scraping system. I might just log out, actually, and log into my paid one just to show you how I would finish this. So, yeah, just going back to Bright Data, guys, I actually use Bright Data every single day. It's one of the key building blocks of Harbor. We use Bright Data all the time. Basically, what it allows you to do is if if you do a basic web fetch inside Claude code, right? Oh, they put That's cool. They put Claude design in here. If if you do a web fetch, right? Do If I just do this, um, do a web fetch. Do a web fetch of this page and find all the metadata. It doesn't actually look at the HTML. It will make something up here. It will tell me whatever I want. Oh, it can see the metadata, but it can't see the like JSON schema, etc., right? From a blank web fetch. If there's Cloudflare on the website, it also cannot do it. Just generally, just full stop. It won't be able to even get onto the website, right? So, what BrightData does is it breaks down those pages into code, right? So, this one did fetch, which is fine. But, if there's Cloudflare here, it will not fetch, right? So, if you need a universal scraping system, the best way to do it is BrightData, by far. Okay, so I'm inside Claude code. I'm on my paid one. Just This is what I wanted to show you guys, right? So, make me a system, uh, skill you could even do. Make me a skill. I'll just do a system for now. I would recommend doing a skill, but it's just going to take too long. So, make me a system that, right? Uh, I give you a keyword. You do a Google search of the keyword. You find the, uh, plus a website. This is a really good system, actually. I'm going to put this into Harbor. You find the, um, top top ranking pages that, uh, directly that are sites directly comparable to my website I gave you. And then, you use BrightData to ex- to extract all the HTML of that page, you synthesize that. This is a really This is a really cool system. You synthesize a report of how to rank for that keyword according to search intent plus HTML you find. Right? And then I give it the Bright Data thing, and then I give it um my API key, which by the way is just a free Bright Data account. It's literally pointless doing anything with it. Okay, so just watch what I'm saying. This actually does this when you're a paying customer of Claude using their number one flagship thing, Claude Code. God damn, I wish Claude would have sponsored my channel the amount of times I've talked about Claude. The reason I use Claude so much, guys, is cuz I just love it and Like it it just works for me. Claude Code was the number one. It always has been for me personally. Like also just one more quick thing. A lot of people, you know, they might be annoyed at me taking sponsors, whatever. I just want people to know that the main criteria for me taking a sponsor is that I I already use or I would use the sponsor. I have used Bright Data, as you can see here. You know, we've processed 1.3 GB of data in a single day before. So like I'm not pissing around. The reason I took them on as a sponsor is because I used them for Harbor to get round Cloudflare. Harbor is a universal scraping system. I needed to get past Cloudflare. Everything else didn't work. Bright Data, it worked, and that's why I use it. And like I showed you in this video, you can get started completely for free if you use my link in the description or in the pinned comment. Okay, so this is pretty much done. You might notice I'm on Opus low. Like this is something that I've been trying to get into, and I would recommend if you're using a pro plan, which is $20, which I will show you tomorrow. I'll do what I'm doing now on a pro plan because I'm curious to see what happens, right? But basically, Opus 4.8 with low thinking lasts so long. I'm not even nearly near my weekly usage anymore. Normally, I'm like just teetering on the border of going over it, but right now, like I'm nowhere near it. So, look, it's now doing a best running shoes for flat feet, and then it gives us a URL. Um which it just needed to be a website, not a full blog URL, and that's fine. So, it only finds things, hopefully, that are relevant to brooksrunning.com. And it looks like this is what it came out with. So, comparable competitors selected, extracting URL from bright data, fetching your page, building report. So, there it should be a report here. Let's have a look. Okay. I didn't want There we go. No. Okay, so right now I made a system that does not use Claude code. It uses Anthropic, right? So, it still it managed to synthesize this report right here. Stretcher data, the winners use, list item, news article, image object, web page. So, this is the kind of report that you can send to you know, Claude code or whatever to then create an an article or a page for you using these exact SEO points, right? That's the point of what we're doing. So, now, instead of just doing, you know, random SEO, we're doing data-driven SEO. With all that being said, guys, I'm going to leave that this video here. I'll do one tomorrow where I do this with Opus 4.8 on low using only the $20 plan. Today was trying to see what we could do for free, and we got pretty far completely for free, but obviously, as soon as we jumped on over to the paid subscription, it was just so much better. And I'll show you how to make a skill that does exactly this for 20 bucks a month. I'm going to leave the video there. Thanks so much for watching. If you are watching all the way to the to the end of the video, you are a true legend. Go and check out Bright Data, use my link, support the channel, and I'll see you very very soon with some more content. Peace out.