54% AI-Generated and Climbing — State of AI

Welcome to Syntax. Today, we're talking about the state of web dev AI, the state of AI results. Um there was a survey that was just completed about 10 days ago. This is from Sasha Grief. He runs the state of web dev API, the state of uh like JavaScript, the state of CSS. You know, all of these things that we do. And he's been running one for 2 years called the state of AI. And it's actually kind of interesting because it gives you I think it gives us a pretty good outlook as to like what are developers feeling, what are they actually using right now, how much of their job are they using it in, and we're going to go through some of the best parts and sort of take a look at like what are people using, what what models are they using, how much of the code are they doing, all of that good stuff. My name is Wes. We're here with Mr. Talinski. How you doing, Scott? Hey, I'm doing good. Doing good. Doing good. Uh drought conditions in Denver, and we're getting a ton of rain right now. So, I'm loving having it. Great time. Enjoying the lovely rain. Uh have you ever been to Hawaii, Wes? No. No, I have not. You must go. Oh, why you simply must go. Uh that was my favorite part about Hawaii was just that it rains all the dang time. I love that. Or at least in Oahu it does. So, yeah, big big fan of the rain. Got to love the rain, and I moved to a place that does not rain very often. So, uh got to love that. Real quick, Amsterdam. We are going to be at Amsterdam for a Syntax meetup on June 10th. This is going to be a part of the opening party for JS Nation and React Summit. These are awesome conferences that are happening on June 11 and June 12th. And you don't have to get a ticket to these conferences to come to the Syntax meetup. Um you can simply just go to syntax.fm/meetup and grab a free ticket for that. But, we highly highly recommend that you also come to the conferences. Some of the the brightest minds in the industry are going to be at this thing. So, check it out. jsnation.com react summit. You can get a combo ticket use the coupon code syntax for 15% off. Again, that's syntax.fm for slash meet up for tickets to the syntax meet up and then also you go to jsnation.com and react summit.com to grab tickets to the conference. We'll see you there. Peace. Well, let's talk about rain on some people's parades. We're going to be talking about the state of AI in web dev. I know there are some people who are still holding out on the fact that AI coding sucks and that it's never going to be any good, but from the results of this survey AI coding has had a massive jump in terms of people who use it. And I think we were speculating a little bit beforehand obviously that it's a little bit of becoming more normalized, but also it's gotten better over the past year obviously. But I I think people maybe are a little less eager to just say F that I'm not doing any AI coding at all. As you can see the percentage change here has been pretty massive. Yeah, I think there's there's a mix of people realizing oh, it actually is is kind of handy. I can see that by if I post stuff on Blue Sky I don't get absolutely annihilated. Yeah. See, I've noticed that too. Yeah. I like I I don't want to be the like like a hype master. I feel like I have a pretty pretty good balance take on this type of stuff and it's certainly not a silver bullet, but I I don't think I think this is a fairly monumental shift in our industry and I don't think it's going away. So, it I think in the last maybe 6 months or so we've seen a lot of the people who have been very against it to say oh, this stuff is actually pretty helpful or oh, it it actually is is much better at at what I want to do right now. So, there's certainly people on the other end of the spectrum which are full psychosis and shipping all kinds of slop, but there's I think there's a lines a day, baby. Yeah, [laughter] yeah, for sure. I I I I Yeah. I I think it's interesting here. We see a big jump where 18% of respondents to this, they have written It seems like 75% of their code they write has been written with AI. And that's a big jump from the year prior than that. And and even higher than that is 19% of folks write let's say 85 to 90% of their code with AI. And and and granted, the this survey, the results are going to be skewed towards people who are writing code with AI, right? It's the state of AI. And it's also going to be skewed towards the people who have responded to this survey. So, again, that's 1,200 people, which in the vast world of our industry is a small small bit. But again, this is a sampling, right? And what would you say you're at right now? Because if you were to ask me, I just feel like I I like I don't know that I can put a number on that because it's just it's just like part of my workflow of Yeah. It it's not like I was like, oh, now I'm going to go to this other place and only write with AI. And and now I'm going back to my notepad Notepad ++ and and hand coding. It's it's just like kind of a back and forth. And sometimes you're jumping in and you're you're you're factoring something, you're reviewing something. It's it's I don't I don't know where I would be at this. Probably at like a 76 if I were were to guess, but like it's it's similar to like how much of your code is written with tab completion a couple years ago. It's it's a kind of a hard thing to to sort of guess there. Well, mine is mine is closer to 90% West strictly because AI loves to write a [ __ ] ton of code when it does not need to. [laughter] It's like simply by volume of code? By volume of code. It's it's just like pouring it in there even if it doesn't need No, I I don't I don't know if if we're talking about like effectively, I'm not quite sure cuz I did just like fully hand code a project the other day from scratch and it was great. I got a lot done, but again it was probably more efficient than what the AI is writing. I I would say mine has definitely obviously increased from the year prior to this. Mhm. So just like many of these things I I think that number is just going to keep going up. And likewise in terms of refactoring, 21% of respondents said that they're constantly refactoring their code with AI, which is a 10% jump from the last year. So yeah, that's a big old jump right there. I think we're going to see big old jumps on all this stuff from the sentiment in general from year to year. Now this is only the second year they've done this survey, so there isn't a ton of historic data, and I do have some critiques about the survey. Love Sasha. I do think there's a number of things that would be very interesting that this survey is missing, which we'll get into those. Let's talk about models and providers. So this includes both experience and sentiment, whether you've heard of it, used it, never heard of it, blah blah blah. The first one being the most people have heard of and used it is ChatGPT. And I think this encompasses all OpenAI models um the way that this is listed. Yeah, it's weird that they label it ChatGPT and I I guess like some of the the most recent uh OpenAI models are called GPT. It And we got to do a better job at naming these models, you know? It's it's such a confusing thing. But I I also do think there's quite a few people that are still just typing into ChatGPT, copy-pasting the code and out in and out of their thing. I I don't think it's a large amount, but I think it's quite a few people doing it still. Totally. Yeah, I I think there is. And that's honestly it's a it's an it's an it's a less scary introduction to coding with AI cuz you can say, is this you know, I just have this quick little question. Okay, I got a little snippet. Okay, it works or it doesn't work, whatever. I can I can eyeball it from there. And that is like a big jump between that and then you have the in-editor tab completion stuff. And then you get into the world of just like, "Hey chat, make me an app." You know, I mean there there there's different levels to this in terms of what you're using. But as far as models go, people have uh used it and negative sentiment is 10% for chat GPT, which I I'm just going to chalk this up to all GPT models. Where Claude is used it and has negative is only 2%. So a 10% negative versus Claude has a 2%. And likewise, Claude also has that 46% positive. So people And this is this I'm going to say this mirrors my experience as well. I I In fact, when we did this, how how much do you rage at AI? I raged at Claude considerably less, which I feel like has to tell you something. If if you scroll down underneath this, there is out of the models mentioned, which of these does your company actually pay or do you or your company actually pay for? I think this is probably a better outlook of like with these like I've heard of it and have positive sentiment. I think what happens is that people have like, yes, all of these people have clearly heard of chat GPT and Copilot, but like like what are people actually using right now? I think that this looks a little bit gives you a better look at it, right? Like 69, nice, percent using Claude. Chat GPT. This is the worst Does this just say Open AI, right? From 50% people at Open AI, 32% Gemini, 6% Copilot. So, you're telling me that 32% of people are paying for Gemini, but only 6% are paying for Copilot? That doesn't add up to me, don't you think? Well, it doesn't add up because Copilot is listed as an agent and assistant. So, Okay. are a write-in where people are like, I use Copilot as my model. Like Like, they don't know what model they're using. They're just using I'm using Copilot. Yeah. I don't think people actually know what any of this stuff is. I had I posted something the other day where I reverse engineered my lights with Claude, and the amount of comments I had from people saying, "You could just use local models." And I said, "Which model could I use to do this?" And they're all like, "Copilot." Or using They were essentially just naming like free trials, which they have some For some reason think that that's just like a local model. Like, I think people are a lot more confused of as to the how this stuff actually works. Yeah. [laughter] I I think that is evident if you look at this data further because you'll see that under models, um, Microsoft Copilot also heavy write-in. And then Kiro is a heavy write-in. Kiro as a model. So, yeah. Copilot doesn't even have a model yet. I I bet they'll probably be releasing one in the next month or so. Like, they they had this big thing couple months ago where they're like, "We now can train on your They had like a big banner at the top of GitHub being like, "Opt out if you don't want us to train on your models." I bet they're going to roll out their own version pretty soon. But, they don't as of recording, they don't have their own model. I think I think this does illustrate something interesting beyond the data here is that people don't know what a model is. No. Yeah. The app is the model, I guess. The app is the model. The files are in the computer. Yes. computer. Let's talk about agents then because I think this leads in nicely to that where again Copilot seems to be the most used but not the most loved where it has a 13% sentiment I a negative sentiment. I think one of the most interesting things in this is a little bit of bubble breaking. You know, we hear about things like Devin on Twitter all the time. And according to respondents of the survey, most people have not heard of it. 42% of people have not heard of it. And the people who have heard of it don't like it. 14% of it have a negative sentiment about it where only 2% have heard of it and have a positive sentiment and only 0.5% have used it and like it. So, Devin is not either either is outside of this bubble or people who have actually tried it aren't aren't a fan. Oh yeah. I think there's going to be two things going forward with like what do people use this? First, is it included in your whatever you're paying for, your Microsoft 365, your Google Workspace. Like I think that those guys are going to have a huge leg up cuz it simply is just part of the package and then they pay extra for it and it's integrated with absolutely everything. They don't go looking for another tool. And I think that's that's partially why Claude and and Anthropic right now are are gunning so hard for like the like office thing because those enterprise contracts are are massive. Yeah, I mean it's evident here by Copilot being number one in terms of the most experience and then Claude Code being number two in the most experience. Crazy enough, uh 34% positive uh sentiment from Claude Code over the 21 positive on Copilot and then Codex OpenAI Codex is only 16% of those who who have used it like it. Uh what do you Yeah. What's wild to me is that the cursor is not even in this list anywhere. Um yeah, Tab nine is in here? I I didn't even know that was a still a still a thing, you know? And I I have thoughts about this, Wes, and and we'll talk about that, too. I I have I have thoughts about that because if you get into the other coding agents, you'll see Open Code was a write-in for 389 people. Open Code is very highly used from people, but It's they have like 10 million something like that monthly or daily active users monthly active users. It's insane how many people are using it. a Cursor a write-in, as well, and Pi a write-in. I think that Sasha needs to I think maybe grow with the industry here and that Open Code, Cursor, Pi, uh Kiro, all of these things need to be in the survey because yeah, I even Pi has a decent usage. It's It's nowhere near these other bad big boys like Cloud Code, but Open Code is a major player and Cursor is a major player. It's a He's a He's a player player. Yeah. And and maybe I think maybe that's part of of maybe Cursor's problem is that people don't see it as a a coding agent. Um they see it as an an editor. And and that's why Cursor's been pushing so hard on like moving away from their like their IDE into like their like agent interface, which absolutely everybody is doing, you know, you got your projects on the left, you got your chat in the middle, you know, that's the same same UI everybody is building right now, but I think that's that's part of their problem is people see them as only an IDE when in reality it's like Cursor's actually a pretty good deal and you still get access to you get access to all the models, you get to try them all, which is pretty cool. Yeah. I've been personally Wes uh what this is a good this is a good time for us to talk about it while we're using. What what's your current usage of choice? Do you know mine is changing all the time. Um I'm probably 50/50 split between Claude and Cursor. And code or Claude the app? No, Claude Claude code. Um and then the other split is is probably Cursor. I do dip into Codex fairly often as well because I have a chat GPT subscription. Um and just kind of get a just get a feel for for how it works. And then I do use the Codex models via Cursor probably more often. Cuz I still I I know that nobody likes having an IDE anymore, but I still find it nice to be able to like click on something and see the code uh that that's being generated. I still like that UI the best. So, that's kind of my my go-to. But then I also like the like the terminal experience quite a bit still. Um as much as I say I don't like TUIs for just quick stuff needing to to I don't like like FFmpeg resize a whole bunch of stuff or download a whole bunch of videos or um upgrade and install, you know, I just Claude dangerously skip permissions and I I find that that's one of my favorites as well. Yeah, I've been using I've been using uh Claude code for like almost everything these days and and that's like in the past couple of weeks I've like really landed on Claude code to the point where I did cancel my uh chat GPT subscription. I find Codex to be just an absolute slog to work in. And I know some people really like Codex because it's like more uh exact but I just I I just did not like any of the results of the front-end code it was writing ever. And and like to me that was such a thing that I just was so annoyed with. Like, it was hot potatoing data from function to function to function to function. It was writing so much unnecessary code that I was just getting annoyed with it. And I'm I don't have that same experience with with Claude. And and I feel like maybe my Claude code setup is a little bit better. And then I'll also dip into uh I have a a a Copilot subscription. So, I'll use Copilot in Open Code now and in Py. I'll use uh Copilot models in both of those. And I'll use Py if I want to have something just run on my system at large. And I'm pretty much using Claude Code only on my like in my projects. And then like if I want to say, "Hey, uh go to my NAS and delete 100 files." I'm going to do that in Py. So, uh I just because I have I have access to those Copilot models in Py is pretty permissive on that kind of stuff. It it's like where Claude Code is going to be like, "Do you want me to read the file that you told me to read, please? Like, can I read it?" And I'm just like, "Shut up and read it." I threw Claude on my Synology, and like the one thing that I hate doing is updating Docker images. Same. Oh, same. I know. Jellyfin on a Docker image, and I hate for some reason I every time I need to update that thing, I'm just like, "Oh, what do I do?" And like cuz there's there's data that's been like mapped. Obviously, you with Docker, you can map things and you don't keep the files in the Docker image, you keep them on your file system and you you sort of map them there. But, so many times I've accidentally nuked all of my like library data. Um and I've had backups, but it's annoying. And now I just do do do do tell Claude Code to update the thing. And and it rips on it. And I I know that probably makes a lot of people mad. That every time I post about that, they're like, "Just use X Y and Z." I'm like, I I hate touch this thing like twice a year and I always forget what to do and now I just type in the box and it it works great. Yeah, I have that same thing with image. Image I run on my NAS and like, I don't know, the updating process for it absolutely sucks. Uh so, because they don't have Synology they don't have like an official Synology thing. So, it's like you've got to duplicate the container and do the other stuff. I don't want to do all of that. Uh I It's funny that you run Claude Code on your NAS. Where what I do is I just have my local agent on my laptop have SSH. Is Is that better? I have no I have no I have no I have no idea. Is there any downside to that? Like, what do you just tell it to go SSH in and I I I just have all my stuff set up, my key set up, and whatever. Everything's on my uh Tailscale network and it's just like, "Oh, I'll just SSH into it." Yeah, and it doesn't It's not like it the password is getting passed into Um yeah. Interesting. Maybe I need that as well. But, I'll give it a shot. Cool. Who knows? Okay. Let's get back to the survey here. So, let's look at paid agent usage, what people are actually paying for in terms of agents. We talked about models and paid usage. 58% of the people who responded this are paying for Claude Code, whether that is through a Max plan or through tokens. We have 42% are paying for Copilot and they're about to be paying more sense. Whole lot more. Did you see that like so, they have like a Copilot has like a new dashboard that shows you like how much you would be paying if you were actually paying per token? And CJ CJ was telling me that his his cost was outrageous in terms of like he thinks he's he thinks people are in for a major rude awakening in terms of because it was like 60,000 bucks or something crazy. Seriously? That's insane. And I saw Peter Steinberger the creator of open claw. He had burned a million and a half dollars or the tokens in a month or so nuts like that. Like That guy's And crazy. And obviously that guy that guy is trying to figure out what coding would look like if there were no there was no like cost. Yeah. Yeah. Yeah. And like whether the cost for that will ever come down like it would it would have to come like even at $60,000, you know? Like even if that were to come down 60X. 60,000 bucks a month. That's a lot of money. That's a lot of dough. I know. Um for the things 24% are paying for open AI codex, 5% are paying for jet brains. And then yeah the cursor needs to be on here. 4% on here. I know but it's a write-in. I don't understand how it was not in here because a year ago cursor was arguably more popular than it is right now. Right. Yes. I know. That's agreed. And then also like there's there's no like even like Zed or something isn't on here. I don't I don't know people are paying for Zed. I don't know but that is like their whole paid product. Okay, let's get into other tools here. It's no surprise that the primary amount of the primary language that people are writing with AI is typescript. I'm surprised to see JavaScript at 77% like I'm writing typescript. I haven't touched a JavaScript file in a billion years at this point. Um No, it's I think again those are probably people being like well typescript is JavaScript so I'll check that off. But like I'm at a point where you don't even have to tell it to write JavaScript. You know, or you don't have to tell it to write TypeScript. It just knows, you know, like I haven't had it accidentally write JavaScript without types in a long time. Yeah, totally. Uh Python 41% makes a lot of sense. Python is still big. I'm I'm happy to see the uh the shift to more JavaScript cuz like when we first started with a lot of this AI stuff, it was all Python all the way down. Um and it's been a major shift towards TypeScript. 11% in Rust, which Rust is actually I will say pretty good to write in AI because of uh the build system and everything or the just the general nature of the Rust language. I think lends itself well to this. Um but yeah, uh HTML and CSS being in here as write-ins, I don't know. I don't know how to feel about that. I think they're they're programming languages. Whatever. Who Who cares? I don't want to have that discussion. Um image generation, most people are using Nano Banana and then just Chat GPT second on here. Uh what do you pick up for image generation? I know you you tried a few things. Yeah, I have been I was Nano Banana for quite a long time. I do dip into Chat GPT's image generation every now and then. Uh and then I recently started paying for Midjourney again. Like I was like very early on Midjourney customer. Um and and then I I can't I canceled it and then I I kind of went back to it because it has like has like the best tools. Like it's almost like Photoshop. Like there's layers and things like that. And I don't like I I I I'm trying to think like like what I actually use it for. Obviously for March Mad CSS, we made those hilarious images of everybody in like 90s basketball thing. Um I've used it to generate some icons here and there. It's often some um like I use it more for like assets, you know? Like if I need like flames. If I'm doing like a a thumbnail or something like that and I need some like flames. Like more of my like like I used to just Google image search for things, and then I would find the like stupid free pic with the fake transparent background, and then, you know, I I just I'm so Google image search for stealing photos is [laughter] awful now. So, I now I just reach for for something like Midjourney or whatever just to get assets. Yeah, Midjourney looks pretty great. I I reach Yeah, pretty much pretty much reach for Nano Banana for generating assets or or just Chat GPT for assets. And I've been doing the same thing primarily like for the Phases podcast that Courtney does, I was using like beach photos for her thumbnails just because like that's her vibe. And she wanted something like nice and calming, so we were using waves. And yeah, finding good royalty-free photos of waves was annoying. I would just I'm just going to give me give me 20 waves photos that are like calm waves and and pristine beaches, and it's like, "Okay, here they are." So much endlessly easier. It's It's so true. That just those websites that like bait you on images are have absolutely ruined uh the the process of of finding images. So, Midjourney is cool. I don't know if I'll keep paying for it. I really like using um Open Router for a lot of these image models because you simply just pay for you want. Like when we were doing the the Mad CSS, I I racked up like probably 40, 50 dollars worth of usage on these image things, and I I don't mind that because then like there's months where I don't generate anything, and you don't have to pay for Whereas like Midjourney, they don't have an API. It's simply as you got to pay the like 11 bucks a month, and I think that's where a lot of these models are going to be going where they don't want people piecemealing um 11 Labs is like this as well. They don't want people just like, "Oh, I'm going to take a sip here. I'm going to generate 12 seconds worth of audio, and then I paid my 18 cents for that, and now I'm not going to use it for the next 20 years." Um they want you paying like 19 bucks a month, and you have to go ahead and cancel if you don't want it, and then they'll delete all of it. Unfortunately, I think that's where a lot of these are going to go, and I bet a lot of the big models out there, like I bet we'll see it from Claude, and I bet we'll see this from OpenAI. They're probably going to release something where you There's going to be no API for it. You simply have to use it via their harness. Yeah. Yeah, I I the the OpenRouter stuff is really interesting. Yeah, cuz what was it? Just even like yesterday, the Grok Imagine showed up on there. It It's 5 cents per image. So, 5 cents per image gen via the OpenRouter. And then there's also stuff on here I haven't used, like a a ProVector, ProVector SVG Craft in terms of like an image gen for vector specific image gen. I haven't used that stuff, and it looks pretty pretty sick, actually. I did some of the like the playgrounds, and it looked good. I also have a replicate.com where I throw Every now and then I'll throw 20 bucks in it because they have a very good vectorization model. And you just use it as you need as you need it, right? Every now and then I'll have something that needs to be vectorized. And they just You can just upload it via the website, and then like you don't have to use their API keys or anything. You just use the the GUI on the website. Also, the Grok image one is actually very good, and it's fast, super fast. Um and when you're trying to like like spitball on ideas for things, I find that that's actually been very Yeah. Yeah, OpenRouter. Shout out to OpenRouter. Um video generation, I mean, need to get too much into this. It's not super related. Well, let's talk about this. Like, what I see no purpose for having AI-generated videos, aside from haha, a pig jumped over like the pig is doing high jump or something like that. Like those funny videos every now and I'll get I'll get a little laugh, but I don't I don't see the the actual Maybe Hollywood is wants this type of stuff, you know, like B-roll, uh like a Maybe that would be it. Like a like a Instead of having to get like fly a drone over New York City, you could just generate it. But like I just think a lot of this AI-generated stuff is just That's a lot of money to spend on some I've seen no good come of it. And in fact, I've seen a lot of awful stuff come from it. Every All the boomers on Facebook are getting duped all day long. Yeah, no kidding. Yeah, I I don't I don't have any any use case for that as well. Um so, I I just don't get it. Um the next one is going to be app generation. So, V 0 12% of respondents have used V 0. 9% have used Lovable. Um I I Again, these aren't tools that I These are like feels like these are like non-coder tools. That's It's interesting to hear it. I think in the early days, these things were absolutely blowing up. Um but now when I hear from like my non-coder friends, often I'll hear is, "Hey, did you know Gemini can make games?" Someone met sent me a met like a friend sent me a message the other day and they said, "Hey, look, is this code any good?" They're asking me like, "I know I made a game." And it's interesting to see like where people are coming at. Also, I think a lot of the like early hype on these things was very paid off. Um it's the like amount of people on Tik Tok or whatever being like, "Hey, I made this thing on Lovable or whatever." I think a lot of those people were were being paid cuz you don't see them talking about it anymore. Yeah, I know. Uh and then lovable really burned a ton of goodwill too with that whole uh we meant to we meant to expose all your data. No, wait, never mind. No, we didn't. [laughter] You know, yeah, that yeah. Um the code review tools are interesting here. 14% of people who are using code rabbit, um 13% of people are using copilot, which I'd assume they're just prompting copilot, cloud code, etc. So, people aren't using like code review tools even if they responded to their uh saying that. And the ones that do actually show up are mostly write-ins like um bugbot um and then Sentry Seer has 15 write-ins. So, again, I don't know if people are actually I I think this whole section is mostly write-ins. So, they need some actual code review tools. And Seer is a great it's a great time to talk about Seer from Sentry is well, it it gives you the root cause analysis of any of the issues in your code and has access to uh with your Sentry account it has access to the whole context around why something broke and how to fix it. And Seer is really super cool. But there's actually some other interesting code review tools from Sentry. Uh I don't know if you've seen [snorts] uh Warden West. Have you seen Warden Yes, I have. Warden is a a new tool from Sentry that I think is really cool. Also, they got this sick landing page. Have you seen this thing? and a bug shows up and then a little laser comes and zaps the bug. Thinking cool. I love this landing page. Oh my god. This is great. Um and what this thing does is it uh it's a good tool to have in your stack that uh really is watching your code base for all kinds of things to fix it, to find issues automatically, whether that is code review or security review or dependency review test coverage, those types of things. Um and it will alert you in your GitHub actions. It is really pretty sick. Uh the way I like to think about this is it's kind of being like hooks for finding issues in your code before you push them. If you were using something like Warden, it might find um potential security holes before that you actually merge something. Uh it's really pretty cool. Their their little example up here is a Warden finds a SQL injection from unsanitized input possibility. And it even gives you a suggested fix directly from your GitHub issues. Man, I I This is a tool that I'm instantly going to add to all my stuff and not just because it's a Sentry tool and uh I work for Sentry, but because this is the type of thing that I think we we we all be needing right now with the current security landscape. You know what is one that everybody needs to add is error handling. Um so often I wrote right code where it doesn't handle the error properly. Um or it will simply just like put a like a a label somewhere on there that something happened. And just having like consistent proper error handling techniques so that obviously the error gets sent off to Sentry, but you also show something to your user as to what happened or or be able to handle it and and the thing will fix itself in a lot of these cases. That needs to be added to absolutely every code base that people are doing cuz there's so many times you're just using an app and nothing happens or like something went wrong, you know? Totally. And in perfect tool for that is Spotlight. This is with all this cool stuff Sentry's been shipping. Uh Spotlight does exactly that. It's basically like local Sentry and you get access to um all the errors in your app, especially in development, not just in production. So as you're going, um your AI is having access to all of the errors in a way that it actually has access to solve them and then can use uh the actual Seer AI tools to assist in debugging and fixing. These are the types of things again that as we slow shove more and more AI code out, we need automated and better tools to find our bugs, solve them, and potentially alert us to security issues. So, shout out to the Century team for actually being so uh future-thinking. So, okay. Let's get into the survey, back into the survey now. So, how are people actually using AI in their coding? 90% use it for uh code generation specifically. 68% up from 8% before, so a 60% increase, use it for code review and assistance. That makes no sense. How how the hell did that go from 1% to 68%? before. Maybe it was a write-in before. Yeah. Uh 68% use it for learning and research. 67% debugging. And then you get into much smaller. 37% use it for image generation. And uh I don't even see video generation on here. So, um Yeah, so I again I don't Yeah. It's the wrong industry for this. Yeah. Totally. Yes. Agreed. Um next one we have here is personal expenses. Like, how much do you spend on AI tools every single month? Um 40% are saying $0. So, there's quite a few people that are just getting by on the free trials. And I think they're hopping from one trial to another trial. I think the probably the great days of those days are very limited. And I would probably say in the next 6 months or so that we're not going to have those free plans or even those cheap plans anymore. Like, right now what you're seeing is everybody is saying, "F you, Cloud Code. I'm going to Codex, where I can use it a lot longer." It's I was like, yeah, that's just cuz Kodak's it's like a game of chicken right now. It's like who can lose more money longer. And if you think that like Kodak's is going to be like that forever, they're happily taking you over right now. We're like, hey, come to us. We'll give you better limits. We won't We'll let you use it with other harnesses. We'll let you do everything. And people are coming to them. But you think that's not going to happen with open AI? It's it for sure going to happen. And the the prices are going to go up relatively soon. Yeah, they are. Really? Just trying to get you to use it. But let's take a look at this. 10% of people are spending between 100 and 500 dollars. And then That's up from 9.6% 0.6% of people are using more than 500 dollars. And I guarantee you if you're using AI every single day like the results of these are saying, you are using thousands of dollars a month on this type of stuff. So, that the the disconnect between what people are actually paying or willing to pay and how much compute they're actually using is absolutely insane. Yeah, it's insane. I would be surprised. I mean, what I actually here's why I the the the fact that zero went 13% down and 100 to 500 went 10% or 9% higher than the year before. I think you'll see that that that that I think you're going to see those numbers kind of invert. People are paying a lot of money. It's getting more expensive and more people are finding it to be useful. So, therefore they're going to continue to spend more money on it. Yes. Um I think let's take a look at local AI because local AI often a conversation topic. 20% are not interested. 20 or 20% of people don't use it and are not interested. 31% of people don't use it but are interested. And then 49% of people are saying they use local AI. I think that is not accurate. There's no way that's accurate because look What are people running their own rigs, their own GPUs? What are they running? The crappiest models possible? I don't I don't think they are. I think people don't understand what local AI is. I don't think so either. Uh Like you can use Yeah, you can use Ollama and and whatever, but if you're using this to code, maybe for background images if it's slower but still good. I I think if if you're worried about Like people are using local AI if they're worried about either cost or privacy, right? And I think a lot of the cost people have have moved to Kimmy, Deep Seek, a lot of these like cheaper models. And I bet that's where we're going to see a lot of people go in in the next 6 months as the real prices for these things start to come out. We're going to People are going to be figuring out how to get like decent results out of these significantly cheaper models or the models that are I don't know if they're they're cheaper because they're they've been made like that or they're just again being heavily subsidized to to get people to hop over. And then the like the privacy people as well. Like I would love to have a rig on my desktop that I could just run all the stuff locally. But like it's I don't think people realize how big beefy of a a system you need. And CJ has a video on it, right? Like you got to have a pretty sick setup. You got He spent thousands and thousands of dollars on on his rig. And it's like it's it's good, but it's like nowhere near what um the things that we're used to using, all these Claude and Open AI models. Yeah, I think that's the whole thing is you get used to the Claude Open AI models where you just type it in and it does its thing. You're going to have a rough transition. I think the benefit comes in from using these smaller models to do smaller tasks and then really getting into the systems of it where all right, this one is doing the to-do list, and this one is doing the the prep, and this is writing the ADR, and this one is reviewing it, and then you have your big beefy frontier models doing the actual coding, but that is a lot of stinking work to just to I mean, it depends on what you're using it for. My homie Tofer, who's been on the show before, he has two DGX Sparks from Nvidia, and he does all of his open claw stuff with those. All of it. So, And what does he use open claw for, though? Like cuz that certainly you can do things like summarizing note or writing as personal assistant stuff. the programmatic stuff. Yeah, like moving files, categorizing things, taking a voice note and and putting it into a to-do app, right? That kind of stuff you don't need like these these front-line models. That stuff worked great 3 years ago with what we had then. Totally. Yes. Yes, totally. So, I again, yeah, local AI, interesting thing, but again, yeah, I don't think these people are really using it like they say they are. One thing I wanted to hit on is this next section, which is the risks and pain points. Job displacement being the largest thing inside of AI risks, which yeah, I think that's a big a big point of I would say stress amongst developers, but I don't think we've truly seen the end results of this just yet. We hear about layoffs all the time, but I think companies who have done good hiring practice from the start, they're still hiring. I mean, like Century is still hiring all the time, and they haven't done any layoffs, and they're you know, it's like companies like that that are that are using AI heavily, but still, you know, finding a way to make their employees effective with it. Military use of AI 45% Yeah, environmental impact 40%. Also concerning. Also concerning. AI slot takeover very concerning. Negative cognitive impacts. Yes. Have you experienced any negative cognitive impacts impacts from using you might be entitled. Yeah, when's the class action of like Yes, it ruined my brain. Yes, I know. It Yes. Yes, I I think think rising AI costs are a problem. Kind of interesting as well. But like you also hear like like people's companies are simply like make mandating them. Like when we were at GitHub Universe last year, they announced like the ability to track how much people are using. And right then and there like yeah, this is what like people want their employees using this stuff to to move faster and to be more productive and you're seeing a lot of people who are just like No, I don't want to use this. Partially because I don't trust it. I don't want to my job to be replaced with it and partially cuz they just don't like a lot of people don't really want to learn how to how to use this effectively, right? Like I think there's a lot of people have simply just used the chat GPT typed into the box had it spit out did 18 pages worth of garbage and they're like that's not very good. And they don't see the the actual benefit, you know, like give somebody who just is like a chat GPT casual listener, give them like show them how powerful like a cloud agent can be. I think that they might be may change their mind. Yes, I I I do think there is a tremendous gap there. Now, the highest thing that people say is a pain point is hallucinate hallucinations and inaccuracies. Yeah, that's part of the course. I think the there's a a meme that shows like pre-AI development and it has like you're building a scooter and it's like here's the wheel, here's the other wheel, here's the base, here's the thing and you put them all together sequentially and then it's like post-AI development or using AI development and it's like some monstrosity and then you're refining it down into the actual scooter. So like I think people who aren't used to this stuff, they don't realize that you almost start with like you start with this jumbled mess and you refine it down rather than you start small and refine up to, you know, code quality or higher things. But hallucinations and inaccuracy, yeah, I mean that's the name of the game is is trying to reduce these things whether it's through deterministic means like linters and things like follow or whatever, you know, skills to help it along. MCP, whatever, code quality being the second highest things, the second highest pain point. Yeah. Yeah, obviously. I think these are big things. Ethical concerns, a little blip on the pain points, but I don't think that belongs inside of ethical concerns. Okay, here here's here's one thing I wanted to touch on, the happiness of people who have been people's happiness with AI has gone from 3.3 out of 5 to 3.4 out of 5. So people who have taken the survey are generally happier happy with AI than they were ever so slightly than the year before, which tracks I think. I mean how could it not? The models have gotten better, the tools have gotten more mature, everything's gotten a little bit better. I'm surprised the jump isn't a little bit higher, but again 3.4 out of 5 is pretty high already in terms of what I would expect here from this survey. What I think would be more interesting than how much do you like AI 1 through 5 would be getting into the psychological stuff here, right? Like I I think about this all the time and maybe I I've been thinking less this would be a good episode to bring Courtney on um to talk about psychology here. How does coding with AI make you feel? How stressed out are you with working when with AI? I'm more stressed when working AI because I don't have as much control over it. I'm like, "Do the thing I want you to do." Uh how how does it make you Yes, fragmented. I would say developers are probably less happy. Like you you can make a lot more things and like I certainly have this problem is that they I have a million ideas. My brain is always buzzing and like having a box that can actually make these put these things into reality, that's that's a bit of a problem for me. It's brutal for my mental health because I'm like, "Yeah, I'm feeling like I got to keep this going." What do I I installed a caffeine app on my computer so now that I can shut it and have my agents run while I my computer's shut. That's not healthy. That's not healthy at There's another question though. It's like is software getting better? No. I mean [laughter] maybe it will but who can use any single thing on their device and be like, "Oh yeah, this to think like from like a user point of view of like stuff I use every single day. Like like there's certainly lots of like workflows in my life that are a lot better and easier of things that I I use every single day, my day-to-day things. Has that gotten better, you know? Like my I always love to talk about like FreshBooks. FreshBooks was this beautiful thing that I loved to use. They're a sponsor of the podcast for many years and I loved it and over the last like 5 years the quality has just gone downhill and there's been no features and nothing ever works and then I'm just like looking at it being like, "Shouldn't this be way better given that we have this magical genie that you could just rub and it fixes all of our problems? Shouldn't it be better? Shouldn't it be better? It's something I ask myself all the dang time and it's not in in too many of these companies are just spending time just shoving AI into their product to make it worse. I mean notion, oh my god, notion is just nightmare to use sometimes. So, yeah, I I don't I don't know. Yeah, I don't think things have gotten any better at least not for my perspective. Some things have, some things have gone just perfectly, but I think those are teams that are always care and do care about craft. So, I I I think it's yeah, it's interesting. Um but yeah, I would love to see this survey include those types of of of things if we're getting into, you know, how do you how do you feel about it? How does it make you feel? Um the opinion section of this is the last one. 72% of respondents now consider AI to be an integral part of their workflow. 72% of respondents. That is high, folks, for the people who are saying it's useless. 72% Do you think 72% of people are being forced by their company cuz I know that we'll get comments being like, well, their companies are making them use it. 100% of the people leaving comments on this video. But like you you're telling me that the 28% of people don't find it as an integral part of their workflow. I guess that's that's fair. I think I'm going to think like is that like are the like maybe those people are like they're certainly using it, but maybe they don't find it that it's a an integral part. integral part of their workflow. Integral integral being the keyword here, yes. Um interesting. There a lot we hear a lot of talk about bubble. Many respondents think that we're currently in an AI bubble, but they also realize AI is here to stay even after it eventually pops. Um 37% agree with that. 33% strongly agree with that. Very small percentage of people disagree with that and strongly disagree with that. So, yeah, I think we're seeing some of that bubble in just terms of the cost that we we've talked about. The cost exploding and it's going to get more more interesting. What does it What does it look like when the the bubble pops, though? That's that's my question here. Is that like I certainly think everything is chaotic right now and everybody's trying to figure out like what what it looks like and there's a lot of money being thrown around. I certainly think we are in a bubble, but it I don't think that like when that pops, I don't think that we're going to go back to to what the way it was Stone tools chiseling into rock, yeah. it's just the the I think the way that we work is significantly changed and there's I don't think there's any returning back to to where we were. The genie's out of the bottle. You're Genie's out of the bottle. Yeah. I Okay. So, I What are your thoughts here, Wes? Because I have some thoughts on the survey specifically. So, here's what I would love to see. I want to know what you would want to see from the improvement of this survey overall, but I know Sasha he he's a great dude. Here's what I would love to see this survey improve on for the next one. I I think it's missing a huge part of the actual AI landscape. It's missing things like skills. It's missing things like MCP. the most impactful AI features that people are actually Yeah. What are you actually using? It's more than just model or agent. There's what skills do you think are effective? Do you think skills are more effective than MCP or do you think subagents? Are you using swarms? Are you using like those types of things? the code? Um How like are you monitoring it? Because like are you are you just firing off several agents at once and not monitoring or are you coming back to it? How are you kicking them off? Are you doing it from your phone? I think that this survey missed a lot of Like yeah, it's interesting that people are using Open AI versus Claude and and whatever. But I think like how people's workflows have changed in the last 6 months have has been the most interesting part of this landscape and and how people are tackling different types of issues. You know, like Pi auto research. Absolutely phenomenally interesting tool of like these are new these are new ways to solve problems. Not like a lot I think a lot of the survey was simply just like we used to write code by hand. Now we are having the bot write the code for us and not just like how how how are the new um problems being solved here? I think that that's the more interesting thing. Like like how are you rigging up your your your whole thing? What's MCP, the skills, all of that good stuff. Yes, I I totally agree. Yeah, I I even here like strategies would have been useful. Like what are people actually are using ARD documents? Are you using just markdown files for documents? You throwing a whole docs thing? Are you using like a task manager or something like beads? Like these are the types of things that I think need to be in the survey if it's going to um be useful. Uh so I Sasha I I I hope you uh take this criticism as critique. I think you this is only the second time they've done this survey. There's probably a lot more that can be learned from this. So I would love to see this survey be a little bit more representative of how people are actually working in AI beyond just you know, uh opening up their VS code and and using co-pilot or or whatever Claude is doing. like that, why I want to see are people are developers actually ripping three agents at once or is it just like us who are are at it? I would I would like to see that are people actually using MCP servers and skills or are most people just simply opening up, typing in the box, reviewing the output? You know, there's there's a big difference between that and I'd like to see some real world stats on it. How much code do you review by hand? How much code are you actually reading that your agent, right? So those types of things, yes, absolutely. Cool. Well, do you have anything for us before we get out of here, Wes? That's it. Thanks for tuning in. Peace. Peace.