The Data Engineering Skills Gap: Are You Left Behind?

What things are spiking in searches in the data fields? Is it that Snowflake or Databricks right now? is a big one for sure. I think what happens with tech that's very interesting is that it's like that first line theory. It's like if there's a long line here at the cash register and someone opens up another cash register, half of the line moves there. All right? And so Snowflake and Databricks is the same thing. It's like everybody's starting to try to learn it now cuz they know it's getting there. So we're always looking to see what's next so that you can be competitive no matter what the time of the day is. technology pops up. What are some of those you're working Real-time Real-time streaming, VT, we think those are good. I think starting to become cloud agnostic is a big one, meaning like back in the day you probably Back in the day, quote unquote, 2 years ago, right? You needed to learn one cloud. Now it's like you need you could benefit from learning a second one cuz there's tools out there that actually can use both clouds. And so Sagemaker is one in AWS, so that's one that's primarily focused on ML, right? ML models. And so that's one where as a data engineer you might not work on those ML models, but you might work with a data science team by your side. So if you have knowledge of what they're working on, you can become a double threat. So the new tools I would say, obviously DBT, real-time streaming, right? Something that actually alerts, like that Amex example, like, "Hey, in real time, let's send you a fraud alert." Anything like that I I think is still very under known. Obviously Snowflake, Databricks is up there. The next one, my guess is related to AI. How do we use AI to do our jobs quicker as data engineers? And I think it's going to be more complicated than just ChatGPT, but you need to know the basics in order to use ChatGPT, right? You're not going to use ChatGPT to help you with SQL if you don't know SQL. But I think the overall lesson there on that topic is what we see a lot is people coming to us and being like, "Hey, the market is bad. I can't get an interview." You look at their resume and they're like they know SQL. And it's like, "Okay, well, is that it?" And they're like, "Like that's been working for the last 15 years." I'm like, "Great. The The in college now knows SQL. The 17-year-old now knows SQL. You can't rely on that forever. And so, I think regardless of what the tool is today or next month or next year, depending on when they're watching this video, the idea is the continuous learning, basically. I would say that's a good place to start. Kinesis, real-time, DBT, be on the lookout for any AI stuff coming out. We might be coming out with something. Oh, wow. Okay, we'll see there. And and what's your what's your take on college? Obviously, you're a competitor to college now. It's a good question. Obviously, you're significantly cheaper than even one semester of most colleges, right? So, I got I got a few hot takes on this one, but master's programs, I think they're going to go extinct. Master's programs, specifically. I'll talk about college in a second. MBAs, master's programs, two-year degree type of things, I think are in danger, right? From what we're seeing, because these coaching programs, online courses, mentorship stuff, such as DEA, Data Engineer Academy, 90% cheaper, 90% faster, more efficient, because people don't want to sit in a classroom for 2 years, spending 100K, having to potentially quit their job. Just like I was saying, like sometimes these are working professionals that just want to learn for 2 or 3 months and get their next job. I don't see a place for master's degrees, unless you're obviously a doctor, a lawyer, or something that you need to do. With college, I like to segment it. I think Harvard and Yale and top schools are underrated. I think they're I think they're not charging enough. I think they're actually underpriced. And this is a take most people don't really hear, but the reason is the value there is the network. I want to go get a billionaire mentor right now, I have to pay 100 grand a year, maybe 200 grand a year. You pay 70 grand a year for Harvard, you become actual friends, build genuine relationships with a bunch of people whose fathers are probably billionaires. And the Harvard brand sticks with you for the rest of your life. I think top schools are underpriced. I really do. With that said, the other 80% we'll see. Not too optimistic about where they're going to be in 20, 30, 40 years, because by the time I have kids in 30 years, their bill is going to be a million dollars, right? Over 4 years. It's going to be crazy. I don't know if people are going to continue to buy that unless they find a way to make it cheaper. Now, with that said, I don't think it necessarily means everybody should just go do self-taught things because realistically speaking, that's not how people are wired. I don't think think the other extreme that people go to is like, "Hey, you can just buy a $10 textbook on accounting, right?" It's like, "All right, well, the person that's motivated enough to do that is going to do that no matter what. Yeah. Most people have the problem of not having the motivation to do that. So, in a way, 80% of the colleges are like babysitters, to be honest. But, that's not a bad thing. Question just is, how do we make it cheaper? Because if you're in a top bottom 40% school and you're taking out 100 grand in debt, we know what the economics are. It's not working, right? People coming out of debt, getting a job for minimum wage, and have 300 grand in debt. Then, they still come to you. Exactly, wanting to actually do something. after the 300. Exactly. And so, I'm starting to hear like, "Hey, this millennials aren't able to afford a house." But, but I'm like, "The next one's going to be they're not able to afford school." And so, again, it's crazy because there's no real restrictions on how who can take out a school loan when in reality, it's like you're giving an 18-year-old a loan of 300 grand. So, again, it's a take that I think is starting to come out more. I'm not saying 80% of schools have to go extinct, but again, I think something remote, like what we do, is a solution. It's an alternative, at least. So, let's see what happens there. But, I would say we compete more with masters. Like I said, like Harvard, you know, I think they're I think they're under priced. I think they're charging what others do, and I'm like, "Huh, don't get that, to be honest." So, we'll see. I haven't heard that take yet. Seems like initially college, the experience was such a big deal, but I feel like with the If you're you know, people in Miami earning 100 a K a month Dude. as 20-year-olds Insane. They're living pretty great experiences, Insane. I know I know a few 17, 18-year-olds that are making about They've built eight-figure businesses with no code tools. Now, they got smart because once they built it with no code tools and they learned the engineering stuff cuz they were like, "Oh, I can do a lot more if I learn engineering." I have [clears throat] a mentor that says this all the time. He's like, "Very soon, maybe this year, there's going to be stuff where you can just be like, all right, like spin up a Stripe competitor, have it ready by the morning." And AI just builds it all. The software might not be the initial competitive advantage, it might be the brand. But, at the same time, if you learn engineering on top of that, then you can actually take that stuff to the next level. So, you still need to know both, which is why again continuous learning, but you're right, man. I know I got 17, 20-year-olds who are just crushing it. Crushing it, dude. Well, why don't you give one last pitch on Data Engineer Academy? I My pitch is there's not enough people doing There's not enough people with these skill sets for me to place in jobs. So, what about you? So, I'll leave you with this. I One of the things you do when you start to build a company as a founder, you start to instill mission values or leadership principles, if you will, build the culture of your company. And one of the things I tell my employees all the time, extreme ownership, right? I hate when an employee says, "Hey, that wasn't my job, so I didn't do it." I'm like, "Great. Imagine being a parent and being like, well, I didn't pick up the kid because my wife was supposed to." Right. So, I just left my kid there for 20 It's like, what are you talking about, right? Extreme ownership, do what needs to be done, get it done for the sake of the team, the sake of the company. I would just say do that for your own career, right? Treat it like a job, treat it like your company, and really self-audit and ask yourself, like we were talking about before, when's the last time you learned something new, right? When's the last time you reached out for help? When's the last time you admitted like, "Hey, I'm not happy," right? And just make that jump. Cuz again, the technical stuff, it will come. I know everybody's obsessed with what should I learn this or that or that? Should I learn this or that or that? It's like, great. That's just about sitting down and putting in the work. But, once what you need to do, which is biggest problem, right? Cuz people don't have clarity, then it's just a matter of doing it. We talked to again, we've talked to 100,000 people.