Data Scientists vs Data Engineers: Which one is for you?

so there was a question on tech interview Pro in our private group and there was a question that asked what are your thoughts on career outlook for senior data scientists versus senior data engineer if someone has a choice between the two roles for their next job advancement which one would you suggest so I answered this and I kind of wanted to make a video talking about this question too because I have a lot of things to say and as you know I work to add facebook back then I used to be a data scientist I think I could say it now we'll see but personally I do feel that data engineers are sometimes a little bit like a second-class citizen I mean sometimes data scientists too but I feel like data engineers can feel like that a little bit more now my roommate is a data engineer so don't tell him that but I personally think in a product heavy company culture I'd say that PMS are the first-class citizen and then data scientists and then data engineer if we're talking in the data space and the reason is because PM's they often are the ones who make executive decisions there are the ones who kind of own the feature on the product right and then for that they tend to ask the des a lot for product questions like Oh what are user retention so can we cut this by by a country oh can we cut this by this and they have a lot of data asks and in some sense it's a little bit of like [ __ ] work because they just want to get it done but then they realize that oh you know this information they actually don't even care about it but they just want to know just because of their curiosity and it doesn't actually change the decision of the product that happens a lot and then for the DES to do those data ask they need the data to be structured in a certain way so then because of that they asked the de's to do it right so as you can see it's a chain of [ __ ] work right yes the data scientists they don't get impact for helping PMS answered your curiosity des data engineers they don't get impact from creating these temporary tables or temporary pipelines just to ask as just to just one task for the data scientist so that's why you know it's like the lower you are in the chain of [ __ ] and sorry not [ __ ] but like the shitty task the more it tends to be not that impactful right so I believe that politically being a des is better than P and data engineer I also talked about you know politics at work a lot and I think a lot of people misunderstood what I said they think that oh no it's terrible there's lots of politics but I didn't say that it's a bad thing politics is part of life politics is part of society when we have multiple people working for different things having different incentives politics is super important then you should be extremely good at it because that is how you negotiate that is how you get together with a lot of people with different incentives so that you can all commonly work on the same goal right or even if it's not the same goal you could work on different things such that it's mutually beneficial for each other cool now the thing is yes there's a lot of like crappy work going down and then that's usually what DS managers and DD managers do they try to fend off these ridiculous requests so that their report can be a lot more productive in real work right okay so let's talk about des or de versus des so I have a lot of friends that are data engineers and to be honest they say that they don't really like data engineers that much because because in some sense it's not intellectually challenging especially at a big company where a lot of these hard distributed data problems are already solved right so you're not going to be doing any spark or Hadoop jobs and stuff like that these are these are abstracted the thing you do as a team is you mostly write sequel based stuff write sequel based pipelines because that's the easiest way to write it and they have abstracted everything such that it makes it easy and then what you have to do is you just have to organize your project areas data and you have to monitor these ETL pipelines and all of these things they're basically like like sequel kind of work or just like maintenance and I'm using the the tools that we've already been that already been built right and you also get paid less than a software engineer so I think a lot of people a lot of data engineers at least have Facebook I don't know about any other companies they're a little bit more dissatisfied because their job is less intellectually stimulating and they get less equity so I think that's one of the biggest cons of data engineer but I am biased because I was a data scientist so I might not know a lot of things about data engineers cool so for data science analytics and I mean analytics especially for people who want to get into data science and they don't have a master PhD in something very particular like in machine learning usually you're going to be ending up doing data science analytics now most people come into this thinking that it's like big data or like machine learning kind of thing but they quickly get disappointed because they realize that it's not that's that's not what it is right especially if you're a new grad coming from school unfortunately you're not going to be doing any innovative machine learning algorithm stuff you're not going to be inventing anything right you have to be a little bit realistic so but I do personally think that data science analytics is super rewarding right like like the thing about new grads is they focus too much on the technical part of work you know they focus too much on try and grow in terms of their technical skills which which makes sense because in the beginning you need technical skills you need these foundational skills to be useful in work to do stuff right but to be honest the technical part is usually the easiest part of the job at a large company at least like if you want to be a data scientist then you're the person that loves to think about the product loves to think about what to do next for the product and you want to be the expert on the product because you love digging into the data finding finding insights and then using these insights to help the company or help the product become better and increase those metrics right and yeah so that is actually what a data scientist is supposed to be you are the core of the strategy and direction of the product and your superpower is that you're able to transform raw data into should teach it decisions that's your job it's kind of like you know when we talk about programmers they transform redbull into workable software right it's kind of like that so I do think that it doesn't really matter what job you choose because the people who advance the most the people who succeed at work are people who do not restrict themselves to death job function right and you should always be striving to think like what can I currently do with my set of skills that would have the most impact on my team like that's what you're supposed to do that's how you're supposed to choose which job you want to do it depends on what skills or interest you have the most but then once you're at that job do not restrict yourself right because the people who do not restrict themselves are the ones who who succeed the most for example I wrote a few examples like engineers with data analytics skills where you can do is as an engineer you could find user problems yourself dig into the data find issues of your product right and then now you know that there's a great future opportunity because of your data skills and then you could argue for it like with your with their PN team with your product team and then just build it and then that would have a lot of impact and then if you're a PM with a with engineering skills then you can prototype an MVP you can or you can create an experiment for a future and then just build it out and then test it out and then see that there's positive metrics or that it's good for the are you found product market fit with that experiment with that 1% experiment and then boom there you go you you're able to create an MVP and then validate your hypothesis and then have this project up in the high priority list on your roadmap high-impact for PMS as a data scientist if you have PM skills then essentially you can set the vision and also the roadmap for your team because you know you're able to have the data skills to back up all your arguments for what should be the high priority the stuff to do for the company or for your team and then you're able to prioritize all these features because like I said have these data skills and then because you have this leadership skills or these PM skills I said you have you're able to launch your team to success because of that so even if you have a weak PM you could carry the team so I mean in conclusion pick a job according to your skills and but never restrict yourself on your job title they pay you to invest in you so that in the future you could make a huge difference so take that opportunity to invest in yourself so that you could be a beast at everything [Music]