133: My Honest Thoughts on The Data Job Market in 2024

Avery: 00:00:00

I'm going to be honest, the data job market has been

2

: 00:00:02

really rough the past year.

3

: 00:00:04

With the rise of AI, layoffs, presidential

political turmoil, interest rates,

4

: 00:00:10

you're only really hearing a lot of

negative things about the data job

5

: 00:00:14

market and tech job market in general.

6

: 00:00:16

You'll hear all these things on

different social media platforms

7

: 00:00:19

like threads or twitter or maybe some

sort of mainstream media platform.

8

: 00:00:22

Platform like CNBC or Fox

News or something like that.

9

: 00:00:26

But what's actually going on in

the data job market right now?

10

: 00:00:29

Well, there's a lot of opinions.

11

: 00:00:30

You'll hear different things if you're

on YouTube or if you're listening via

12

: 00:00:33

podcasts or on X or threads or Facebook

or from your friends, it's really hard.

13

: 00:00:39

And everyone kind of has a

different opinion about it because.

14

: 00:00:43

What's the actual truth?

15

: 00:00:44

No one really knows.

16

: 00:00:45

No one exactly really knows how

the job market is going right now.

17

: 00:00:49

And I can tell you what I'm experiencing

from being a data analyst, career

18

: 00:00:53

coach for over 60 different students.

19

: 00:00:54

I can tell you about posting every day

and interacting on LinkedIn or from doing

20

: 00:00:58

this podcast and talking to industry

experts, you know, people in the field.

21

: 00:01:01

But here's the truth.

22

: 00:01:02

Those would still just be

kind of anecdotal opinions.

23

: 00:01:05

It's what I'm experiencing.

24

: 00:01:07

It's what the people around

me are experiencing, but it

25

: 00:01:08

wouldn't be quite comprehensive.

26

: 00:01:10

So, but more importantly, it

wouldn't really be data driven.

27

: 00:01:13

And it's always better to be data

driven, especially on channels like this.

28

: 00:01:17

We're data analysts, right?

29

: 00:01:18

We want to go off of what the data says.

30

: 00:01:20

Let's go ahead and dive into some data.

31

: 00:01:23

I was lucky to get my hands on this data.

32

: 00:01:24

This data was collected by a company

I was recently introduced to.

33

: 00:01:28

It's called Live Data Technologies,

and they track real time employees

34

: 00:01:31

Employment data, leveraging

publicly available data sets.

35

: 00:01:34

So basically what the company does is

monitor different platforms and sees who's

36

: 00:01:39

leaving jobs, who's coming into jobs.

37

: 00:01:40

They're basically looking around the

internet and publicly available data

38

: 00:01:43

sets and trying to make sense of it all.

39

: 00:01:45

The company sells the data and

the insights that they produce.

40

: 00:01:47

Pick up on this data to product

builders, investors, talent teams,

41

: 00:01:51

all sorts of different people.

42

: 00:01:52

And luckily for us, they've agreed

to make some of this data and some

43

: 00:01:54

of these insights freely available

to benefit the data community.

44

: 00:01:57

So special shout out to them

specifically Jason Saltzman.

45

: 00:02:00

When I looked at this data,

I had five main takeaways.

46

: 00:02:02

I had five things that I was like,

huh, I didn't necessarily expect that.

47

: 00:02:06

Or I was like, oh, that's what I thought.

48

: 00:02:08

And this data confirms it.

49

: 00:02:09

And you want to make sure you stick

around to the end because the last one.

50

: 00:02:12

I think that one will make

you feel the best and the

51

: 00:02:15

most optimistic spoiler alert.

52

: 00:02:17

All right, so let's dive into number one.

53

: 00:02:18

For a good portion of the 2010s,

data scientists was labeled the

54

: 00:02:22

sexiest job of the 21st century.

55

: 00:02:24

And as a data scientist myself, I

like to think that I'm pretty sexy.

56

: 00:02:28

So I kind of agree.

57

: 00:02:29

No, I'm just kidding.

58

: 00:02:29

The businesses really saw it

as a really sexy role and very

59

: 00:02:34

valued for their business.

60

: 00:02:35

And you got paid a lot.

61

: 00:02:36

You can work remotely and

that's still the case.

62

: 00:02:38

But I would say that the

data scientists role.

63

: 00:02:40

Uh, it's kind of broken up

into different types of roles.

64

: 00:02:43

I think originally it was kind of

just the data scientist role, but like

65

: 00:02:46

now we see a lot more data engineers.

66

: 00:02:49

Now data engineers did exist

back then, but it wasn't

67

: 00:02:52

nearly as popular as it is now.

68

: 00:02:54

There's other roles being created

all the time, like analytics engineer

69

: 00:02:56

is one of the more new roles.

70

: 00:02:58

Um, so one of the things I

looked into is like, okay, with

71

: 00:03:00

these different data job titles.

72

: 00:03:02

Um, yeah.

73

: 00:03:02

Yeah.

74

: 00:03:02

Which one of these titles have had the

most growth in the last five years?

75

: 00:03:06

And it's not really a surprise.

76

: 00:03:08

It's data engineering.

77

: 00:03:09

There's a couple reasons

behind this, I think.

78

: 00:03:11

Number one is we thought data

science was sexy, and it is sexy.

79

: 00:03:15

Doing things like machine learning,

predicting things, using, you

80

: 00:03:18

know, AI, those types of things.

81

: 00:03:20

Obviously is very cool, but the problem

is data science can't get a whole lot

82

: 00:03:23

done without a data engineer The data

engineer needs to be there first to kind

83

: 00:03:27

of set things up get the data all clean

prepped stored Usable in the right ways

84

: 00:03:31

and that just wasn't really the case in

the early: 2010

85

: 00:03:35

this huge rise of data engineer where

it's actually the fastest growing data

86

: 00:03:39

role out there That's not to say that the

data scientist It's not quick growing, but

87

: 00:03:43

it's actually growing quite a bit as well.

88

: 00:03:45

It's just not growing as fast

as it was maybe in early: 2023

89

: 00:03:48

But still growing quite a bit.

90

: 00:03:49

The other reason I think these

data engineer jobs are being so

91

: 00:03:52

in demand in the last year and a

half specifically is due to AI.

92

: 00:03:55

AI is a really interesting problem

because There's all these AI models

93

: 00:03:59

out there, but really the model is

only as good as the data you give it.

94

: 00:04:03

The better data you give it, the better

the model is, and also the more data

95

: 00:04:06

you give it, the better the model is.

96

: 00:04:08

And data engineers have this unique

skill set of being really equipped to

97

: 00:04:13

store data incorrect places and make

it easily accessible to everyone.

98

: 00:04:16

So data engineers are great fits

for AI companies and AI products.

99

: 00:04:19

And so I think that's kind of why we're

seeing a data engineer boom right now is

100

: 00:04:22

because those skills are really in demand.

101

: 00:04:24

Now for the same reason with with AI

being good for data engineers, is AI bad?

102

: 00:04:29

For data analysts, and I can't even

tell you how many messages I get

103

: 00:04:33

of people asking me, oh, like, is

being a data analyst a good choice?

104

: 00:04:36

Is it gonna be overtaken by ai?

105

: 00:04:38

Am I going to lose my job to

AI in the next five years?

106

: 00:04:41

And let's go ahead and take this

chart that we showed earlier.

107

: 00:04:44

Just focus on data analyst jobs in

particular, take out the other job

108

: 00:04:47

families and take a quick look.

109

: 00:04:49

So what you'll notice here is if we look

at this graph and just do the solo shot.

110

: 00:04:52

Is that data analyst

jobs are still growing.

111

: 00:04:55

There's still growth over time.

112

: 00:04:56

Now you might be tempted to be

like, no, Avery, look at the top

113

: 00:04:58

of that chart in the top right

corner, it's pretty stagnant.

114

: 00:05:02

Well, that's actually stagnant

growth compared to: 2019

115

: 00:05:06

So the role is still growing at

like 14 percent year over year

116

: 00:05:09

when you compare it to 2019.

117

: 00:05:11

So it's still growing quite

a bit every single year.

118

: 00:05:15

Leads me to believe that data

analyst role is still a great role.

119

: 00:05:18

It's not being replaced by AI.

120

: 00:05:19

I don't really think it'll ever

be replaced by AI, but it's

121

: 00:05:22

certainly not happening now.

122

: 00:05:23

And I don't really see it

happening down the road.

123

: 00:05:25

I see AI more as a tool that

helps analysts analyze faster.

124

: 00:05:29

It's almost like when Microsoft Excel

did, you know, the data analysts then

125

: 00:05:34

lose their job because all of a sudden we

could do these calculations in a computer.

126

: 00:05:37

No, it just helped them

do their job faster.

127

: 00:05:39

So I see AI as a tool that helps

analysts get their jobs done

128

: 00:05:42

quicker versus something that's

going to ultimately replace them.

129

: 00:05:45

It's a tool essentially, like a hammer.

130

: 00:05:47

I think data analysts are still

very valuable for companies.

131

: 00:05:50

They're providing them great insight at

a little bit more of affordable rate.

132

: 00:05:53

And it really helps these companies

get like the low hanging fruit

133

: 00:05:56

of all things in their data.

134

: 00:05:58

Because to be honest, AI is sexy,

machine learning is sexy, but a

135

: 00:06:01

lot of companies aren't there.

136

: 00:06:02

A lot of companies just

need to be more data driven.

137

: 00:06:05

And I think a data analyst

is a great Trust me, there's

138

: 00:06:07

so many companies out there.

139

: 00:06:08

Like, like, obviously there's Google,

there's Tesla, there's Facebook

140

: 00:06:11

where they're doing cutting edge

machine learning stuff all the time.

141

: 00:06:13

But for every one of those

companies, honestly, there's probably

142

: 00:06:16

thousands of other companies who

just need to make a report or

143

: 00:06:20

just had some data pulled in SQL.

144

: 00:06:22

Like it's, there's a lot of opportunities

for data analysts out there.

145

: 00:06:25

And that was my second takeaway.

146

: 00:06:27

My third is that job hopping is, if

you look at this chart right here,

147

: 00:06:32

it'll show you the average tenure

of the different data job titles.

148

: 00:06:35

And that basically just shows you how

long they're staying in a specific role.

149

: 00:06:39

You might notice that database roles,

they're staying there quite a bit earlier.

150

: 00:06:43

The rest of these job families look

like they're pretty similar in terms

151

: 00:06:46

of how long they're staying there.

152

: 00:06:47

And it ranges anywhere

from two and a half.

153

: 00:06:49

to one and a half years.

154

: 00:06:50

And what I get from this is that

is the average that someone is

155

: 00:06:53

spending at a company before

switching to a different company.

156

: 00:06:56

I think that's a good thing.

157

: 00:06:58

I think that should give

you confidence to do it.

158

: 00:07:00

I think in the past it was frowned upon

to leave a company early, but now I think

159

: 00:07:04

it's not nearly frowned upon as much.

160

: 00:07:06

I think more people are doing it and I

think it's good because I talked about

161

: 00:07:09

this in my episode with Zach Wilson,

where he discussed how he went from

162

: 00:07:13

like 30, 000 to like 500, 000 in like

seven years or something like that.

163

: 00:07:17

And one of the reasons he was able to do

it was he switched jobs every 18 months.

164

: 00:07:21

And for some strange company, we live

in an economy where you're actually

165

: 00:07:24

probably worth more to another company

than your own, they're willing to pay

166

: 00:07:28

you more than your current company

is, which is weird and messed up.

167

: 00:07:32

And we can go into that, but.

168

: 00:07:33

The point here is that it looks

like everyone's job hopping.

169

: 00:07:36

And so you might consider it as well.

170

: 00:07:38

Point number four.

171

: 00:07:39

And that is that data hiring is happening

literally in so many different industries

172

: 00:07:43

and so many different companies.

173

: 00:07:45

Uh, I'll pop up on the screen,

a couple of graphs here.

174

: 00:07:47

We'll look at the first one,

which is where companies are

175

: 00:07:49

hiring data analysts in 2024.

176

: 00:07:52

And what you'll notice here is

there's so many cool companies

177

: 00:07:54

like Capital One, Accenture,

Deloitte, Data Annotation, Google.

178

: 00:07:57

What I want you to point out here

is like, Obviously, Google's here.

179

: 00:08:00

Obviously, Tesla's on this list.

180

: 00:08:02

Apple's on this list.

181

: 00:08:03

But there's a lot of like more traditional

companies that aren't like big tech

182

: 00:08:07

companies that aren't fang companies.

183

: 00:08:08

And a lot of the times I think that we

associate the data analyst role with tech

184

: 00:08:12

and because it is kind of a tech role,

but data analysts work at manufacturing

185

: 00:08:16

companies, they work at finance companies,

they work at healthcare companies.

186

: 00:08:19

They don't only work at tech companies.

187

: 00:08:21

The tech companies are kind of the

sexy ones, and they often have a high

188

: 00:08:25

salary, but there's so many different

roles at so many different companies.

189

: 00:08:27

And sometimes I think we forget that,

that like, it's not just Facebook.

190

: 00:08:31

It's not just Netflix that

are hiring data people.

191

: 00:08:33

It's manufacturing companies.

192

: 00:08:34

It's consulting companies like Deloitte.

193

: 00:08:36

It's healthcare companies like Optum.

194

: 00:08:38

There's more opportunities for

data analytics outside of tech

195

: 00:08:41

than there is inside of tech.

196

: 00:08:42

And I think And then these graphs here

that show what companies are hiring the

197

: 00:08:47

most data engineers and data scientists.

198

: 00:08:49

I will point out that data

scientist companies are a little

199

: 00:08:51

bit more of those tech companies.

200

: 00:08:52

Meta, Microsoft, TikTok, Google, right?

201

: 00:08:55

Those are a little bit more of what you

typically feel in terms of tech companies.

202

: 00:08:59

That being said, there's still

consulting companies on this list.

203

: 00:09:02

There's still banks on this list.

204

: 00:09:03

There's still finance companies on

this list, manufacturing companies.

205

: 00:09:06

So don't just think that it's only tech

companies that are hiring data scientists.

206

: 00:09:10

Data roles.

207

: 00:09:10

Also quick note, it's interesting to see

that Meta is leading and hiring both for

208

: 00:09:14

the data scientist and the data engineer

position just because they did pretty

209

: 00:09:18

big layoffs like two years ago, year

and a half ago or something like that.

210

: 00:09:22

I think part of this was they just

overhired during COVID for different

211

: 00:09:26

parts of their company and now they're

kind of transitioning into an AI company.

212

: 00:09:30

We'll see how that goes, but I imagine

they're hiring a lot of resources

213

: 00:09:34

to do that and that's probably why

you see such a big surge in data

214

: 00:09:38

scientists and data engineers.

215

: 00:09:39

Um, but also Meta probably

just hires quite a bit as well.

216

: 00:09:42

Okay, takeaway number five, and

this one is my favorite, and that is

217

: 00:09:45

that data jobs are quite resilient.

218

: 00:09:48

This chart right here basically

compares data scientist, data

219

: 00:09:50

engineer, and data analyst levels to

the average white collar job levels.

220

: 00:09:55

Specifically, what we're looking

at is the percent of people who

221

: 00:09:58

are hired after leaving a role.

222

: 00:10:01

So basically, the higher

the percentage, the better.

223

: 00:10:04

Um, and what you can see that all

three of the data job families

224

: 00:10:07

are higher than the average white

collar worker, which basically

225

: 00:10:10

means that these jobs are in demand.

226

: 00:10:12

That means if someone in the data

family is laid off, they are more

227

: 00:10:15

likely to land a job quickly than

your average white collar worker.

228

: 00:10:18

Now that also could be true for if

they're switching jobs as well, which

229

: 00:10:22

just allows more career flexibility.

230

: 00:10:24

And like we talked about earlier,

job hopping usually means you're

231

: 00:10:27

making more money that way.

232

: 00:10:28

So to me, this is a great sign that

basically data jobs are quite resilient.

233

: 00:10:32

I think they're quite.

234

: 00:10:32

flexible and uh, no job is layoff proof

of course, but it does look like these

235

: 00:10:37

data job families are still very high

in demand and will allow you to quickly

236

: 00:10:40

land a job if you're laid off or if you

need to switch jobs for whatever reason.

237

: 00:10:44

With that, I hope you realize that

the state of data jobs is maybe not

238

: 00:10:48

as bleak as you thought it may be.

239

: 00:10:50

Things might seem grim but honestly

these numbers look pretty healthy

240

: 00:10:54

and I think we're in a good situation

and I think that situation will

241

: 00:10:57

continue into the next year as well.

242

: 00:10:59

Thanks again to Live Data Technologies

for sharing this data with us.

243

: 00:11:02

I'll have a link to them down below in the

show notes you guys can check them out.

244

: 00:11:06

And as always if you're looking for

another episode to watch I really suggest

245

: 00:11:09

this one right here or in the show

notes you can find that link as well.