These are the tricks of the trade to hiring the best workers

A decade ago, the Great Recession triggered massive layoffs as companies large and small scaled back operations or shuttered. The job market was flooded with applicants, and it wasn’t uncommon for employers to receive hundreds of resumes for a single posting. Desperate, many Americans were underemployed or accepted similar positions that paid significantly less than what they were making prior to the crash. Back then, a common refrain was, “Why can’t good people get jobs?” But with a strong U.S. economy and unemployment at 3.7% (as of July 2019), employers are now grumbling that they can’t find enough qualified workers. The question seems to have inverted: “Why can’t jobs get good people?”

Wharton management professor Peter Cappelli contends the economy doesn’t have as much to do with the hiring process as we would like to believe. Instead of relying on luck, companies should focus on trying to fix the hiring process by taking a more structured approach that is based on best practices, not subjectivity. They also should consider bringing back trained recruiters who specialize in hiring and know how to ask the right questions. Cappelli recently published an article in Harvard Business Review with the apt title, “Your Approach to Hiring Is All Wrong.” He spoke with Knowledge@Wharton about why companies are wasting time and money chasing down a cast of thousands when they should be “narrowing the funnel” to find the right fit. (Listen to the podcast at the top of this page.)

An edited transcript of the conversation follows.

Knowledge@Wharton: When it comes to hiring, what’s luck got to do with it?

Peter Cappelli: It certainly has something to do with it. It’s certainly easier to get jobs now, that’s for sure. And there’s not nearly as much attention given to people who can’t get jobs. It’s still the case, though, that if you look at the proportion of people in the labor force who don’t have a job and want one, it’s about 7.1%, which is a pretty big number when you figure the unemployment rate is 3.7%. It’s almost double that number, so there are still a lot of people who are unemployed and would like a job. How many of those people are impossible to employ? Well, that’s a pretty good question, and we don’t know the answer.

But remember that most employers aren’t interested in hiring people out of unemployment. In the U.S., we fill 66 million jobs every year. If you look at the number of unemployed, it’s a tiny fraction of that — it’s a few million. So, most jobs are filled by people who already are working for somebody else. That’s what every employer wants, right? They want to hire somebody who is currently working someplace else, ideally doing the identical job. In fact, the preference has been for people who are not even actively looking but, “Let’s go find somebody who seems to be happy someplace else, because maybe they’re a happy camper and we can hire them.”

I think the hiring process is still remarkably frustrating for candidates because, on the employer’s side, it’s still not very sophisticated and not very careful. For the candidates, it’s still pretty opaque. I think on the employer’s side, the question is always how much sympathy to have for them because they are in control of everything. They could spend more on recruiting. They could raise their wages and get better candidates. There are all kinds of things they can do, and as far as we can tell, the kinds of things they’re doing are not particularly sensible and haven’t been very effective for them.

Knowledge@Wharton: In both the recent HBR article and your 2012 book, Why Good People Can’t Get Jobs: The Skills Gap and What Companies Can Do About It, you discussed that there has not been a lot of effort by companies to look into the payoff of hiring someone that companies don’t really know. That’s pretty shocking if we consider what an ROI-centric and data-driven world we live in these days.

Cappelli: Even more shocking than that, employers don’t ask whether what we are doing is working, and most of them can’t ask because they don’t collect the data. There have been a few surveys of this, and only about 30% of employers say that they are trying [to figure out] or that they care about whether their hiring process is giving them good candidates and good hires.

What they care about is time to fill a vacancy and how much it costs to fill a vacancy. That’s what is measured. If you think about this cynically and say, “OK, let’s see if we could judge restaurants this way?” MacDonald’s would be the best restaurant in the country, right? They get it to you fast. It’s pretty cheap. That’s what most employers are doing. They’re looking at cost per hire, time to fill, and they’re not looking at whether our practices give us good candidates or not.

Imagine if you’re trying to learn to play golf, and you go to a practice range and can never see where the ball goes. Let’s say you’re practicing in the dark and they don’t turn the lights on. You can imagine that you’re doing all kinds of really cool things that are really producing great hits down the field. But, actually, you’ve got no idea. There’s a pretty good chance that you’re not doing very well, and that’s the predicament most employers are in right now.

They’re doing all kinds of crazy things, but they’ve turned the lights out and can’t see the results, and they are not paying attention to even the things that we know would lead you to better hires.

Knowledge@Wharton: What are some of those things?

Cappelli: I think the biggest thing that predicts whether somebody is going to be a good candidate or not is what they have done before. Past experience, past behavior is still the best predictor of future behavior. And they’re not spending an awful lot of time to try to figure out what did somebody actually do? Specifically, what did you do in your last job, and how did you do it? They ask them sometimes, generally at a superficial level, but often they don’t.

The craziest thing that we do is spend all this time interviewing people in an unstructured way with each interviewer asking whatever they want. It appears that the amount of time it takes to fill vacancies is going up, and some economists think that that’s because it’s harder to find candidates. But the evidence suggests that we have doubled the amount of time we spend interviewing, which is typically about the worst way to hire people. Because unless you’re trained as to how to interview, the questions you’re asking probably aren’t predicting anything. What they probably are doing is leading to demographic biases.

Knowledge@Wharton: Interviewing someone is a highly subjective process, correct?

Cappelli: It doesn’t have to be, but it especially is if everyone is picking the questions they want to ask, and no one else is asking the same thing. If you’ve got five people interviewing somebody, and they’re each asking different questions, they could come to wildly different conclusions about the same candidate.

We don’t train people as to what to ask. The questions that matter are really all about past behavior. The problem with that is most candidates know what the typical questions are, and they’ve got canned responses to them, so you’re not learning anything from that. If you look at the people who do this well, like the executive search people who do it for a living, they spend a lot of time before they interview you figuring out what they’re going to ask by learning what you have done. They’ve talked to a lot of people about you. They know the projects you’ve worked on. They know the ones that worked well and the ones that didn’t. They can zero in on detailed questions and go deeper to try to find out what really happened. They’re more able to get at the truth because they already know a lot before they start to interview you. I don’t know anybody who does that, frankly, at the hiring manager level.

Another problem is that we’ve given up on sophisticated recruiting. A lot of companies have gotten rid of their recruiters, others have outsourced it, so we’re basically requiring first-line managers or the hiring manager to make the calls on whom to hire. They don’t hire very often, they don’t have training to do it, and we don’t check afterwards whether they’ve done a good job or not.

Knowledge@Wharton: Just because someone is a hiring manager doesn’t mean they know how to write an effective job description or conduct an effective search. That’s a highly specialized skill, right?

Cappelli: Right. We also never give them any feedback. Here’s the first problem in hiring: Typically, if I’m a hiring manager, I figure out what I think this job requires. Then I ask everybody else around me who might work with this person, “What would you like to see in a candidate?” But then we never whittle that down. We just add it all in, and you end up with these job requirements that are impossible to fill.

Knowledge@Wharton: It’s like a Christmas tree with too many ornaments.

Cappelli: A Christmas tree, the purple squirrel or the unicorn candidate, right? They just don’t exist. The joke is, “We want you to have three years’ experience in a programming language that doesn’t exist yet.” You end up not being able to find anybody who could fill it. Or you don’t have the money to pay to get somebody at that price point who has all those attributes.

Knowledge@Wharton: I find it ironic that companies often ignore internal hires. They’re not looking at people they already know in terms of their skills and how they function in the company culture.

Cappelli: I think that’s exactly right, and I think it explains the complaint from employers that hiring is really, really difficult. One of the reasons it’s really difficult is they’re just doing so much of it. And the reason they’re doing so much of it is because they’re losing people out the back door all the time. This is like bailing out a sinking boat: You’re complaining about the size of the bucket you’re using and not looking at the hole in the boat, which is really the problem.

If you’re not promoting people from within, that means we’ve got to go outside to fill that vacancy, and we’re doing it for all the jobs in the company now. So, we have to understand hiring in all these different labor markets, which is really difficult to do. The other thing we know is that if you don’t advance people from within, they leave. The No. 1 thing right now that job seekers say they’re looking for is career advancement. If you are not giving it to them in your own company, they’re going to leave.

Now, we can’t keep everybody, and there will be some people who are more valuable elsewhere than they are to you. But if we aren’t trying to fill any of our positions from within, we’ve got a big hole in the bottom of our boat, and we are never going to be able to bail ourselves out.

Knowledge@Wharton: We have heard a lot in recent years that the old model of starting with a company out of college and staying until retirement has gone away. But I have to wonder if we’ve created a chicken-and-egg problem for ourselves. We’ve decided that must be true of everybody, even though some people may want to stay with the company for the long haul.

Cappelli: That’s a great point, and it explains something broader: It’s not that any organizations have thought this through, looked at the numbers and said, “This is what we’re going to do.” They’re going with their gut, and they’re typically going with the gut of the people at the very top.

You’ll hear CEOs sometimes say, “We can’t afford to train people because they’ll just leave.” Well, everybody leaves at some point, so the simple idea that people might leave is probably not a sufficient reason to say that we shouldn’t train. I think the big problem is that no one is thinking about this critically, no one is looking at evidence to see what works. They’re just going with their gut on things. And that’s proving, not surprisingly, to just not work.

Knowledge@Wharton: Do you feel like there’s been a re-thinking about the value of human capital in recent years? You used to hire someone and invest in them. Now, many companies don’t feel like employees are worth that investment.

Cappelli: Yes, and I think there are some good reasons for that. I suspect, in a lot of companies, we were not holding people accountable — particularly in large corporations, we were not thinking carefully about what they were contributing. We had developmental opportunities that were incredibly expensive. Look at rotational assignments, for example. They’ll move you and your family to Europe for three years so that you could learn something about Europe. Well, you’re never going to produce much in that time because the first year or two, you’re just figuring out what’s going on, and then we move you back. That’s incredibly expensive.

It is probably true that we were overspending on some of these things, but I think the idea that we shouldn’t do any of it has become quite popular. I think that’s partly a human resources problem in that human resources didn’t try to defend against the onslaught of cuts. The new approach, which began with the shareholder value movement, was to say, “Let’s cut if we can’t see that there’s any value there.” That is not a crazy way to go about things, but the HR people were not able to show, “Hey, this really pays off.” If you want to test this out, go into your own organization and ask them this most basic question: How much does it cost us if somebody quits? And see if you can get an answer. If all you get is the hiring cost of replacing them, they’re not paying attention because the cost is hardly just that. The big cost is leaving the position vacant, bringing in somebody new who doesn’t know what they’re doing, waiting for them to get up to speed.

Knowledge@Wharton: Wharton management professor Matthew Bidwell has research that shows it takes three years for an external hire to get up to speed.

Cappelli: Yes, it takes three years for new hires to produce as well as somebody who is promoted into the identical position. And it takes seven years, he found, for somebody who is promoted from within to make as much money as somebody hired from the outside. So, the hiring costs per se are trivial compared to those costs.

Knowledge@Wharton: Since your book in 2012, or looking even farther back, has there been a significant change in what employees want out of a job? What’s most important to them?

Cappelli: We all probably know this story about the hierarchy of needs. First, you’ve got to eat. In 2012, the unemployment rate was really high, and a lot of other people had given up looking and were sitting on the sidelines, waiting for jobs to come back. If you could get any kind of job, you were probably delighted to have it. As the labor market gets tighter and there are more jobs, people can be pickier. I’m not sure that what they want is so different, but their ability to get things is going up.

But it hasn’t gone up as much as people think. For example, wages really have not gone up by much. Real wages – what our pay check can buy – are up but only 1.5 percent over the past year in what is supposed to be the best job market for employees in a generation. They are still well below where they were in the 1970s: It is stunning to think that the average worker got paid more 50 years ago than they get now. Efforts to improve work/life balance, which is the other thing candidates say they want, is still mainly still talk. Maybe they’re trying to do more on flexibility — you can work any 80 hours per week you want — but they’re not doing anything to cut back the total demands or even to help people work smarter. I don’t think what people want has changed so much now, but they can demand more. In some places they’re getting it. IT workers and those with technical skills see wages going up and employers constantly trying to do nice things for them, especially in Silicon Valley. But that is a small slice of the economy.

Knowledge@Wharton: Are you seeing generational shifts? Millennials are becoming a bigger part of the workforce, and Gen Z is starting to enter the workforce. Do they want different things?

Cappelli: I don’t see any evidence that there are distinctive goals and values among these different cohorts, whatever labels they have been given. As far as I can tell, no demographer has ever claimed that there are. We know younger people are always different than older people because of their stage in life, but we somehow think that the younger people today must somehow be different from what older people were at that age. Some of that is because older people don’t remember accurately what they looked like to their parents when they were younger. The concept of “Millennials” lumps together young people in their 20s with those who are pushing 40. I don’t see that they have much of anything in common. The younger ones are just getting started in adult life; the older ones have homes and kids, so they want different kinds of things.

If you ask Baby Boomers, “Did you want work/life balance when you were entering the labor market?” they’ll say, “Yeah, of course we did. We just couldn’t get it. If you asked, you’d get shown the door.” Now, you can get it. So, it’s not that they are different; it’s because the context is different. It’s the fundamental attribution error in psychology, which means that when we look at people’s behavior, we are inclined to assume it is because of who they are, rather than what’s going on around them. That’s a lot of what’s going on here.

Knowledge@Wharton: Technology is a big piece of the hiring process, and it factors into questions about why people are struggling to get jobs. When I was in college, I was encouraged to call recruiters and try to get informational interviews, things like that. Now, there are a lot of technological barriers that keep job seekers from talking to hiring managers.

Cappelli: Yes, that’s a big change, although the advice you get from people is still the same: See if you can get around the technology and find some person. It’s just harder to do now, right? That’s why so many people talk about networking to see if you can get an introduction to somebody who’s behind the curtain.

In the 1990s, employers were really desperate for hiring people. Wages were going up and it was, by all accounts, still a tighter labor market than it is now. When that happened, companies tried to make it much easier for you to apply because they wanted as many applicants as they could get. I’m not sure that was such a smart thing. Everything could be submitted electronically. Then they got software that would scan applications. Applicant-tracking software is what this became known as. It’s still there, so now it is so easy to apply. You might think that employers today just don’t get many job seekers, but they get tons of applicants. I think the evidence is that, if you count people who go to a typical website to hunt for a job, about 2% of those will ever get an interview. Estimates from employers say that they still get an average of about 50 applications per job, and that doesn’t count the “passive” candidates that recruiters find.

It is so easy to apply that the companies have had to put up these technological screens as a way to weed them down. The problem is that most people still think it’s not a particularly great way to do it. There are ways that you can game the screening with keywords and stuff like that. So, we’ve got a problem. It’s another reason why you could be a particularly good candidate who just doesn’t fit the keywords perfectly and never gets through those screens unless you can somehow explain yourself to a real human.

Knowledge@Wharton: I think a lot of companies would say they want nontraditional candidates, people who may not be an obvious fit for the job. But isn’t it hard for those candidates to get through that software?

Cappelli: I think that’s right, and I think what you’re hearing is aspirational. It’s also true that even at the vice president level, they’re saying that stuff. But down at the recruiter level, they’re still being assessed based on how well resumes match these job descriptions. And in order to assess whether somebody is really a good fit, even though they’re different, it takes a lot of eyeball time.

Knowledge@Wharton: How can companies deploy this technology better?

Cappelli: One thing that is already happening, and the reality of it is not quite up to the promise, is the application of data science to the hiring process. This is using machine learning. Machine learning is a technique for finding associations — patterns — in data between the attributes that you’re interested in and the outcome that you’re hoping for. What attributes of job candidates relate to, or seem associated with, good performers versus bad performers?

There’s pretty good evidence that, on some dimensions, these algorithms might be better at assessing candidates than a typical human might be because they don’t have the biases. If they’re screening an applicant, for example, they’re not going to be impressed by how they look because they don’t see them. There are all kinds of things that affect humans that you can be sure are not going to influence the algorithm, so there’s some promise here. There are also some big risks. When algorithms make mistakes, they make mistakes at scale. They make them for thousands of people because we are applying the algorithm to hundreds and in some cases thousands of applicants. There’s also more litigation risk because if you make mistakes at scale, somebody is going to find it. If you’ve got 100 recruiters and one of them is completely biased, it’s going to be very difficult for plaintiff’s attorneys or the EEOC (Equal Employment Opportunity Commission) to find out what’s going on with that one person. But if your algorithm is biased against some demographic group and you’re doing it to thousands of people, somebody is going to find that, and you’re on the hook.

Knowledge@Wharton: In the HBR article, you point out that companies have this interesting legal relationship with an applicant. They can’t be biased, and there are certain questions they can’t ask. I don’t think a lot of us think about that when we press the button on LinkedIn to share our resume.

Cappelli: The employers have a lot of legal obligations to applicants, and it also costs them money to process all these things even though they’re trying to do it cheaper and cheaper — which is why my advice is that you don’t want to just try to get as many applications as you can, because we’re not so good at screening out who’s good from bad. Every one of them exposes you to liability, and every one costs you money. You’d really like to get a narrower funnel of people to apply who really fit. That requires being more accurate about what you really want, not promising people things that are not true, and telling them the truth about things that not everybody is going to like. “Yes, you’ll have to travel a lot for this job.” What does “a lot” look like? “Well, four days a week.” If you don’t tell them that and just say “travel required,” they might apply and say, “Oh, I can handle that,” until they discover what it means and quit. If they know it’s four days a week, many of them won’t apply if they think that’s impossible for them. Then you’re weeding out people who would have been bad hires for the job.

The thing to remember is, the worst thing you can do in hiring is not leaving the job unfilled, it’s to hire somebody who is going to fail. They’re going to mess up. You have to get rid of them. Then you have to replace them. It’s like putting a bad part in a machine. It’s a really costly thing to do, and we’re not paying enough attention to that.

Knowledge@Wharton: It’s clear that using technology for hiring people is probably not going away, but what parts of the hiring process really need the human touch? Where is that most important?

Cappelli: We really need to look and see what’s actually working for the employer. One reason is because the defense against getting sued for discrimination if your hiring practices have an adverse impact against certain classes of candidates is that you can show that your practices actually work. If you can’t show that, you’ve got a problem. If some vendor claims they work, that doesn’t help you very much.

Where do we need a human touch on this? Human touch is pretty expensive, so the prior question is how much money do you want to spend on this? The problem is we’re trying to drive that number to zero without considering that it’s costing us an awful lot to drive that number to zero, in terms of longer time to fill, bad hires. We could save a lot of money by making better hires, and we could also save by doing fewer things that don’t work, such as irrelevant interviews conducted by executives whose time is very expensive.

We’re drowning our line managers in time spent on hiring. They’re not good at it. They don’t have the time. They don’t know what they’re doing. Can’t we give that back to somebody who has some expertise in this? They can do it a ton faster, and they can do it a lot better. I’d say that’s the human touch: Use the time of people who are experts to do fewer but deeper interviews.

Knowledge@Wharton: I’d like to end with the same question that we ended with when we spoke with you back in 2012. What would your advice be to job seekers today?

Cappelli: Well, it’s a whole lot easier. That’s a good thing. Candidates can get much much better information about jobs now Most people believe that sites like Glassdoor and Payscale give you more reliable information than you would get from an employer about working there. I think you can also negotiate more for things unique to your needs than you could have in 2012, so that’s probably all for the good. Employers are much more likely to accommodate things like working from home. I think the advice of trying to get around the software to learn about jobs and pitch to recruiters or hiring managers is probably still true, if you could do it. A lot of jobs are still filled without posting the vacancies, particularly in smaller organizations. Trying to figure out how to get to somebody who might give you a more careful read, rather than simply going through the applicant-tracking system, still seems like pretty good advice. But now the odds are much better that you’re going to get a payoff. You probably don’t have to apply to 100 jobs to get one bite. Now, the odds are much more in your favor. I also think it might be much easier now to negotiate with your current employer for opportunities rather than have to leave to find them, and that’s a very good development.