906.226.2829   |   Client Login

Choice-Supportive Bias Gives False Confidence to Your Hiring Decisions

By Wendy Beach, VP of Talent Science Solutions at Stang Decision Systems

Spoiler alert: here comes some real-life science to distract you from the daily news! In all seriousness…there is beauty, I’ve found, in the science behind what helps our company match people to jobs and jobs to people.

Because it’s good to understand why we do things a bit differently, this blog on choice-supportive bias is our first in a series of discussing the relevant cognitive biases which prevent many people from making good hiring decisions.

Once we make a choice, we have the tendency to support that choice (dig in our heels, close our minds, stick our heads in the sand…insert your favorite saying here) even if it doesn’t make sense in the face of new information. Industrial psychologists call this choice-supportive bias, and it is one of many cognitive biases that affect our decisions.

I would argue that choice-supportive bias has become even more significant as our world has become more competitive and information leading to constant decisions is available at an ever-increasing pace.

What!? Isn’t it a contradiction to say that in an increasingly fast-paced and competitive business world the “status quo” is becoming even more ingrained in our personal and professional lives? I’d argue we are living in a world of “revolving status quo.” In other words, we try even harder to maintain normalcy and consistency in certain parts of our lives because other parts are changing so quickly.

The two main reasons for this are: (1) the increase in disruption via information and technology available at any moment; and (2) choice-supportive bias. The information and technology pushes us to change, and choice-supportive bias pushes us to lock in previous choices. The conflict between the two leads to pockets of rapid change and pockets of locked-in behavior. Oddly, this implies that rapid change in many areas of business run parallel to systems that may be archaic or obsolete.

Anyone in business, from manufacturing to service, has every moment of the day filled with constant choices and decisions. Since our company is focused on talent, I’m going to simplify this conversation (whew!) and emphasize the importance of resisting choice-supportive bias when attracting, retaining and developing talent. Specifically, I’m going to use the resume as a shining example of how cognitive biases can ultimately reduce decision accuracy.

“Eyeball to eyeball and a handshake.” Does anyone else feel a bit of nostalgia for the times when this was the way you decided if a person was a good fit for your team? In the 1940’s, believe it or not, the resume was a new trend, and it typically included weight, age, height, marital status, and religion (not very PC!). It became popular as companies grew and roles became more skilled or specialized. Resumes also helped as workers had more types of work experience and more geographic mobility. For a time, this was an improvement that made a lot of sense.

The reality of today, however, is quite different. In this fast-paced world, most people do not want to spend hours crafting the perfect resume. What if a good candidate simply doesn’t use the right keywords, or has a job title or education that doesn’t describe his or her full skill set? What if some candidates pay graphic artists and editors to make their resumes shine in a way that others never will?

From the perspective of the hiring manager, who wants (or has time) to spend hours sorting through hundreds of resumes–or worse–resumes with cover letters? Hiring managers and human resources staff are often faced with mountains of paper and/or electronic documents in varying formats, fonts, and length. Nonetheless, thanks in part to choice-supportive bias, the resume has become the default tool for evaluating candidates. Resume, cover letter, status quo, check.

When a “new” idea came along in the not-too-distant past to allow technology to help us with the tedious task of going through resumes, we didn’t start from scratch and look at what computers and data could truly do to elevate the process. Instead we, as a society, collectively held on to the “status quo” while adding an electronic layer. Our “choice-supportive bias” resume protocol was still intact, and the document was simply stored on a computer.

Resume parsing, keyword searching, arbitrary cut-offs of grade point averages…those were all born of using a computer to get through big stacks of documents faster. Did that increase our odds of a great fit? Actually, even though electronic applicant systems were perceived to increase efficiency, they quickly become talent pool limiters, and often increase inefficiencies when it comes to finding the most suitable candidates for a job. There are many internet forums that focus on how much candidates hate applicant tracking systems, and internal human resources staff tend to agree.

What if we used computers and data to help us get to know people better as a first step in a hiring process, not simply weed them out? Can computers really help us be MORE human?

You’ve probably guessed by now that using technology to make the hiring process more accurate and more “human” is our focus at Stang Decision Systems. Our clients are bypassing resumes altogether and working with new ways to see their candidates as a complete person. Curious how? We’d be happy to show you.

When the resume step is eliminated, we start to wonder why we held on to it for so long. Could it be that we’re simply human, with some pesky choice-supportive bias? I think so, but I’d love to hear other theories.

As with any limiting quality, we should not expect to be other than human, or beat ourselves up over it. What we can do, as intelligent decision makers, is become more aware of our biases and search for ways to overcome them. The human brain has a wonderful autopilot mode, but when decisions are truly important, we need to take the wheel and steer toward better options. That is truly progress beyond the status quo!

By |2018-03-07T16:37:20-05:00February 27th, 2017|Research, Updates|1 Comment

Sports Analytics Field Trip

By Scott Birkeland, Ph.D., Vice President of Stang Decision Systems

I recently attended the Midwest Sports Analytics Meeting in Pella, Iowa. Thanks to Russ Goodman and Central College for putting together fun and informative sessions. This conference not only gave me an excuse to hang out with my former college roommate (St. Thomas math professor Eric Rawdon), but it also allowed me to see, first-hand, some of the cutting-edge research that is taking place in the field of sports analytics.

scott1

Eric Rawdon and me at the Central College entrance (I’m on the left).

During the conference, I attended a variety of presentations. Some of the topics included measuring how teams deliver value to fans, an analysis of strike zone errors in MLB, ways to differentiate offensive explosiveness vs. offensive efficiency in college football, software that helps coaches create data-driven practice plans, and several talks that used NFL play-by-play information to analyze tactical decisions (e.g. win probability at various points in a game, fourth down decisions, field goal accuracy, etc.).

All very interesting and thought provoking topics (at least to me!).

As I reflect on the conference, one of the big takeaways I have relates to decision-making and why it is that those involved with sports teams often don’t use the information that is available for them to make optimal decisions. An example that was frequently discussed during the conference is the debate in football about “going for it on fourth down.” Historically, coaches have preferred to punt on fourth down, even though, in many situations, the data suggests that they shouldn’t.

With all the analytics that are now available it is surprising how often teams go against what the data say they should do. This is true for more than just “going for it on fourth down.” It is true for drafting strategy, negotiating player contracts, preventing injuries, and developing optimal practice plans–just to name a few.

Throughout the conference, I had several discussions with folks around why teams do not consistently use analytics to their advantage. A couple of reasons were repeatedly mentioned.

First, people who are coaching and/or managing teams often have a sports background, but not a math/statistics/analytical background. Because of this, they do not necessarily understand the methodology behind the numbers. And, given their role as a leader, they must be able to convince their team why they are taking the actions they do. If the leader of the team doesn’t understand the data or how it was generated, it becomes more difficult to inspire the team to act based on that information. Coaches tend to rely on what they know best, which often means doing what they’ve always done and not using analytics.

Second, many pointed out that because coaches’ decisions are so closely scrutinized (especially in major professional sports) that if they make a non-traditional decision, they leave themselves open for significant criticism from fans and the media–even if the decision, from a data analytics perspective, is the correct one. Therefore, it is often easier and more comforting to make decisions based the way it has always been done rather than do something different. As one person put it, “it is hard to get overly criticized for doing something that everyone has been doing for the last fifty years.”

From a coach’s perspective, I can appreciate these reasons. At the same time, I also realize that it is important to utilize any advantage that you might have, even if it is something that might stretch your comfort zone. I believe that we all need to question the way things have been done (whether you are working for a sports team, a Fortune 100 company, or a mom and pop company) to see if there is a different strategy that gives you a greater likelihood of succeeding.

At its core, that is what data analytics does. It allows end users to gain competitive insights. Oftentimes, these insights contradict conventional wisdom. In my view, this should be viewed as an opportunity rather than a threat!

By |2018-03-07T16:37:20-05:00December 1st, 2016|News, Research|1 Comment

Should you be hiring transformers?

Here you are, in need of help. Someone left a key role to move to another position elsewhere, there was an unfortunate sickness, or you earned a big new client. Or, you may have become aware of a gap or opportunity, and you want to find just the right person—or people—to join the team.

When the decision to search for new employees is made, or made for you, you probably don’t want to spend a lot of time or effort on HOW you hire; you just plain want to HIRE!

However, you do know that the cost of a “wrong” hire is huge. Each “failed” employee search costs tangible major dollars in training, lost productivity, and time to fill…not to mention intangible costs such as low morale, lost opportunities, and the toll on your employer brand. It’s scary when a new employee “transforms” into an adversary soon after you put in the effort to find and train them to be in your world. So, what can you do to find a better match right away? That person who, in all of their forms, is just right for the job? Do you have to learn and apply new terms and buzz phrases to find your new “heroes?”

One term we intuitively use at SDS and help our clients use by working with us is “transferable skills.” Sometimes you’ll hear “transformative skills.”

These are “buzz phrases”…but a buzz often starts when something has impact. There are many interpretations and definitions out there for transferable skills. Here’s one simple definition I like from www.businessdictionary.com:

“Aptitude and knowledge acquired through personal experience such as schooling, jobs, classes, hobbies, sports etc. Basically, any talent developed and able to be used in future employment.”

Most hiring managers agonize when thinking they should consider these skills as they’re searching for that “needle in a haystack” perfect new employee who can do anything. They build lists of potential transferable skills, look online, research endless job descriptions, and honestly, go in circles. If you Google the phrase, you’ll see career coaching lists helping people build transferable skills, lists of military to civilian transferable skills, and countless ways to help people understand how to better search for a match which leverages a potential employee’s different work and life experiences.

Sounds difficult and exhausting, doesn’t it? To think of, and figure out how to search for, the transferable skills which might work for your crucial role can be time consuming, inexact, and worrisome.

Another way to consider and cover transferable skills is through customized talent science. If you work with a system, like ours at SDS, which has been capturing and leveraging transferable skills for over 15 years, you’re all set.

The concept of transferable skills allows for a person’s aptitude, behaviors, and outlook to factor in to their probability of being successful in certain jobs. It’s a little bit like that phrase “wherever you go, there you are.”

By including transferable skills, you are able to cast a much wider net when recruiting, which dramatically increases your odds of finding the right person. It widens your candidate pool! One of the problems we often encounter at SDS is companies getting too focused on finding a person with the exact experience they are seeking, and, as a result, they end up excluding many people who actually have the ability to do a great job for them. They actually narrow their pool and miss out on high potential candidates, simply because some work histories don’t precisely fit what they were expecting to see.

At SDS, we have many examples of helping companies hire people who, at first glance, appear to have no business even applying…but after assessing for their specific transferrable skills, we learned that these individuals could succeed in the jobs. Some of the most successful operators our oil refinery clients have hired actually had their main work experience in surprising areas such as the fast food industry.

The takeaway here is that many terms in the world of talent seem more complicated than they really are. Transferable skills are assets almost any applicant will possess—if you customize how you look for and understand them. We take care of that by working closely with you on what you DO know—your business—and using what we know to apply the talent science which “highlights the human” in human resources, and applying the full-spectrum talent fit that includes transferable skills.

Cars that turn into warriors, or villains, need not apply.

By |2018-03-07T16:37:20-05:00August 3rd, 2016|Careers, Research|2 Comments

How’s the fit? Is your selection process resulting in “good” hires?

As I’ve talked about in a previous blog post, a primary goal that drives our work at SDS is to achieve a 90%+ selection process accuracy rate. In other words, we want more than 90% of the people who are hired through our tools to be considered “good” hires.

In order for us to determine whether we have met that goal, we need information about how well these people are performing on the job. However, collecting information about people’s on-the-job performance can get tricky. It’s not always easy to determine with confidence who is a “good” hire and who is not. There are many questions we ask ourselves which help us to trust our performance evaluation data:

  • What types of information do we use to determine whether or not someone was a “good” hire?
  • Whose input do we consider to be the most useful?
  • How is this information collected?

The answers to these questions are critical factors which help us when determining if our selection process is resulting in “good” hires. I’m going to focus this blog on explaining some of the most critical factors we consider when collecting performance ratings.

CONTEXT MATTERS

When supervisors are asked to make evaluations on their people, the context in which these ratings are gathered can dramatically influence the true accuracy of the evaluations. For example, if a supervisor is asked to make evaluations that will be used to determine pay and promotion, the supervisor may rate someone differently than if they were evaluating a candidate for purposes of determining future training. In either of these cases, there are motivations that affect how a supervisor evaluates their subordinates that may have little to do with making precise, accurate evaluations. These motivations may include factors such as maintaining the morale of the supervisor’s team, motivating people to improve in the future, or ensuring subordinates receive pay increases.

Because of these motivations, we prefer to not use ratings obtained within an organization’s performance appraisal system. Instead, we prefer to collect our own ratings with a survey that is ONLY used for purposes of evaluating the selection process. We let evaluators know that their ratings of each individual will not be seen within the organization, and will be used for research purposes only…which ultimately leads to continuous improvement in their selection process. By doing this, it allows supervisors to focus on making accurate evaluations of their people without having to worry about the implications of these ratings.

PROCEED WITH CAUTION WHEN USING OBJECTIVE CRITERIA

For many jobs, people are evaluated by objective criteria, at least in part. For example, a production worker in an assembly plant may be evaluated by number of products assembled; a police officer may be evaluated by number of tickets written; or a car salesman may be evaluated by number of cars sold. On the surface, these criteria seem very reasonable. However, as Borman points out, there are several reasons why objective criteria often do not accurately reflect someone’s true performance on the job.

First, objective criteria may only reflect a small part of one’s job (e.g. the number of tickets written by police officer). Second, when using objective criteria, someone’s performance is often dependent on factors that are outside of his/her control (e.g. production worker is dependent on many others when assembling products). Finally, the numbers obtained from objective criteria might be difficult to evaluate. For example, a car salesman at one store might sell the same number of cars as a salesman at another store, but the market for cars at these stores could be very different.

Ultimately, when using objective criteria as a performance measure, you need to diligently research how these measures are obtained in order to ensure that the information truly reflects the person’s performance.

SUPERVISOR’S PERSPECTIVE DOESN’T ALWAYS TELL THE WHOLE STORY

When collecting performance ratings, the supervisor is often considered to be the person who is in the best position to make these evaluations. However, there is often value in collecting ratings from sources in addition to the supervisor. As many jobs are complex in nature and include working with people at all levels of an organization, ratings from sources such as peers, subordinates, customers, etc. might capture information about someone’s performance that isn’t reflected in supervisor-only evaluations. By collecting ratings from multiple perspectives, a more comprehensive portrayal of employee performance can be obtained.

Clearly, there are many issues to consider when collecting performance information. While the purpose of this article was to introduce some of the most common issues that we often wrestle with, we realize that each organization and each job has its own factors that need to be considered. As such, we realize that there is no perfect measure of how someone is performing, which in turn can make it difficult to determine who is a “good” hire. What we do know, through experience, is that following best practices for collecting performance ratings can greatly enhance the accuracy of this information.

By |2018-03-07T16:37:20-05:00July 12th, 2016|Research, Updates|1 Comment

6 Signs Your Employee Selection Process is Broken

In this blog series, Spencer Stang, PhD, discusses why organizations fail to hire better employees, and gives insight into what companies can do to make better hiring decisions.

Occasionally, conflicts arise between being honest and making somebody upset, or not being honest and keeping people happy. My rule of thumb in these cases is that truth trumps tact. I would rather have somebody tell me the truth rudely than to have him or her pass along a polite lie.

I’ll try not to be rude, but the truth is that your employee selection process is almost surely broken. You may have put a great deal of time and effort into your process, and you also, most likely, know that it still isn’t where it should be. Here are six of the most common reasons your process is broken, and what you can do to correct them. As with all generalizations, some exceptions apply.

1. You Require Resumes

For most openings your best candidates aren’t in job hunt mode. Many of the people you would want to hire don’t have a resume and won’t bother making a resume based on a job posting–no matter how enticing. The first step of the selection process needs to be so easy that the curious candidate just falls into it. People always tell us, “I wasn’t really looking for a job when I saw the posting.” These are the people you want to attract!

Clients will sometimes say they want the application process to be challenging (e.g. send resume to apply) because it eliminates people who are too lazy to make a resume. There is some truth in this notion, but it is misplaced at the beginning of the process. The employee selection process is a two-sided relationship, and before you ask anything significant of a candidate, you first need to prove yourself to him or her.

In other words, show the applicant that the job is real, the company is real, the opportunity is real, and demonstrate that he or she will be treated with respect. Once you have established yourself, then it’s okay to ask applicants to do any number of assessments as part of your due diligence, and the applicants will understand and respect the process. Bottom line, you don’t want to ask too much, too soon, and you always want to treat candidates as you would want to be treated yourself.

2. You Immediately Make Candidates Create a Username and Password

Imagine that I tell you that I hold the secret to success and happiness. I go on to say that by following three straightforward rules, you are statistically guaranteed to be more successful and happy than the average person. All you have to do to learn these rules is create an account with a username and password . . . and the username is your private email. You can imagine that only a small percentage of people are going to create the account, because it is likely to be a waste of time. What if instead of making an account, you only had to scroll down the page to read the rules to happiness and success?

In this case, most people would scroll down–if nothing else for the sake of curiosity. Furthermore, after you read the rules, if they actually made sense, and they had a basis in research, the credibility of the source would go up significantly. Essentially, the more you prove yourself, the more credibility you build, the more information, time, and money a person will trade with you.

Respect must be earned, and immediately asking a person to create an account is not a way to earn respect. To see this in your current process, look up the number of people who click on your job posting and compare it to the number of people who actually apply. Most companies get fewer than one in ten people, and half of that loss is due to the “create an account” initiation process.

3. You Don’t Communicate Consistently and Honestly 

Starting with the basics, if a person applies for a job with your organization, you should tell that person if he or she is no longer under consideration, and/or if the job is filled. If you don’t have the time to do this, then you don’t have the time to run a hiring process.

Note that in your communication process, it’s okay to tell candidates that the process is taking longer than expected, or that it has been put on pause for a time. Research suggests that when you don’t say anything, most people will actually assume that something is worse than the truth–so lean towards honest disclosure without getting into the gory details.

Finally, unless you work for the CIA, never lie. Don’t lie to make a candidate feel better or to cover for a mistake made by somebody on your staff. Don’t lie to keep your company from being sued. When you lie, it leads to a culture of lying that will ultimate hurt your HR team and organization.

4. You Collect Data on Candidates Who Have No Chance of Being Hired

In the era of Google and Facebook, data has never been more valuable. It’s so valuable that many organizations use morally questionable methods to get people to provide them with their personal information. Once you know that a person will not be hired, you should stop collecting data on that person as soon as it’s reasonably possible. Do NOT use the hiring process as an excuse to collect data that you or your applicant tracking system vendor thinks may be useful or valuable later. This is a creepy practice that is used by far too many organizations.

Most organizations with applicant tracking systems ask knockout-type questions as part of the process (e.g. are you at least 18 years old? Can you work in the U.S.? etc.). Some of these processes ask these questions after candidates have already provided an extensive amount of personal information (which is bad). Some ask these questions first, but then go on to collect personal information even if the applicant doesn’t meet the minimum qualifications (also bad).

By asking the knockout questions first and then eliminating people who don’t meet minimum requirements, we can respect each applicant’s time and personal information, and improve the legal defensibility of the entire process. The only thing that doesn’t happen by using a more respectful process is you don’t get to store gigabytes of personal data on unqualified applicants, that you are probably never going to use anyway (unless you actually are Google or Facebook).

5. Your Process is Not Customized to the Job

This is an easy one to spot. If you use the same selection process across your organization, then your process is definitely broken. To state the obvious, accountants, programmers, machinists, and programmers are all different. Asking applicants for different types of jobs to go through the same process either because it’s simpler, or because “corporate wants a standard process,” is not going to get you the accuracy rate your organization deserves.

It will also feel generic to applicants, who then may not take it as seriously. Your application process should ask different questions based on the job. If you use personality assessments (and you should), then you need to match the personality profile to the job. Additional assessments, including job knowledge testing and basic ability testing (math/reading) as well as the interview also need to match to the role.

Furthermore, the intensity of the process should vary based on the consequences tied to a bad hire. If you are hiring a maintenance supervisor at a refinery, the consequences of a bad hire could literally be life or death. In this case, it makes sense to put more time and effort into your process than you would if you were hiring a ticket taker at the local theater.

6. You Think Hiring is More an Art Than a Science

Over the course of my career, I’ve had many people tell me that hiring is “more art than science.” Typically, this happens when a person is about to hire somebody who has a relatively low probability of success on the job. In other words, it’s a catch-all excuse for not following decision science. I know this well because I’ve made the mistake myself. The very first person I ever assessed for a client interviewed extremely well, had a perfect background for the job, but tested very poorly (i.e., big red flag!). The client dismissed the test results and I went along with their assessment noting, “the tests aren’t perfect.”

Within six months, the new hire crashed and burned, and cost the organization a great deal of money. In hindsight, I should have known better, but I got caught up in the “art” of the process. My gut told me that this person was going to be great in spite of his test scores. I knew that statistically the tests were just as accurate as the interview process, and yet I put unwarranted weight on my own observations. Had I factored the test results appropriately, I would have made a different recommendation–the right recommendation.

This, of course, is just an anecdote, and anecdotes make for bad science. After having the opportunity to track hiring “exceptions” on a much larger scale, we have solid evidence that when it comes to predicting future job performance, science trumps art. When you hire someone who goes against the science, you are four times more likely to be hiring someone who is going to fail than if you hire someone recommended by the process (science).

In other words, you shouldn’t ignore your gut, but you also shouldn’t trust it. If you want to put probability on your side, your process should include properly validated and weighted measures of education, experience, soft skills, technical skills, aptitude, character, and personality fit. If you “feel lucky,” you can skip all that and opt for short cuts and trusting your gut. But I’m guessing that’s not a risk most organizations would knowingly want to take.

If your organization is already following the recommendations made in this post, then you are off to a great start. Next month we will focus on “Problem Seven” which gets into a more advanced diagnosis allowing you to optimize a process that is already working well.

By |2018-03-07T16:37:21-05:00June 14th, 2016|Careers, Research, Updates|0 Comments

Is Experience Overrated?

Companies routinely use “prior work experience” as a significant factor when recruiting and hiring new employees. A look at any job posting board will no doubt show that most jobs require candidates to possess a minimum number of “years of experience” to even be considered for employment.

Recently, however, this practice has come under fire. Many are now suggesting that experience is overrated, and that companies place too much weight on experience when searching for job candidates.

These folks often point to the fact that many of today’s most successful companies were started by individuals who had little to no experience at the time of start-up (e.g. Bill Gates, Steve Jobs, Mark Zuckerberg, to name a few). They also suggest that advantages of not relying on “experience” when hiring includes: (a) the need for diversity on teams, (b) experienced hires often need to unlearn the bad habits that they might have acquired over the years, (c) it is not as expensive to hire people who have less experience, and (d) experience from one company doesn’t always generalize to other companies.

At some level, I agree with these points. I would argue, however, that for most jobs, experience plays a role in who is likely to succeed. The problem is that by only looking at someone’s basic job history you are not likely to gain much insight into how that experience translates to on-the-job success. You need to dig a lot deeper than that.

JOB TENURE DOES NOT EQUAL JOB SKILLS

Just because someone has spent many years working in a job, it does not necessarily mean that that individual has acquired the skills and work habits that you might expect. Consider two candidates applying for a maintenance mechanic job. Candidate A has seven years of experience working at a manufacturing plant. Candidate B has two years of experience working at a chemical processing plant. An initial review of these candidates’ qualifications might very well assume that Candidate A is more qualified for the job than Candidate B, based on the fact that Candidate A has more years of experience in the maintenance mechanic field.

However, “years of experience” is often an inaccurate measure of relevant experience. Instead, you need to consider multiple factors when evaluating these candidates. Some of these factors might include:

Breadth of Experience

Number of years in a job does not always equate to breadth of experience. For example, let’s assume that even though Candidate A has been a maintenance mechanic for seven years, he has spent nearly all of his time rebuilding valves that are specific to his manufacturing industry. Candidate B, on the other hand, has worked in all areas of his facility exposing himself to hundreds of pieces of equipment. Clearly, Candidate B has a wider breadth of experience than Candidate A.

Complexity Level of Experience

Certain tasks are much more difficult to learn than others. For example, as a maintenance mechanic, troubleshooting equipment is likely to be much more difficult than performing tasks that are the same or nearly the same every time (such as rebuilding a valve). A Candidate who has experience performing tasks that are complex or performed in a challenging situation (such as responding to an emergency) is likely to be more valuable to an organization.

Quality of Work

Finally, just because someone has substantial experience performing a job, does not mean that the person has performed that job at a level that your organization would expect. With our maintenance mechanic example, let’s assume that Candidate A, over the years, has learned “shortcuts” to doing his job that lead to poor quality of work. These shortcuts might allow him to get by on his current job, but would not be acceptable in a different job.

The bottom line is that in order truly evaluate someone’s experience–“years of experience”–isn’t what is important. What is important are the skills and work habits that a person has acquired throughout his or her career. The answer to that question requires a more thoughtful assessment than simply asking for “years of experience.”

By |2018-03-07T16:37:21-05:00April 11th, 2016|Careers, Research|0 Comments

Algorithms Beat Experts When It Comes to Hiring

The status quo is broken. Specifically, resumes, unstructured interviews, and combining predictive measures using professional judgment doesn’t work well. Over one hundred years of research in the area of industrial and organizational psychology supports this thesis, so I’m going to skip the basics and dive into the good stuff.

The good stuff, in my opinion, is more toward the cutting edge of decision science. We know that human beings, on their own, are imperfect decision makers. We know that well over 100 documented cognitive biases have an effect on our ability to make optimal decisions. We also know that many decision tools have demonstrated the ability to improve the accuracy of hiring decisions.

Job simulations, cognitive assessments, structured interviews, well mapped personality assessments, situational judgment tests, biodata, and physical ability assessments are all potentially valuable for predicting job performance. Moreover, when we combine the results from multiple job-related assessments in a statistically optimal, or even reasonable, fashion, the overall prediction is far more accurate, on average, than predictions made by experts who rely strictly on their “guts” to make these decisions.

To re-phrase the last sentence, algorithms beat experts at predicting future job performance. This general finding has been tested hundreds of ways with different types of predictions and different types of experts. The results are powerful, they are conclusive, and they are massively underutilized in the real world. We aim to help change that by including statistically optimal scoring generated by carefully derived algorithms with all of our assessment results.

So, for example, if you have a candidate who has taken three assessments, we will provide you with four scores:

Works Hard = 9.1
Works Smart = 6.4
Works Safe = 8.5
Baseline = 8.4

The first three scores indicate the individual assessment results and the final score indicates the overall score, appropriately weighted, across the assessments. We call this weighted composite score a Baseline score.

The Baseline score is the single best indication of a candidate’s probability of success on the job and Baseline scores are directly comparable across candidates. So, in a situation where different candidates have different strengths, the Baseline score can be used to quickly and accurately rank order the candidates in terms of their probability of success on the job.

Think about employee selection decisions that you have observed. Most hiring decisions come down to a person or group of people trying to compare candidates in an apples and oranges fashion. Candidate Joe has the most appropriate college degree for the position, candidate Sue has better job experience, candidate Pat had the best energy level in the interview, and candidate Kyle scored highest on the math test.

Who should you pick? How much weight do you put on each of these factors and how do you combine them to look at the “whole person” and compare that person to the needs of the job? Does a college degree really matter for this job? Is job experience at one organization easily transferable to another? Does “energy level” in a 30-minute interview suggest high energy on a day to day basis? Does “high energy” really matter if the person isn’t smart enough to be trainable?

People who have been involved in hiring decisions readily see that the complexity of most decisions quickly goes beyond our abilities. Fortunately, the human brain has wonderful mechanisms for dealing with complexity. One of these mechanisms is the use of simplifying strategies commonly referred by decision making researchers to as “heuristics.”

Our brain knows that, on average, a loud noise is more important than a soft noise. Things that smell good are more likely to be edible than things that smell nasty. A restaurant with many cars in the parking lot is more likely to be a good option versus a restaurant with few cars. In all of these cases the rule of thumb has some merit, but it is also likely to lead us astray at times.

Imagine that you are in charge of picking players for an NBA basketball team. There are lots of players from all over the world to choose from so you decide that you aren’t going to look at anybody who’s height is under 6’. You know that there are some great players who are less than 6’ tall, but you also know that you don’t have time to evaluate every player and by cutting out all people under 6’ your pool of candidates seems much more manageable.

Cutting part of the pool allows you to focus your time on players with the highest probability of success at the expense of a tiny proportion of great players who are under 6’ tall. This same scenario applies for any type of minimum requirement when hiring. Requiring 5 years of work experience, a 3.0 GPA, or a college diploma will simplify your hiring decision . . . at the expense of precision. What if we could simplify our options without losing precision . . . wouldn’t that be a better option?

 

Picture1

Source: http://www.nytimes.com/interactive/2013/11/03/sunday-review/so-you-want-to-play-pro-basketball.html?_r=0

 

As decision makers we face a dilemma. We need lots of data to make an accurate decision and we need to simplify that data to make sense of it. Simplifying is smart, which is why our brain does it automatically using heuristics. Over simplifying, however, is not smart and leads to many preventable mistakes, which is why we need a better way to make important decisions.

Algorithms provide the better way. An algorithm can efficiently combine data from a wide variety of measures in a fashion that minimizes data loss. An algorithm can be used to quickly and accurately rank order 7000 candidates in seconds and the results are far more accurate than if a team of people scour the same information for weeks. Algorithms can also be tracked over time and updated with improved algorithms.

Oddly enough we can model the decision making of experts using policy capturing studies, and the resulting algorithms are more accurate than the people they were modeled after (crazy, but true). In fact, we have modelled the decisions of NFL teams and created algorithms that rank order NFL draft prospects. On average, these algorithms are 36% more accurate than the selections made by actual NFL teams. We’ll get into the details of why this is true in future blog posts, but the simple explanation is that algorithms are extremely consistent whereas humans, even expert humans, are not.

So, when you think about humans being beat by algorithms, how do you react? Anger . . . “they” can’t be better than us! Fear . . . what are people going to do if the algorithms start making all the decisions? Disbelief . . . what does this guy know, he’s just a guy writing a blog. Or amazement . . . imagine the great things people could do if they improved their decision making when making life changing decisions! Many years ago I was a bit irritated that computers were beating Chess champions and, much later, Jeopardy champions. Today I have come full circle and I am simply amazed that people can build machines, computers, and algorithms that make people better at doing things they love to do.

Just as a bulldozer amplifies the power of a person and allows her to move more dirt faster than a hundred people digging with their hands, algorithms allow us to make faster, more accurate decisions in a less biased manner. The initial fear and pushback against using algorithms to improve our decision making is both understandable and irrational.

Let’s get over it and start making better decisions.

By |2018-03-07T16:37:21-05:00February 8th, 2016|Research, Updates|2 Comments

Science-Based Hiring is MORE Human Focused, Not Less

It’s about time we have new tools for recruiters and HR professionals

Wendy Beach, VP of Talent Science Solutions

How many of us get the opportunity to learn to use a whole new set of tools in our line of work?  When that happens, how many of us see it as an “opportunity”?  I’ve recently had the great fortune to adopt new tools which exceed my expectations in connecting companies with the best-matched talent.  Having been in the world of talent for over 20 years, I was a skeptical old dog, but if I can learn some great new tricks – tools – anyone can!

At SDS, we call our unique tools “predictive analytics,” or “talent science.”  They include customized job analyses, assessments which measure aptitude, safety, culture and behaviors specific to each job, and knowledge/skills testing supported with data, algorithms, and structured interviews.  (By the way, we throw out an old tool that doesn’t work anymore and slows the job down – the resume.  Ask us more about that if you’re curious!) 

Our proprietary tools require the ability to use them in the right combination for each situation–much like construction workers reach for the right tools at the right time. At SDS, we use our talent science tools in working directly for our clients, but we can also help HR professionals at companies learn how to use them.  Sounds great, right?  Well, not everyone I talk to is thrilled.  The biggest challenge I’m finding is fear of the unknown.  That’s not new when new tools come along in an industry.  Those who adapt and adopt thrive, and those who dig in their heels often end up left in the dust.

In my contributions to this blog series, I’ll respond to comments I’ve heard in my first year with SDS.  The comments I hear tend to be based on lack of knowledge and understandable skepticism.  We often say our biggest competition is the status quo.

COMMENT #1:  I pride myself on my “gut feel” about candidates.  That’s why my company needs me.  Why should I use a system that treats people like numbers? 

RESPONSE:  SDS is NOT about treating people like numbers. In fact, our tools “highlight the human” in human resources. We illuminate a 3D picture of someone who is otherwise flat on resume paper.  We actually create a wider and better matched talent pool of candidates instead of kicking applicants out if they don’t have certain keywords or GPA thresholds applied in most “status quo” systems.

WHAT YOU MISS WITH YOUR STATUS QUO

A campus recruiter from a global corporation told a story recently about how he met a student on campus who didn’t even apply for his jobs because he didn’t meet the GPA cutoff.

The student had a 2.9 instead of a 3.0 GPA.

However, when they talked further, the recruiter found out the student had been working two jobs, had developed a product which was being patented, and was helping his family after an unexpected tragedy.

The recruiter realized the qualities the student exhibited were exactly what he wanted to find – but wasn’t able to find – through their usual way of recruiting.

Wouldn’t you like to spend more time with candidates you know are a closer match because they’ve taken customized assessments–matched to the exact job before you even see them in a list ranked for probability of success?  Knowing that those not on the list receive a great candidate experience which doesn’t waste their time and is actually positive for them, so that they might come back for other opportunities?

SDS talent science tools are smart additions to your daily work, and make you look even better to your clients and your company.  You get to use new tools to make your outcomes soar.  If your company could increase productivity by even 10% with a piece of machinery or equipment, that would be a no-brainer decision.

Think of this as new HR tools and equipment.  Those who adopt earlier than others will be heroes to their leaders by making better decisions with their new tools.  They’ll also likely become more involved at the strategic tables of their operations.  Where HR is already part of the executive team, adopting these new tools will be an easy decision.

By |2018-03-07T16:37:21-05:00January 25th, 2016|Research, Updates|0 Comments
Go to Top