One of my clients, an accomplished CFO involved with global M&A responsibilities, wanted to make sure that his new finance hires could really use all the brainpower they had been blessed with, so he incorporated a critical thinking test as part of the interview process. He was pleased with the results, seeing an increase in the problem solving abilities of his new team members. However, when he decided to take the instrument himself, there was a different reaction …
“I don’t understand why I only scored in the fiftieth percentile!” he thundered.
I pointed out that that is well within the norm for senior managers. “There must be something wrong with the test,” he continued.
“You and I examined the reliability and validity coefficients together,” I said.
“I wasn’t as focused as I should have been when I took it,” he implored.
“Would you like to take the instrument again?” I offered.
(after a long pause) he said, “Well, no, I guess not. It really is a bugger, isn’t it?”
“A lot of folks say that,” I said. “And, remember, you now have people working for you whose processors can scream at warp speed.”
The #1 Sought After Skill
Critical thinking has been rated the #1 desired skill in key contributors and senior level leaders, according to surveys conducted by organizations such as SHRM and AMA. And, as Socrates understood, although it can be learned, organizations today don’t have the luxury of teaching this skill. They need people already adept at:
- Accurately understanding problems,
- Analyzing evidence, and
- Making good decisions.
With 7-10,000 baby boomers retiring every day, the need for critical thinkers has never been greater.
Hiring High Potentials or High Performers?
During my years as a retained search consultant in the golden days of the late 1990s, companies had more room for choosing both types of candidates — those who could make a significant impact now and those who were solid, but were oozing with potential to be developed. Today, the market demands both.
So, how do we determine critical thinking ability in candidates?
To Test or Not to Test?
There are a few good instruments out there, including Pearson’s Watson-Glaser II. However, many search consultants I know choose to develop their own critical thinking interview questions. Both are valid methodologies.
And, what about combining the two strategies? After you have culled down the list of candidates to the top three to five, give each a high-quality, critical-thinking assessment. Then, incorporate the results into your last round of interviewing, developing questions specifically targeted to the possible weak links identified by the testing. I believe this to be the most powerful approach.
Drill Down from Targeted Questions
So, what types of questions really get at critical thinking ability? Behavioral questions can be used, but be careful — remember that candidates will tend to showcase war stories/accomplishments which only show their best. Motivation questions, while trendy and powerful, are as much about heart as head — while they are mandatory for any 2012 interview, they don’t really get at critical thinking directly. Situational questions can also provide us with a respectable platform from which to examine a candidate’s critical thinking ability.
And, once we have developed the right question for the right candidate, the next key is following up their answer with drill-down questions.
Here are a few sample questions and follow ups you might build out from:
Describe a complex situation in which you …
… had to think through information which conflicted with your own viewpoint or beliefs.
… had to make a critical choice based on incomplete data or inputs.
… convince others to examine different approaches surrounding a contentious topic.
Behavioral Drill-down/Follow-up Questions
List your basic assumptions when you first considered the situation.
What were your individual actions?
How did you determine which actions to take?
Please describe all the intended and unintended outcomes.
What could you have done differently to achieve similar outcomes?
What could you have done better?
A cross-functional (you pick the disciplines) team you are serving on is tasked with identifying, analyzing, and reporting on operational efficiencies. The efficiency data provided to the team by senior management is accepted by all team members as accurate, but you recognize it as faulty. Describe, in detail, how you would proceed.
A strong-willed and influential peer attempts to win you over to their position by using erroneous information as foundational to their argument. Give a detailed description of how you would respond.
The general manager emails you a consultant’s report which outlines findings about a recent decline in customer loyalty. In the email, the GM also makes a general statement that “the path forward seems obvious,” that she wants you to “take action now,” and that she sees “no need for any discussion.” After reading the report, you determine that there is more than one way to proceed. In detail, describe what you would do.
Situational Drill Down/Follow-up Questions
What assumptions are you making?
Why did you choose to proceed that way?
How did you determine that was the best course of action?
What other unknowns have you not accounted for?
Describe all the elements of your decision making process.
List three other pieces of information that it would helpful to know about in advance.
- Make sure you consider the potential liability of testing.
- Use only the highest quality tests and expert interpreters. Tests should have reliability and validity coefficients in the high seventies or eighties if possible.
- Remember — all tests and interview questions have built in error.
- Certainly, hiring decisions should never be based solely on test results.
- Optimally, integrate both testing and targeted, personalized questions into the process.
- Consider assigning a 1-10 rating for answers to follow-up questions if the same questions are asked of all candidates.
- Ask references to rate the candidate from 1-10 on: accurately understanding problems, analyzing evidence, and making good decisions. Then ask them to back up their ratings.
- Consider additional measurements of resiliency, impulse control, and other stress-related areas. High performers need to have a high tolerance for stress in order to problem solve when it counts.
- Remember that formal education and degrees do not necessarily translate into critical-thinking ability. Aristotle once said “It is the mark of an educated mind to be able to entertain a thought without accepting it.”
- Take the test yourself.
Photo from Bigstock
At Coburg Banks, we’re not a major fan of interview brainteasers.
In fact in one of our previous posts, 27 Behavioural Questions You Must Ask Your Interviewees we openly discredited their efficiency!
We just feel that they’re a little too off-putting for candidates, who are already feeling the pressure and unless utilised correctly, they really don’t reveal that much!
However, if you’re going to do something, you might as well do it right!
This week, we’re breaking down brainteasers, so you can make a more informed decision – are they right for your business?
Why use brainteasers?
Generally, the best hires are those who can identify problems quickly and solve them efficiently.
Critical thinking brainteasers have been created to assess candidates on the following key skills…
– Problem Solving.Can they at least attempt to solve problems as they arise? You don’t want an employee who keeps running to you, every time something goes even slightly wrong.
– Analysis.Can they look at the big picture and analyse all the available information to find a solution? You don’t want an employee who continuously overlooks important considerations.
– Creativity.Do they think outside the box? Sometimes, all it takes is a little bit of creative thinking so you don’t want an employee, constrained by the ‘rules’ (not all the time, anyway!)
– Performance under pressure.These questions will probably be completely out of the candidate’s comfort zone and it’s unlikely that they’ll have prepared for them. Can they keep it together?
Recruiter Pro Tip:
If performance under pressure is the most important factor to you, check out our recent blog post: 6 Left-Field Questions to Catch Someone Off Guard.
Asking questions that have no relevance…sounds like a great idea – right?
We’ll let you decide.
1. The pizza puzzle.
Q1. “If you were a pizza delivery man, how would you benefit from scissors?”
This little gem is attributed to Apple Inc and there clearly isn’t a ‘right answer.’
However, you can still use it to assess your interviewee’s ability to think critically and make well-formed arguments.
A great candidate will take time deliberating over exactly what it takes to be a pizza delivery man or woman and will consider exactly how they would use the scissors.
Clever so-and-so’s may even come up with reasons not to use scissors, which is still perfectly acceptable, as long as there is thought behind the decision!
These types of questions are likely to frustrate some interviewees so watch out for those who aren’t willing to play the game.
It’s an interview after all and you make the rules.
2. A calculated question.
Q2. “How would test a calculator?”
This is a fairly ambiguous question, attributed to IBMand you’d certainly assume that your average candidate won’t have dealt with the situation before!
You’re looking for a detailed and strategic method of testing the calculator.
First they’ll have to work out exactly what the question means:
Is it referring to the general functionality of the calculator (the buttons) or its actual mathematical ability?
Then they’ll have to come up with a systematic and efficient means of testing it.
They may require some paper to make notes; “visual” thinkers will need some way to visualise the process.
If a candidate appears to get confused by the question and the method they come up with doesn’t make any systematic or organisational sense, this could be a reflection on their general administrative skills.
However, there’s nothing wrong with a candidate who asks you to repeat the question or explain in more detail; sensible candidates will want to ensure the question is answered well, without rushing in without a clue.
3. Apples and pears.
Epic Systems are to blame for this epic interview question:
Q3. “An apple costs 40 cents, a banana costs 60 cents and a grapefruit costs 80 cents. How much does a pear cost?”
Hats off if you’ve managed to work out the answer to this one (without sneakily peaking below)!
It’s all about the vowels.
“If you charge 20 cents per vowel, the two-vowel word ‘apple’ would cost 40 cents, three-vowel ‘banana’ 60 cents, and four-vowel ‘grapefruit’ 80 cents.
Therefore, a pear would cost 40 cents.”
Clever – right?
If anyone manages to work this riddle out in a high-pressure interview situation, then as far as I’m concerned, you’re on to a winner.
(But do beware of anyone who’s too quick to answer – they may have heard it all before!)
However in general, if the candidate can come up with any sensible price and a good explanation of how they came to it, then they certainly deserve brownie points.
Use your initiative; is their answer sound, thought-out and practical?
If a candidate blurts out a number without being able to explain their reasoning, then they’ve clearly not bothered to consider the question properly.
4. What do a fox, a hen and a farmer have in common?
They’re all trying to get to the other side of the river…
Q4. “A farmer needs to cross the river with his chicken, a sack of corn and a fox.
His boat unfortunately only fits himself and one other thing.
The fox and chicken are hungry, so if he leaves the fox with the chicken, the chicken will get eaten, whilst if he leaves the chicken with the corn, the corn will get eaten.
How will the man get safely across with all 3?”
Do you recognise this question? Could you spontaneously answer it?
This is a really common brainteaser and is often utilised during group activities, to assess how well candidates can work together to solve a problem.
The answer is simple, when you know it:
The man takes the chicken across and then goes back for the fox.
He can’t leave the fox and chicken together so when he drops the fox off, he picks up the chicken. He then goes back for the corn.
He can’t leave the corn and the chicken together so when he picks up the corn, he drops off the chicken.
When the corn is safely with the fox on the right side of the river, he goes to collect the chicken.
Goodness knows why a farmer would be hanging around with a fox…
Candidates who take the time to understand each varying element and work their way through potential answers strategically, are most likely to be impressive problem-solvers.
In group interviews, look out for candidates who facilitate the discussion, taking into account everyone’s opinion and helping them reach a conclusion; they’re the leader.
If a candidate rattles off the answer immediately then chances are, they’ve faced this brainteaser before (a good, honest candidate will tell you if this is the case!)
Candidates who fail and don’t appear to ‘get’ the question or forget a major part of it (for example, they let the chicken and corn cross together) may have issues with listening, as well as critical thinking.
Q5. “How many potatoes (in kg) does McDonald’s sell in a year in the UK?”
There are hundreds of variations on this Oliver Wyman interview question, each assessing mental arithmetic and critical thinking.
– How many square feet of pizza are eaten in the UK every year?
– How many gas stations are there in the US?
– How many pennies, if placed on top of each other, would it take to reach the top of Big Ben?
The answer to the question is roughly 200 million kg of potatoes, however, like our other brainteasers, it’s not about finding the perfect answer; it’s about how they work it out!
Another toughie! If your candidate gets anywhere near, then that’s impressive enough.
If they don’t get anywhere near but explain calculations that are based on sensible presumptions, then it’s a great sign.
Ask your candidate to talk you through their sums as they go along. You’ll soon suss it out if they can’t add up properly!
Again, hasty, unthoughtful answers show a disregard for the interview and interviewer.
6. One to really catch them out!
Q6. “Tracy’s mother had 4 children. The first child was named April, the second was named May, the third June. What was the 4th child called?”
What would your answer be? July? Or did you work out the sneaky little plot twist within?
This cruel interview brainteaser depends on it’s capacity to draw the listener’s attention away from the answer, focusing on the unimportant information.
The question has already revealed the answer – ‘Tracy’s mother had 4 children’ – the fourth child must be called Tracy!
The names April, May and June meant nothing.
Of course, there is a right answer – Tracy – and anyone who gets it exhibits a critical thinking, common-sensical brain behind those eyes.
Candidates commonly answer with one of the two following things (unless they’ve completely missed the point)…
1) July. Don’t be too harsh; these poor souls have simply fallen into your well-laid trap!
2) We can’t know. Having missed the revealing reference to Tracy, some candidates may try to outsmart the question by concluding it’s impossible to know. (It is probable that the fourth child is called July, but not certain.)
Unfortunately, both answers are just wrong!
7. That age-old question.
Q7. “How do you know if the light inside the fridge is on or off?”
This, somewhat philosophical, interview question has been attributed to Schlumberger.
There are a variety of ways to answer this question, some great and some not so great, so you’ll have to be the judge…
- Use a camera.
- Drill a hole in the fridge door.
- Find the sensor and test it with your thumb.
- Touch the light bulb and see if it’s warm (be careful with this one, though!)
Although varying in practicality, all of the above do genuinely answer the question!
Unlike some of our brainteasers, this question is fairly easy to answer; you certainly don’t have to have any technical knowledge or special skills.
Candidates with no answer whatsoever (no matter how outrageous) are clearly lacking in creativity.
You can use brainteasers at any stage of the interviewing process, from telephone to face-to-face interview…
…however they’re most effective during group interviews, giving you a chance to assess a candidate’s teamwork, as well as, problem solving skills.
Recruiter Pro Tip:
Brainteasers could be about any topic you fancy, but it is critical to remember that they could be off-putting for certain candidates.
Never base your entire interview on brainteasers; you won’t learn anything about your candidates and you’ll come across unprofessional.
We recommend throwing (at most one or two) brainteasers into the interview, if you feel like it’s going very smoothly, your candidate is confident and everything seems a little too rehearsed.
Above all else, try not to be too hard on your candidates.
Remember, they’ll be judging you just as readily as you are judging them!
Good Luck!- Charles Trivett