Personality Tests May Make People Look Better Than They Are, Says UA Psychologist

NASHVILLE, Tenn. - Two University of Arkansas researchers have identified a flaw in the way some popular personality tests question and assess people’s traits. This means many corporations that use such tests in the hiring process may be getting a skewed view of their new employees.

Eric Knowles, professor of psychology, and graduate student Chris Condon have spent several years examining biases that affect personality assessment. Their latest study focuses on a test called the NEO-PI-R, which consists of 240 questions designed to determine whether a person is mentally stable or neurotic, extraverted or introverted, open or closed to new experience, conscientious or careless and agreeable or disagreeable.

They chose to study the NEO-PI-R because it tests a broad range of traits and is widely used within industry and the field of psychology, Knowles said. The UA researchers present their results on Friday, Feb. 4, at the Society for Personality and Social Psychology meeting in Nashville, Tenn.

"Our study showed that the NEO-PI-R contains a double bias - first in the way the test was constructed and second in the way individuals endorse the questions," Knowles said. "Both of these biases can affect the accuracy of the test."

Condon and Knowles began their research by sorting the NEO-PI-R questions into distinct classifications. They confirmed that the test contained a balanced number of questions about

good (or socially desirable) qualities and about bad (socially undesirable) qualities. However, they found a distinct imbalance in the way the questions addressed good and bad qualities.

Each test question was phrased either as an assertion or a negation. Assertions made a direct inquiry about a personality trait - for instance, Are you an extravert? Negations made an indirect inquiry - Are you not an introvert?

Though these questions take a different form, both address the same personality trait - extraversion.

Ideally, said Knowles, personality tests should contain not only a balanced set of questions about good and bad traits but also an equal ratio of assertions and negations within that pool of questions. Knowles and Condon claim the NEO-PI-R fails to maintain a balanced ratio of assertions and negations.

According to their calculations, the test asks a total of 128 questions about good qualities. Of those questions, 105 were assertions and 23 were negations, resulting in a ratio of 4.56 assertions to one negation. But of the 112 questions addressing bad qualities, 64 represented assertions and 48, negations. Therefore socially undesirable traits received far more equal treatment, with a ratio of 1.33 assertions to every one negation.

"The NEO-PI-R is not the only personality test guilty of this imbalance," Knowles said. "We’ve seen a tendency on other scales to index socially desirable traits directly while treating socially undesirable traits with a more equal mix of direct and indirect questions."

This imbalance creates a bias in the way the NEO-PI-R was constructed, but since assertions and negations address the same personal qualities, one might wonder why an imbalance skews results. A rational, extraverted individual should read the questions Are you an extravert? and Are you not an introvert? and answer yes to both.

That’s the problem, said Knowles.

Through previous studies, Knowles and Condon have shown that only 15 percent of people answer questions about their personalities in a rational and accurate manner. The other 85 percent will give irrational or contradictory answers depending on how the questions are asked.

"People have a tendency to answer no to a negation and yes to an assertion even when they indicate the same personality trait," said Knowles, who calls this phenomenon an endorsement bias.

To prove that such a bias affects the results of the NEO-PI-R, Condon and Knowles examined the test scores of more than 1,050 people. Individuals taking the test had been instructed to answer each question by ranking its accuracy on a scale of 1 to 5. For example, an extremely sociable person would answer the question Are you an extravert? with a score of 5. Ideally, the same individual would answer Are you not an introvert? with the same ranking.

However, the researchers found that subjects consistently gave higher rankings to assertions and lower rankings to negations. Since the test used more assertions than negations to address good qualities, people scored high on desirable traits. And since more negations were used to address bad qualities, people scored lower on undesirable traits.

In other words, the NEO-PI-R made people look better than they actually are.

"Our research demonstrates how this test - and many like it - can misrepresent people," Knowles said. "The bias can be corrected either by addressing good and bad traits in a more balanced way or by finding out which subjects are endorsement biased and compensating statistically."

Though tests can be corrected, Knowles suspects the root of the problem may be more inscrutable. An ancient Greek proverb entreats each person to "know thyself." But that may not be as easy as it seems, said Knowles.

"People consider it the most natural thing in the world to describe themselves to another person. But actually, there’s a complicated psychological process that determines how we define ourselves, and that process may have as much to do with our experiences as our personality," Knowles explained.

"The simple answer is that there’s no simple answer. It’s not an easy process to come to know and recognize yourself - let alone trying to express it to somebody else."