PSYMETRICS FAQ'S

What Is A Test?

A test (for the purposes of our discussion) is a standardized device used to measure an individual’s skills, abilities, knowledge, competencies, intellectual capacity, personality characteristics and/or interests.

When compared to other selection methods such as the job interview, reference checks, experience and academic achievement, professionally developed and validated tests have been shown to be the best predictors of job performance. More specifically, ability tests have been found to be four times more effective than the typical job interview in predicting future job success.

Tests can serve many purposes within today’s highly competitive organizations. Those organizations that hire and retain the best individuals will have a marked advantage over the competition and that is why the use of pre-employment testing continues to increase. Talent Board's recent Candidate Experience Research reports that 82 percent of companies are using some form of pre-employment assessment. And this percentage continues to grow year to year.

Why Should Companies Use PsyMetrics Tests?

The question should be, “How can companies afford not to use tests?” The use of tests in business has a great deal of advantages over using the traditional job interview alone or other commonly used selection procedures. When you use tests to evaluate applicants, you are comparing “apples to apples.” Tests ask the same questions of everyone. So you can compare each applicant on exactly the same skill sets, dispositions and behavioral characteristics. The use of tests affords the human resource professional the opportunity to ask a great deal of job-related questions in a relatively short amount of time making tests much more efficient than any other hiring method. Skills tests allow you to test for skills that cannot be measured during the interview. Appropriately developed tests do not ask biased or illegal questions. Tests allow the applicant’s answers to be compared to the responses of hundreds or even thousands of other test takers that have taken the test under the same standardized conditions. And maybe most importantly, professionally developed tests, like the PsyMetrics EPS, have been developed based on scientific research that shows they are, in fact, predictive of future job performance. There are no other selection methods that can make all of these claims. When combined with the job interview, work history, reference checks and other screening methods, tests can significantly increase your ability to identify applicants who will succeed and be top performers within your organization.

Effectively screening potential employees is an organization’s ethical responsibility. An effective pre-employment testing program can go a long way in ensuring a safe, productive and satisfying working environment. Testing is a cost effective, efficient and effective means of identifying conscientious, top performing employees who will contribute to the safety and productivity of your organization.

How Are Organizations Today Utilizing the Benefits of PsyMetrics Tests?

Selection
During the selection process, tests can help identify job applicants who have the competencies and dispositions that are required of the position being filled.

Pre-employment tests generally focus on four areas: cognitive abilities (e.g., problem solving, spatial reasoning, mathematical ability), job-related personality characteristics (e.g., service orientation, self-confidence, conscientiousness), interests (e.g., investigative, artistic, social interests) and skills (computer literacy, software application skills, typing). Assessing applicants on only one or two of these areas can lead to an incomplete picture of the applicant. By using tests that measure job-related attributes, employers can assess the applicant’s suitability for the job and can then select those who are most likely to result in the best fit for the job, work group and organization.

Retention
Tests are an important component of achieving the proper fit between the job applicant and the job. Matching the right employee to the right job leads to increased employee satisfaction, increased productivity and reduced employee turnover.

Team Building
Tests can be used in creating effective work groups or teams. Understanding personality types and skill levels of team members is essential for helping the team or work group operate effectively and efficiently. Matching the right individual with the appropriate team function is critical to helping the team achieve its maximum potential.

Training
Tests can also be used to determine the training needs of individuals, work groups, departments or the organization as a whole. Test results can indicate employee strengths and weaknesses, therefore identifying areas where training may be beneficial. Identifying training needs and providing targeted training in these areas can lead to significant increases in productivity and job satisfaction.

Tests can also be used to measure training effectiveness. Testing trainees on relevant competencies or characteristics before training serves to establish a pre-training baseline of skills, knowledge and/or behavioral traits. Testing them again, post training, will let you determine the degree to which they grasped the training content…and will assist you in determining if they are ready to apply what they learned.

Career Path
Tests can be used to assess an employee’s interests, skills and personality characteristics. This information can then be matched to specific jobs or job groups when outlining the employee’s career path.

When properly used, organizations that utilize job-related and professionally developed testing instruments benefit from a more satisfied, motivated and productive workforce.

What Are Some of the Myths Surrounding Pre-Employment Testing?

Three of the most common myths surrounding pre-employment testing are: 1) “tests are illegal and those who use them will get sued,” 2) “pre-employment tests cost too much,” 3) “testing takes up too much time.” Each of these will be addressed below.

Myth #1 – Tests are illegal and those that use them will get sued.

The Equal Employment Opportunity Commission (EEOC) justifies the use of tests and any other selection procedure as long as they are related to successful job performance. Tests are just another method of gathering employee information in order to make the most educated hiring decision. Look at it as an interview on paper or an online interview (if the test is taken online). Obviously there are certain questions that should not be included in tests intended to be used in the business setting, for example, questions about age, religion, or other private and protected information. But these types of questions should not be used in any hiring method (e.g., interviews, job application). The same state and federal guidelines that apply to interviews, background checks, and so on also apply to tests.

If tests are developed and used properly, they can actually reduce the likelihood that you will get sued. Tests standardize the applicant data collection process. Every applicant is asked the same questions, in the same format, reducing bias and stereotypes. And given that professionally developed and validated tests increase validity over that of the interview, you are likely to hire fewer problematic employees. Hiring less non-compliant employees reduces problems in the workplace that can lead to liability issues such as negligent hiring lawsuits. “Companies that adopt pre-employment integrity tests to screen job applicants can reduce their exposure to negligent hiring claims.” Simply put, companies that hire the best employees are less likely to get sued over companies that hire problem employees. And professionally developed, job-related tests can significantly increase the quality of your hires.

Myth #2 – "Pre-employment testing costs too much."

The costs associated with a bad hire that leads to turnover are significant. When you factor in additional recruitment costs, training costs, management costs, low productivity and poor morale, most HR professionals would agree that these costs would run at least twice that person’s yearly salary.

According to statistics provided from the Saratoga Institute, Kepner Tregoe, Inc. and the Bureau of National Affairs, the average cost of turnover for a 2,000-employee company with an annual turnover rate of 12% (US yearly average) is four million dollars per year!

When you compare testing costs with the very significant costs associated with making a bad hiring decision, it is obvious that the investment made in testing is relatively insignificant.

The EPS offers a low cost solution for increasing the accuracy of your hiring decisions. Investing approximately the cost of lunch in evaluating an applicant today can literally save you thousands tomorrow.

Myth #3 – "Testing takes up too much time."

The EPS is extremely flexible and efficient. You decide how long the testing portion of your selection process will take. You can start with a couple of prescreen scales that can be administered in less than 6 minutes to eliminate those applicants who do not possess the most basic skills or characteristics required of the job. This step can then be followed up with a more comprehensive approach that tests for additional skills or behaviors. You simply choose from our extensive library of tests and custom build your test battery. You can add tests to your battery until you have satisfied the number of competencies you need to assess while staying within any time restraints you might have.

Also available are pre-assembled batteries whose administration times range from 20–45 minutes. These test batteries have been assembled by our staff of Industrial Psychologists based on the job requirements of typical organizational jobs or functions or industry types (e.g., Sales, Service, Retail, Banking, etc.).

What Are Some of Legal Issues that Pertain to Tests?

Title VII
There are federal laws and regulations that govern the use of employee selection procedures. The Equal Employment Opportunity Commission (EEOC) enforces many of the laws involving personnel actions including hiring. One of the most relevant laws focusing on employment decisions is Title VII of the Civil Rights Act of 1964 (Title VII), which prohibits discrimination based on race, color, religion, sex or national origin for companies with 15 or more employees. Moreover, Title VII also authorizes the use of “any professionally developed ability test provided that such test, its administration or action upon the results is not designed, intended or used to discriminate” on any unlawful basis.

In a 1971 landmark case (Griggs v. Duke Power Co.), The Supreme Court concluded that employment practices that had an adverse impact on minorities but were not proven to have a business necessity were in violation of Title VII. In 1972 Congress amended Title VII to include this legal standard.

As a result of this Supreme Court ruling, the Federal Government set out to unify and standardize the regulation of employee selection procedures. This effort resulted in the adoption of the Uniform Guidelines on Employee Selection Procedures (Guidelines). These Guidelines outlined the government’s position with respect to the prohibition of discrimination in employment practices based on race, color, religion, sex or national origin. These Guidelines apply to all private and public employers that are covered by Title VII.

Key Points of the Uniform Guidelines on Employee Selection Procedures Title VII and therefore the Guidelines apply to employers with 15 or more employees.

The Guidelines define a selection procedure as:

“Any measure, combination of measures, or procedures used as a basis for any employment decision. Selection procedures include the full range of assessment techniques from traditional paper and pencil tests, performance tests, training programs, or probationary periods and physical, educational, and work experience requirements through informal or casual interviews and unscored application forms.”

So as one can see, the Guidelines apply not just to tests, but to all selection tools and methods, including the interview, job application, reference checks and more.

One of the most misunderstood issues regarding the Guidelines is that of validation. When does a selection procedure need to be validated? The Guidelines state:

“These Guidelines do not require a user to conduct validity studies of selection procedures where no adverse impact results.”

Therefore, if a company were to periodically examine its hiring process to ensure that it continues to be free from bias and discrimination, chances are that company would never be challenged. It is those organizations that pay little attention to the content and methods used in hiring and the potential adverse effects these procedures could have, particularly on protected groups that can land them in hot water.

Tests and Adverse Impact The Guidelines define adverse impact as:

“A selection rate for any race, sex, or ethnic group which is less than four-fifths (4/5) (or eighty percent) of the rate for the group with the highest rate…”

This definition applies to all selection procedures (e.g., interviews, decisions made from job history, tests, reference checks, etc).

What is important to remember is that as long as the entire selection process does not result in adverse impact, companies are not likely to be challenged. It is when the whole selection process (e.g., interview, reference checks, testing, etc.) results in adverse impact that the individual components of that process are scrutinized. And for each individual method that is found to cause adverse impact, validity evidence is required.

When it comes to tests, some people have the misconception that they are all discriminatory, cause adverse impact and are therefore illegal. This is simply not the case. While some members of protected classes do tend to score lower on cognitive tests that measure verbal and mathematical constructs, the EEOC justifies the use of these tests so long as the constructs they measure are essential for successful job performance. For example, if you are trying to fill an accountant position, you are justified in testing applicants on mathematical concepts that will be required to perform the job. In addition, researchers who focus on personnel selection have consistently found that:

“There is no evidence that well-constructed personality inventories systematically discriminate against any ethnic or national group.”

All the empirical evidence including Federal legal guidelines and standards supports the use of testing in the employment setting as long as the tests have been professionally developed and the skills, competencies and/or behavioral dispositions they measure are essential for successful job performance. And, as always, it is the employer’s responsibility to periodically audit the entire hiring process (e.g., the interview, testing, the job application, and any other selection method used) to ensure that it continues to be fair and free from bias.

The Americans with Disabilities Act (ADA) The ADA went into effect in July 1992. With respect to employment, it covered employers with 25 or more employees. In July of 1994, it was expanded to include employers with 15 or more employees. The Act states that employers “shall not conduct a medical examination or make inquiries of a job applicant as to whether such applicant is an individual with a disability or as to the nature or severity of such disability.”

The ADA is intended to protect those employees with disabilities from discrimination. Professionally developed, work-related skills, behavioral and personality tests (such as the EPS) measure job-related attributes and dispositions. They do not measure, nor are they intended to measure any form of disability. The ADA does not prohibit employers from using tools that assess an applicant’s sales and service ability, their level of trustworthiness, management skills, job-related personality characteristics, for example. The EPS has been carefully reviewed to ensure all items comply with the ADA.

This summary was intended to be a brief overview of how “tests” are defined and used in the employment setting. Some of the most common misconceptions about tests and a couple of the major legal issues affecting not only tests, but the whole selection process in general were also presented. Each State may have its own set of requirements for selection tools and methods. If you are unfamiliar with such requirements, we would urge you to check with a qualified labor law attorney.

Do I Have to Use an Off-the-Shelf Test That May Not Fit My Exact Needs?

One of the toughest aspects of implementing a testing program is finding a test or series of tests that meet a company’s specific needs. Each organization has its own set of critical functions and the jobs within them require specific knowledge, skills and abilities. In many cases companies also have time constraints with respect to the amount of time an applicant will have to take the test(s). Many test publishers have pre-assembled test batteries that can address the majority of an organization’s assessment needs, however, there are also many instances where a perfect fit between job requirements, testing content and time limitations is not available. The alternative is an expensive proposal from the test publisher to develop a customized test from scratch.

With the PsyMetrics Elite Profiling System (EPS), organizations have the option of implementing any of the EPS “pre-built tests” that have been designed to assess the cognitive and personality characteristics associated with the most common jobs, or they can “mix and match” from a comprehensive library of short cognitive, personality, skills and/or interest scales to create a truly customized solution within their required testing time frame.

How Long do the PsyMetrics Tests Take to Administer?

EPS consists of cognitive, personality, interests and skills based tests. The personality and interests tests include both attitudinal and behavioral statements. Responses are made on a 5-point Likert scale ranging from Strongly Agree to Strongly Disagree. This item format is extremely efficient, allowing applicants to answer a great deal of questions in a very short amount of time.

The EPS cognitive and skills tests were also designed to be completed in a short amount of time. The cognitive and skills tests include brief questions covering various job-relevant cognitive abilities and skill sets. The answer format is multiple-choice. The tests are concise, focused and valid predictors of performance.

Most PsyMetrics tests are completed within 15-30 minutes.

All EPS tests are designed as stand alone products with the flexibility and available option of being combined into customized test batteries. Combining cognitive/skills tests and personality/interest assessments has been shown to increase validity above and beyond using one or the other.

With its customizability and efficient item format, EPS is ideal for seamless integration into any ATS test delivery platform. This enables companies to incorporate an “all-in-one” approach to their applicant recruiting and hiring process.

What Types of Questions Make Up the PsyMetrics Tests?

The EPS tests include both attitudinal and behavioral items with a work-related frame of reference included in the test instructions and inserted throughout the test items. Incorporating work-related (context-specific) instructions and behavioral items, and less invasive attitudinal test questions into the assessments are critical in reducing the likelihood your selection process will be challenged. Researchers have found that applicants will have a more negative perception of the testing experience if they do not see a clear relationship between the test and the job or if they find the questions to be excessively invasive. “If companies are interested in using employment tests that are perceived as being job-relevant, inoffensive, and non-invasive, then they should consider selecting tests that include job-relevant items as opposed to tests that are derivatives of clinical assessment instruments.”

Below are examples of a non-context specific test question and an EPS work-related (context-specific) question.

Non-context specific: “It is easy for me to manage pressure.”
(Here there is no reference to the job and can seem as somewhat invasive.)

Context-specific: “It is easy to manage work-related pressures.”
(Here there is a specific reference to the job and is less invasive.)

The EPS behavioral test instructions also include a work frame of reference. For example, the following statement appears as part of the test instructions:

“The following set of statements describes work-related behaviors and attitudes. As you read each one, think of how it relates to you during your day-to-day work situations. Each attitude or behavior is followed by a rating scale that defines the degree to which you agree or disagree with each…”

Incorporating work references throughout the assessment increases the perception that the test is job-relevant and therefore, in the eyes of the applicant, justified.

What Steps Does PsyMetrics Take in its Test Development to Maximize Test Validity?

In addition to creating a more positive testing experience, tests that incorporate a work-related context into the instructions and test items have been found to have greater validity than tests that do not include this work frame of reference. Moreover, these context-specific tests obtain incremental validity above and beyond both non-contextual items and cognitive ability.

As cited above, for behavioral tests, the work-related frame of reference built into the EPS scale instructions and questions leads to more valid responses than if one were to utilize more general (non-context specific) instructions.

How Does PsyMetrics Address the Issue of Potential Faking on Non-Cognitive Tests?

One of the biggest concerns employers have when using personality or behavioral tests is the ability of the applicant to fake the test and the degree to which the test’s validity will be compromised.

During the application process of any job, most applicants will do what they can to present themselves in the most favorable light possible. Some may exaggerate on the job application, others may “stretch the truth” during the interview, and still others will attempt to fake themselves through the testing process. While skills and cognitive tests are not conducive to faking, personality or behavioral tests are more susceptible to elevated scores.

Extensive research has been conducted to better understand base rates of faking and the effects on validity. Researchers seem to be split on the frequency to which faking takes place during the testing process. For example, Hough, et. al. (1985) found that during the actual screening process, the base rate for faking was rare. However, others suggest that as much as 30-50% of the population attempts to elevate their test scores.

Assuming there is some evidence for elevated personality test scores, what effects do these elevated scores have on the test’s ability to screen out poor performers? Some of the research aimed at answering this question concludes that there is little impact to a test’s validity based on the level of response distortion. According to these researcher’s these tests still predict job performance.
However, in our own research, we have found that “test faking” does have some impact on reducing test validity. For example, in recent studies we observed that when you exclude candidates who have scored high on social desirability/faking from the study sample, test validity tends to increase.

To combat the issue of faking during the testing process, the EPS includes a Candidness scale that identifies individuals who may be trying to “fake” the test or may be attempting to elevate their scores. The EPS score report highlights these candidates. For those candidates who score “Low” on Candidness, PsyMetrics recommends that more in-depth questions be asked during the interview process to obtain more insight into the applicant’s responses. A more thorough evaluation of the candidate’s work history, background and references should also be a priority.

What is the Reading Level of the PsyMetrics Tests?

When evaluating any testing instrument, particularly for entry-level or lower level positions, it is important to make sure the test questions are not written at a grade level that is too high for the position. For example, if a test is written using college level vocabulary and grammar, someone applying for a low level position that may not require such a high level of reading may not do well on the test simply because of how the questions are worded and not because the individual lacks the motivation or skill to do the job.

During the development of the EPS scales, every effort was made to keep the reading level of the questions as low as possible without taking away from the face validity or having to modify the job-related context of the questions. As a result of this effort, the reading level of the scales range from the 7th to the 9th grade. This reading level ensures that the EPS questions are applicable for entry level through managerial positions.

How Does PsyMetrics Demonstrate Test Validity?

The most important aspect of any assessment device is its ability to predict what it is intended to predict, i.e., its validity. As mentioned previously in this manual, the EPS scales were developed based on years of research using the latest techniques to increase efficiency, job-relatedness, applicant comfort level and validity. The test items were written based on extensive interviews with job incumbents, supervisors, managers, job observations, a review of training materials and a review of the psychological and skills testing literature. The original sets of test items were then validated using various validity methods (i.e., criterion-related, construct and self–report). Through extensive item analysis, the most valid and reliable test items were retained and used to create the EPS scales.

The following summarizes the concept of validity and the various validity methods utilized in validating the EPS scales.

Validity
A test’s level of effectiveness is directly related to its validity (the degree to which the test measures what it is supposed to measure) and its reliability (how consistent the test is at measuring what it is supposed to measure). The EPS scales have undergone significant research across various job categories utilizing several validation strategies. The results of all the research conclude that the tests within EPS are valid predictors of critical aspects of job performance. The three validation methods used to establish the validity of the EPS are summarized below.

Criterion-Related Validation
The concurrent, criterion-related validation method requires that the test be administered to current employees. Performance data is then gathered on those employees. If the test were a valid predictor of job performance, one would expect a statistically significant correlation between test scores and the performance data collected. In other words, those employees who score high on the test are the same employees who demonstrate high levels of performance. Those employees who do poorly on the test would likely be those who demonstrate poor performance.

The correlations obtained throughout all of the criterion-related validity studies conducted using the EPS tests indicate that the tests are valid predictors of job performance.

Construct Validation
In addition to the concurrent validation strategy described above, construct validation studies have been performed for many of the EPS scales. This validation strategy attempts to demonstrate the degree to which the instrument in question actually measures the psychological construct it is intended to measure. This approach generally involves administering the test in question along with another well-researched and established instrument that measures the same construct. If the two instruments measure the same construct, one would expect to find a significant correlation between the two. From the construct validation studies conducted using the EPS scales, we can conclude that the EPS scales measure the construct they were designed to measure and therefore are construct valid.

Self-Report
An additional strategy utilized to establish the validity of the EPS scales was to compare test scores to anonymous self-reports. Some tests measure behaviors that are not always observable yet could have a serious negative impact to your organization; for example stealing or illegal drug use. The anonymous self-report validation strategy makes it possible to collect past behavior information (e.g., stealing history, illegal drug use frequency) in a non-threatening manner. Test scores are then compared to the self-report ratings to determine the tests ability to identify those counterproductive behaviors. The results of the validation studies using the self-report data collection method offer strong support for the validity of the EPS scales intended to predict counterproductive workplace behaviors.

Are the PsyMetrics Tests Reliable?

In addition to the validity studies described above, reliability analyses have been performed for each EPS scale. Reliability refers to the degree to which the scale items are consistent in measuring the skill or construct the scale is intended to measure. The results of these analyses prove that the EPS scale items are consistent in measuring what they are intended to measure.

The EPS scales were specifically developed to help today’s organizations make the right hiring decisions. Extensive research utilizing various statistical methods all conclude that the EPS scales are valid and reliable tools for predicting a wide range job-related skills and behaviors.

How Should Test Scores be Interpreted?

1. Percentiles vs. Percentage

Test results are generally presented in terms of percentiles. A percentile is different than a percentage in that a percentile compares someone’s score to everyone else who has taken the test. For example, if someone scores in the 75th percentile, that means they scored better than 75% of the other applicants who have taken that same assessment.

A percentage on the other hand is simply the percentage of questions answered correctly. If someone were to answer 5 out of ten test questions correctly, their score would be 50%.

Most PsyMetrics candidate score reports use percentiles so one can see how the candidate scored relative to everyone else who has taken the assessment. This is important because when evaluating job candidates, one is comparing them to other applicants. So it is important to see how they all score relative to each other. The PsyMetrics assessments are designed so that the higher an individual scores, the higher the probability they will possess the skill or attribute they are being tested for. Therefore, the higher their percentile score, the greater the chances are that they will exhibit those skills or attributes at a higher level than candidates with a lower percentile score

2. Test Score Range Interpretations

Generally speaking, there are two percentile thresholds one should pay close attention to: The 30th and the 70th percentiles. These serve as a “rule of thumb” when interpreting the probability that a job candidate will succeed or fail on the job.

Job candidates who score above the 70th percentile generally have a higher probability of possessing a high level of the skill or behavioral characteristic being measured than those scoring lower. However, those scoring below the 30th percentile have a higher probability of possessing lower levels of those skills and behaviors.

Those scoring between the 31st and 69th percentiles can be considered as being “average” in the areas being assessed. While there are no truly absolute ranges and a top performer can definitely score below the 70th percentile, these are guidelines to use when comparing job candidates and making a hiring decision.

The test user should utilize various applicant evaluation tools when making a hiring decision (e.g., assessments, work history, the interview, reference checks, background screening, etc.). No one tool is perfect, but by combining all of the sources of applicant information available, the likelihood of selecting the right candidate increases significantly. And tests have been proven to be one of the best tools to aid in this process.

Should Companies Discuss Test Results with Candidates?

Each organization should be responsible for developing standard policies and procedures with respect to communicating job application results with job candidates. Candidates should be informed of what to expect next in the process. It should be emphasized that all phases of the pre-hire evaluation (i.e., job history, application information, interview, assessments, reference checks, etc.) are taken into consideration when making a hiring decision. Explain to the applicant that others are applying for the same position and that each applicant will be evaluated based on the same job-relevant criteria.

Organizations should not point out a specific selection criterion as the reason why an applicant did not get the job. Blaming a test result or a reference, for example, does not give the applicant much comfort and could produce resentment in the applicant. The fact is that the reason to hire or not hire a candidate should never be based solely on any one phase of the pre-employment process. The human resource administrator should carefully examine all of the information gathered about the candidate (e.g., work history, assessment results, reference checks, background checks, etc.) and use this information as a whole to determine the best candidate for the job.

The hiring decision should always be based on whether there is a match between a candidate’s job relevant skills, abilities and interests and the job itself. EPS assessments represent one source of critical information in helping to make that decision. Using all sources of candidate information available to the human resource professional will result in a more comprehensive view of the applicant and the best employee-job fit.