January 2017 Monthly Article - Choosing Effective Talent Assessments To Strengthen Your Organization

EPG
SHRM Foundations
Effective Practice Guidelines Series
Choosing Effective Talent

Assessments to Strengthen
Your Organization
Choosing Effective Talent Assessments to Strengthen Your Organization
This publication is designed to provide accurate and authoritative information regarding the subject matter covered. Neither
the publisher nor the authors are engaged in rendering legal or other professional service. If legal advice or other expert assis-
tance is required, the services of a competent, licensed professional should be sought. Any federal and state laws discussed
in this report are subject to frequent revision and interpretation by amendments or judicial revisions that may significantly affect
employer or employee rights and obligations. Readers are encouraged to seek legal counsel regarding specific policies and
practices in their organizations.
This book is published by the SHRM Foundation, an affiliate of the Society for Human Resource Management (SHRM). The
interpretations, conclusions and recommendations in this report are those of the authors and do not necessarily represent
those of the SHRM Foundation.
2016 SHRM Foundation. All rights reserved. Printed in the United States of America.
This publication may not be reproduced, stored in a retrieval system or transmitted in whole or in part, in any form or by
any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written permission of the SHRM
Foundation, 1800 Duke Street, Alexandria, VA 22314.
Selection of report topics, treatment of issues, interpretation and other editorial decisions for the Effective Practice Guidelines
series are handled by SHRM Foundation staff and the report authors. Report sponsors may review the content prior to
publication and provide input along with other reviewers; however, the SHRM Foundation retains final editorial control over the
reports. Editorial decisions are based solely on the defined scope of the report, the accuracy of the information and the value it
will provide to the readers.
The SHRM Foundation does not explicitly or by implication endorse or make any representations or warranties of any kind
regarding its sponsors or the products, services or claims made by its sponsors. The SHRM Foundation does not assume any
responsibility or liability for the acts, omissions, products or services offered by its sponsors.
For more information, contact the SHRM Foundation at 703.535.6020. Online at shrmfoundation.org
16-0720
Table of Contents
iii Foreword
v Acknowledgments
vii About the Authors
1 The Business Case for Effective Talent Assessment
2 The Assessment Landscape
2 The Evolution of Assessment Technology
3 Future Trends
4 Tools of the Assessment Trade
4 Job Analysis
5 Assessment Methods
8 How to Select the Right Assessment Method
10 Validity
11 Adverse Impact
11 Tradeoffs Between Validity and Adverse Impact
12 Assessment Utility: Business Benefits
12 Costs
13 Candidate Reactions
14 Legal Issues
17 Conclusion
19 Sources and Suggested Readings
33 Endnotes
i
FOREWORD
Organizational success is largely driven by the acquisition of the best talent

available. However, statistics reveal that 25 percent of mission-critical
positions often remain vacant after five months. As organizations compete
for the top candidates, they expend large sums of money, time and other
resources in advertising and recruitment. Research shows that the careful
use of talent assessments can result in increased productivity, cost savings
and better retention of top employees.
To help you sort through the many assessments available and find those
that will work best for you, the SHRM Foundation has created Choosing
Effective Talent Assessments to Strengthen Your Organization. This guide
focuses on the tools of the trade and how to select the right assessment
to achieve your business goals. It helps employers better understand the
value of assessment, as well as the legal and technical issues involved. In
addition, it offers a clear roadmap to selecting the effective assessment
methods to target top talent and boost productivity.
This report is part of the SHRM Foundations Effective Practice Guidelines

series, which now includes more than 20 titles. Created in 2004 for busy
HR professionals, the series integrates research findings with expert
opinion on how to conduct effective HR practice. It provides the tools to
successfully practice evidence-based management. Other recent reports
include Talent Acquisition, Creating a More Human Workplace and The
Aging Workforce. To ensure the material is both practical and research
based, the reports are written by subject-matter experts and are then peer-
reviewed by both academics and HR professionals.
The SHRM Foundation champions workforce and workplace transformation

and inspires HR professionals to make it happen. Our educational
resources are now used in hundreds of college classrooms worldwide.
The SHRM Foundation also awards hundreds of scholarships annually to
support education, conference attendance and professional certification
for HR professionals. All this good work is made possible by the generous
support of donors like you.
We encourage you to learn more. Please visit shrmfoundation.org to

download other complimentary resources and to find out how you can get
involved with the SHRM Foundation. Thank you!
Mark Schmit, Ph.D., SHRM-SCP

Executive Director, SHRM Foundation
iii
ACKNOWLEDGMENTS
The SHRM Foundation is grateful for the assistance of the following individuals in
producing this report:
REVIEWERS PROJECT MANAGER

Howard J. Klein, Ph.D. Dorothy Mebane, Ph.D.
Professor of Management and Human Manager, Foundation Programs
Resources SHRM Foundation
Fischer College of Business
The Ohio State University
Jim Link, SHRM-SCP

Chief Human Resources Officer
Randstad North America
Kameron M. Carter
Ph.D. Candidate
Department of Management and
Organizations
University of Iowa
The authors would like to thank Emmanuel Rilhac, head of EMEA & Global Functions,
CEB, Arlington, Va., and Donna L. Weiss, director, Product Management, CEB, Arlington,
Va., and two anonymous reviewers for their comments on and earlier draft of this report.
Their comments contributed significantly to shaping the final product.
v
ABOUT THE AUTHORS
Elaine Pulakos, Ph.D., has spent her career working with organizations to
design and implement talent management systems and processes in the
areas of staffing and assessment, performance management, leadership
development, and succession management. She is a recognized talent
management thought leader and frequently invited speaker to address
both practitioner audiences and the scientific community. She has authored
numerous articles, book chapters and books, as well as three best-practice
volumes for the Society for Human Resource Management (SHRM).
Dr. Pulakoss work has been recognized with several awards, including the
Society for Industrial and Organizational Psychologys (SIOPs) Distinguished
Professional Contributions Award, the M. Scott Myers Award for Applied
Research in the Workplace, and the William A. Owens Scholarly Achievement
Award. Elaine is a fellow of the American Psychological Association and
SIOP, for which she also served as president. Currently, she leads two
groups within CEB: global measurement product development and PDRI, a
CEB company.
Tracy Kantrowitz, Ph.D., is research director at CEB Talent Assessment. In

this role, she is responsible for the development of assessment content and
research related to the use of assessment tools for employee selection and
development. Dr. Kantrowitz has published in leading journals and presented
at national conferences on topics such as predictors of job performance,
computer adaptive testing, unproctored Internet testing and mobile
assessment. Tracys primary areas of expertise relate to test development,
validation, research methodology, statistics and psychometrics, job analysis,
and selection system design.
Tracy was previously chair of the Professional Practice Committee for

the Society for Industrial and Organizational Psychology (SIOP). She has
received awards from SIOP, where she was elected fellow, including
the M. Scott Myers Award for applied research in the workplace and the
Distinguished Early Career Contributions - Practice award.
Dr. Kantrowitz holds a Ph.D. in industrial/organizational psychology from the

Georgia Institute of Technology.
vii
SHRM Foundations Effective Practice Guidelines Series
Research reveals that careful talent assessment results in significant

productivity increases, cost savings and decreased attrition.
CHOOSING EFFECTIVE TALENT

ASSESSMENTS TO STRENGTHEN
YOUR ORGANIZATION
THE BUSINESS CASE FOR EFFECTIVE TALENT ASSESSMENT

Organizations today compete fiercely for talent. Recent
statistics indicate that 25 percent of critical positions remain
unfilled after five months.1 Firms invest an enormous amount of
money, time and other resources in advertising and recruiting
strategies to attract the best candidates because todays
executives understand that one of the most important drivers
of organizational success is outstanding talent. Research also
reveals that careful talent assessment results in significant
productivity increases, cost savings and decreased attrition.
Questions Leaders Are Asking Organization Challenges

On average, one in four new hires
How can we reduce turnover?
leaves within a year.2
Each position has many applicants,

How can I find people who will
but 65 percent of them do not meet
succeed in specific positions?
basic requirements.3
How can we identify effective Only 12 percent of todays

leaders? employees are enterprise leaders.4
Given the proven bottom-line value of attracting and retaining

talent, why do so many organizations and HR professionals
employ ineffective, haphazard approaches to selecting
employees? This report addresses how and when to use
assessments to improve hiring and retain the most valuable
employees in a competitive market.
Although the term assessment can include a wide variety

of tools, including resume reviews, applications, reference or
background checks, and interviews, the focus here is on more
formal scientific assessments. After a brief survey of the value
of different types of assessments, we will focus on practical
explanations of technical and legal issues and then offer a
roadmap to choosing the methods that offer the best return on
investment for your organization.
1
THE ASSESSMENT LANDSCAPE services.5 Approximately two- Assessment Trends

thirds of HR professionals report Machine learning and
The nature and variety of
that their organizations are analytics are introducing

assessment tools have evolved
putting a premium on positive new assessment
considerably in recent years as
applicant experiences, although techniques.
analytics and data have become
relatively few collect data to
more important in the business Globalization requires
monitor reactions to assessment
environment. Formal assessments localized assessments
procedures.6
are used to hire new employees, that can easily reach
evaluate current employees One of the most dramatic overseas.
for different roles, identify shifts involves the evolution
employees with high leadership
Technology is enabling
of the candidate-driven job
potential, compare talent within on-demand, online
market,7 which has changed assessments with
an organization against industry how organizations compete for
or geographical benchmarks, immediate results and
talent and the nature of their feedback.
understand talent strengths and recruitment and assessment
gaps, and develop employees processes. Candidates want brief, More firms are using
long-term value. Advances in informative, and engaging hiring immersive, compelling
technology mean that more tools experiences and transparency assessment experiences.
are accessible online, on-demand, so they know what to expect, Organizations want
and with immediate reporting and where they stand and how much personalized and
feedback. effort to put into a hiring process. contextualized
Technology has also given rise to All these new expectations are assessment.
more available data on individuals transforming assessment methods
and practices.
Candidate-centric hiring
and new tools that enable requires shorter, more
organizations to mine these data for transparent and engaging
competitive advantage. For example, The Evolution of
candidate experiences.
social media and other data Assessment Technology
collected via technology are used Unsupervised online testing
for predictive analytics, modeling has become the standard for questions from a large set of items
trends, passive assessment of assessment administration based on how each candidate is
candidates and uncovering insights because it is so efficient, responding: If a candidate responds
to enhance business performance. decreasing the time needed to fill correctly, the next item will be more
The use of immersive, personalized vacancies and reducing costs. 8 difficult; if a candidate responds
assessment experiences is on the Initially, online testing prompted incorrectly, the next item will be
rise. two major concerns: test security easier. A customized assessment
and candidate integrity. with a unique combination of items
In the hiring context, organizations
are using these new tools and Concerns about security arose focuses on each candidates level of
experiences to provide job because early versions of online ability and knowledge. CAT creates
previews and sell the organization assessments were static, meaning more secure and more efficient
and its brand. Some firms want that all candidates received assessments, with fewer items in
to provide brand-enhancing the same items, which could less time, to meet the demands of
hiring experiences that will attract be copied and shared easily.9 candidate-centric hiring.
candidates and retain them If tests are copied and shared,
as customers, even if they are CAT represents a critical
they become invalid and must be
not offered jobs. This trend is innovation in online assessments10
replaced, creating unsustainable
important because poor candidate but does not resolve the problem
costs for companies.
experiences are broadly shared of candidate integrity or making
throughout social networks and Computer adaptive testing (CAT) sure that the person taking
can yield decreased use of the methods are now the norm because an online assessment is the
organizations products and CAT allows for dynamic selection of same person who is applying
2
Test Equivalence Is a Key Issue Technological Advances

As assessment devices evolve from booklets to PCs to mobile phones in Assessment
and tablets, we need to consider whether the mode of administration, Unsupervised online
rather than the assessment itself, is leading to lower or higher scores. testing.
The ideal situation is that no matter what device is used, the candidate
earns the same score.
Computer adaptive
testing.
for a job. To ensure candidate as much as by their measurement

Mobile assessment.
integrity, some organizations and predictive qualities. Low effort assessment.
ask candidates who have
progressed through the hiring Some assessments, including
personality and past-experience important in predicting future
process to complete a brief onsite
assessments, show little performance.
assessment at a convenient point
to verify the candidates online difference in scores when
score. completed on mobile devices Future Trends
or PCs.12 But research on Technology will continue to
Mobile assessment now provides cognitive ability assessments is spark revolutionary changes
even more convenience than inconsistent, with some studies in assessment methods and
assessments via desktop finding equivalence across practices. For example, the ability
computers. Consumer data device types,13 and others finding to passively target candidates
indicate that mobile devices differenceswith lower scores from available information sources
may be more readily available on mobile phones.14 Overall, could lead to transformational
to underrepresented groups, however, initial research shows changes in hiring processes.
including African Americans, relatively small differences Today, applicants apply for
Hispanic Americans and women.11 restricted to certain assessment positions based on job postings,
Assessment administration on items. but tomorrow organizations may
mobile devices may therefore
Low-effort assessment uses new search data for key talent in
result in larger and more diverse
technology and machine learning ways that bypass postings. This
candidate pools.
to sift through large amounts of is uncharted territory and will
Differences in mobile devices unstructured information, including require new policies on privacy,
mean that mobile assessment resumes and applications, to legal and ethical considerations.
can be complex, requiring collect predictive information As hiring practices span global
more nuanced evaluation. For about candidates. The information markets, human resource
example, larger screens on is coded and validated against professionals will also have to
tablets allow some forms of work outcomes, such as job become familiar with additional
assessment that will not work performance and retention, to laws and regulations.
well on phones. How information produce algorithms that managers
is input, whether via touch Perhaps most important, early
can use to inform their hiring
screen or keyboard and mouse, adopters of new technology
decisions. These techniques
presents unique challenges. should be aware that without
automate identification and
A new class of assessments sufficient research behind it,
scoring of predictive information
that use mobile-centric design an innovation might undermine
so that large numbers of
principles has arisen to optimize rather than help an organization
candidates can be screened in
the user experience. We are to make effective, defensible
minimal time at low cost.
seeing a demand for innovative hiring decisions. In some
assessments that go beyond Low-effort assessment is still countries, litigation and audits of
multiple choice and look and in its early stages, so all the hiring practices will lead to new
feel more like games. The result advantages and disadvantages standards and precedents related
of these innovations is that are not yet clear, but it is probably to new assessment technology,
assessments are being judged best suited for situations in which and organizations must be alert
by their design and accessibility past experience is especially for these changes. Later in this
3
report, we will look at the science Investigator Job: Sample Competencies

behind assessments, legal Communication: Speak and write clearly and concisely, with self-
requirements and workers rights confidence and using effective inflection and gestures.
to help organizations navigate
these choppy new waters.
Critical thinking: Question assumptions and identify merits in logic.
Collaboration: Elicit cooperation from other individuals and
organizations.
TOOLS OF THE
ASSESSMENT TRADE Professionalism: Maintain a professional demeanor and appearance at
all times.
Organizations can use a broad
array of techniques to make Handling stress: Remain calm and levelheaded.
selecting the right assessment
tool easier. The first order of
business is to do a thorough job
Job Analysis Steps
analysis, followed by a review of
the various types of assessment
STEP 1. Observe or interview job experts to develop a
methods available.
list of work activities and competencies workers need.
Job Analysis
STEP 2. Ask job experts to rate which work activities and
The first step in selecting an competencies are most critical for effective job performance.
assessment tool is to understand
what the target job or role requires
STEP 3. Analyze the data to prioritize the most critical work
the employee to do by conducting
activities and competencies for your organization.
a job analysis15 that focuses on
identifying critical work activities
and/or competencies. STEP 4. Select assessments that measure the performance of the
most critical work activities or competencies required on the job.
An in-depth discussion of job
analyses is beyond the scope
of this report, but they typically
include four major steps, as to broader performance
Investigator Job: Sample outlined above. requirements.
Work Activities
Every organizations unique One useful source of job analysis
Provide testimony by character is reflected in how it information is O*NET, a database
stating facts and answering performs a job analysis and then that is continually updated by
questions. chooses assessments. Business surveying a broad range of workers
Gather and review information strategy, organizational culture from hundreds of occupations.
to obtain evidence, or and contextual factors always play Employers and job seekers can
develop background major roles. Contextual factors may leverage the O*NET database
information on subjects. include high-versus-low growth, to obtain information for virtually
high-versus-low change and high- any job in the global economy.
Integrate information to
versus-low team interdependency. The database forms the heart
uncover relationships
A recent study of thousands of of O*NET OnLine, an interactive
between individuals, events
organizational leaders across 85 application for exploring and
or evidence.
companies showed the importance searching occupational information
Work in a team environment of organizational context factors (http://onetonline.org). Of course,
as a team member or leader. in successful identification of it is always best to verify that job
Calm and reassure victims leaders at all levels.16 These results information from O*NET fits the
and others in tense situations. reinforce the value of thinking specific job and situation in your
beyond job-specific competencies organization.
4
Work behavior is the tip of

the icebergthe observable Work Behavior Assessments
performance on the job. Work Behavioral interviews.
behavior assessments require
Work simulations.
Work candidates to perform simulated
Behavior
aspects of the job and provide Physical ability tests.
an observable measure of
Acquired capabilities knowledge, performance that should forecast
skills, judgment, experience
real on-the-job performance.
Innate mental and physical
These assessments should Acquired Knowledge,
abilities, personality, interests
correspond closely to the specific Skills and Experience
work that candidates will perform Assessments
and tend to be expensive, time- Bio data/experience.
consuming and somewhat limited Job knowledge.
Assessment Methods in their use.
After you understand job
Writing skills.
Acquired knowledge, skills and
requirements, you can select Situational judgment.
experience sit just below the
assessments that measure the
surface. Although less directly
most critical work activities or
observable than work behaviors,
competencies. For example, is the
assessments of job-relevant Innate Ability and
assessment goal to select someone
knowledge, skills and experience Attribute Assessments
to perform a specific job in the are extremely effective predictors
near term or to predict employee Cognitive ability.
of job performance. Because these
potential for future leadership roles? assessments require measuring Personality.
In the former case, the assessment unique capabilities relevant to Integrity.
can be targeted to the most critical specific jobs, they are not generally
competencies for the job. In the applicable across jobs.
Physical fitness.
latter case, a broader and more
comprehensive whole-person Innate ability and attributes,
including personality and Below we discuss specific
assessment will probably be more
interests, lie deep below assessment methods, all of which
useful.
the surface and also have are backed by strong research
Assessment types run along a been proven to be effective over decades. They are all readily
continuum from more observable, predictors of job performance. available and effective for small
direct measures of performance Because assessments of innate and large companies. We do not
to less observable, indirect characteristics do not focus on include some newer assessment
measures of performance. An job-specific traits, they are the methods with less research
iceberg analogy helps illustrate most flexible and applicable for available to back them up.
the different assessment types. use across different jobs. Cognitive ability assessments
typically consist of multiple-choice
Example of Cognitive Ability Test Items items that measure various mental
abilities, such as verbal and
Verbal ability Numerical ability Reasoning ability
numerical ability, reasoning, and
Innocuous means the 16% of 62.5 is: 132435465
reading comprehension. These
same as: a. 844 a. 4 assessments have proven to be
a. Harmless b. 8.44 b. 5 extremely useful predictors of
b. Preventative c. 084 c. 6 job performance and are used
c. Distasteful d. 8.4 d. 7 frequently in hiring for many types
d. Futile of jobs.17 Some measure distinct
abilities, whereas others measure a
5
combination to provide a measure of job performance,18 particularly Example of an Integrity

of general mental ability or g.18 in combination with measures of Assessment Item
cognitive ability. In work situations, It is okay to misrepresent
Cognitive ability tests conscientiousness, extraversion, the truth if being completely
administered online in agreeableness, openness to honest will create problems
unsupervised testing situations experience and emotional that small white lies can
often employ adaptive methods. stability are measured most solve.
The mobile environment poses frequently.20 Research suggests a. Strongly disagree
some unique challenges that conscientiousness is the
because the typical questions b. Disagree
most robust and useful predictor
are too complex and lengthy c. Neither agree nor
of performance across jobs.
for mobile phones, although disagree
Achievement orientation is also a
they may work well on tablets. strong predictor in many jobs.21 d. Agree
Newer test designs will probably e. Strongly agree
be optimized for mobile Personality assessments typically
administration. consist of several multiple-choice
or true/false items that measure
Personality assessments a variety of personality factors. Examples of Physical
measure stable, job-relevant Desirable answers are often quite Fitness Tests
characteristics that have been obvious in this format, so there
shown to be effective predictors Run a mile.
is considerable concern about
how easy it is to fake a positive Lift two 20-lb. weights for
impression. On the other hand, 50 repetitions.
Examples of Personality even when candidates do respond
Test Items with what they assume are the
Standard format: desired answers, personality perform a job. By nature, they dont
assessments are still useful for work in an online environment.
I have remained calm in
predicting job performance.22 Some
situations in which others Biographical data (bio data)
assessments avoid the problem
have become upset. assessments ask candidates
by using a format that presents
a. Definitely true questions about their past
equally desirable response options
b. Somewhat true experience in areas that have
representing different personality
been shown to predict future
c. Neither true nor false traits. This item formatasking
performance.25 For example, a
d. Somewhat false candidates to choose which
question about the number of
answer is most like themis less
e. Definitely false volunteer organizations to which
transparent than a simpler scale.
someone has belonged could
These assessments work well
Format that Mitigates indicate the extent to which he or
across different devices.23
Intentional Desirable she is willing to pitch in and help
Responses Integrity assessments measure others. A question about where
Choose which item is most attitudes and experiences related ones spending money came
like you and which is least to honesty, trustworthiness from in high school could indicate
like you. and dependability.24 They a history of responsibility and
are typically presented in a independence.
a. I persist with tasks until
multiple-choice format. Similar to
completed. Some bio data assessments
personality assessments, integrity
b. I treat customer are scored using data-driven
assessments can be easily
complaints with highest techniques, meaning the points
administered across different
priority. associated with each response
devices without problems.
c. I deal well with extremely option are determined according
stressful situations. Physical fitness tests measure to the relationship between
overall fitness, strength, endurance that response and an important
or other capabilities necessary to job outcome. Other bio data
6
automated scoring algorithms are Example of a Job

assessments are rationally scored,
used. Accomplishment records Knowledge Item
meaning the test items are scored
are effective predictors of job The principle of the lever
according to the degree to which
performance.28 is essential to the use of a:
each response option measures
an important competency for a. Hydraulic jack
Job knowledge assessments
a particular job. Biographical measure critical knowledge areas b. Plow
inventories consist of multiple- usually technical knowledge c. Car steering wheel
choice items, and like personality needed to perform a job d. Forklift
assessments, they tend to be effectively.29 These assessments are
equivalent across different modes of appropriate to use when candidates
administration.26 must possess a specific body of
Another form of bio data is knowledge prior to job entry, not How Are Writing
an assessment called an when candidates will be trained Assessments Created?
accomplishment record. In this case, on the job. Job knowledge tests
First, administrators prepare
candidates prepare a written account typically consist of multiple-choice
a written prompt and give it
of their best accomplishments in items, but they sometimes include
to about 200 candidates. At
key competency areas, such as essay items. In addition to hiring
least two human evaluators
planning and organizing, customer assessments, job knowledge tests
independently score
service, and conflict resolution.27 are used often for certification and each writing sample using
Candidates also provide the name licensing assessment. defined ratings scales and
of an individual, such as a past Knowledge tests can be come to an agreement.
supervisor, who can verify each administered well online using These scores and scales
accomplishment. Evaluators are large banks of items and adaptive are used to train automated
trained to score the accomplishments technology, but there is no definitive scoring algorithms, which
in a consistent manner using research about whether they can are then used to score later
standard rating criteria. Sometimes be administered effectively on writing samples.
mobile devices. Certification and
licensing assessments, especially
Examples of Bio Data
those that are high stakes, are still after the initial investment can be
Assessment Items
administered in a supervised testing significant. This type of assessment
To how many volunteer environment for the most part. is suitable for online environments
organizations do you
Automated writing assessments as long as it is possible to confirm
belong?
can be developed to measure that the candidate, and no one
a. 0
various competencies beyond the else, is the one providing the
b. 1 writing sample.
writing itself, including a candidates
c. 2 to 4 persuasiveness, critical thinking, Situational judgment tests present
d. 5 or more attention to detail, commercial focus candidates with situations just
and political savvy. Technology is like those they would encounter
Where did you get your
used in lieu of human scoring with
spending money in high on the job, with viable options
a technique called latent semantic
school? for handling each situation.30
analysis. Writing assessments are
Allowance from family Candidates might be asked to

especially useful in selecting people
select the most effective, or the
Own earnings to fill positions with significant
most and least effective, ways of

writing demands, such as attorney,

Partly allowance, partly handling a situation from a few
analyst or investigator.
earnings options. These tests are more
Development of the writing prompts expensive and complicated to
Other sources
and scoring algorithms requires develop than many other types
Had no spending money quite a bit of time and resources, because developing scenarios
but the efficiency and cost savings with several equally attractive,
7
Example Situational viable responses is difficult. They often use them to assess qualities
Judgment Item are typically administered in written like teamwork, communication,
You have just prepared or videotaped form, and both leadership, planning and
a report and checked it formats can be administered online, adaptability.
for accuracy. Before you but administering them on mobile
attend a meeting to submit devices with smaller screens may be Work simulations measure how
your report, you review challenging. a candidate performs activities
the typed version and note that the job requires, and of
Structured behavioral interviews
many serious errors. In this course, employers should use
unlike more traditional, unstructured
case, you would: them only when knowing how to
interviewsconsist of specific
a. Show the original and questions that assess critical perform those activities is truly a
the typed version to competencies required for specific prerequisite. 32 These simulations
the person in charge of jobs. can be time-consuming and
typing and demand that expensive to develop. Sometimes,
Interviews are among the most
the errors be changed trained evaluators do the scoring,
common hiring methods used
before the meeting.
in all sorts of organizations.31 but today assessment and
b. Present the report at the Unfortunately, there are rarely any feedback are often automated.
meeting, point out the agreed-on standards for evaluating
errors and explain the responses, and unstructured Physical ability tests are used
typist made them. interviews are not particularly useful to select people for physically
for predicting job performance. demanding jobs, such as police
c. P
resent the errors to the
officer, firefighter or warehouse
typist, ask him or her to An important characteristic of any
make the corrections and worker.33 These assessments require
effective interview is that it provides
explain to those at the standardized rating criteria to help candidates to perform actual work
meeting that the report is interviewers judge the quality tasks to determine whether they can
still being typed. and effectiveness of candidate meet the physical requirements of a
responses. Interviewers must learn job. The tests are often scored on a
d. Present your report at
to conduct an interview properly, pass/fail basis and are often timed.
the meeting without
probe for additional information and Physical ability tests are specific to
mentioning the errors.
apply the rating criteria accurately. work activities, whereas physical
Notify everyone of the
corrections after the Structured behavioral interviews fitness tests measure more general
meeting. can assess almost any aspect of overall physical health, strength and
competency, but employers most endurance.
Structured Behavioral Interview Question and Sample Rating Criteria
Tell me about a time when you were able to establish rapport with someone in a situation that made it difficult to do so.
What were the circumstances? What did you do? What were the results?
Low Moderate High

1 3 5
Made minimal attempts to Attempted to understand the other Effectively reached out and sought
understand the persons persons perspective. Developed a to understand the persons views.
perspective. Developed only a positive working relationship with Developed an extremely positive
surface-level relationship in a the person in a moderately difficult relationship with the person in an
simple situation. situation. extremely difficult situation.
8
Sample Physical Ability Tests HOW TO SELECT THE RIGHT Candidate reactions: Do
ASSESSMENT METHOD candidates react positively or
Firefighter negatively to the assessment?
In this section, we provide
Climb a ladder while
guidelines to help human resource The results, explained in the
carrying equipment.
professionals make informed table below, demonstrate that
Drag a hose. decisions about assessments. We no simple formula for choosing
will look at four important criteria, the right assessment exists
Carry a person down from a
and the answers will be based on because each type measures
building.
solid research over time:34 specific aspects, and each has
Warehouse Worker advantages and disadvantages.
Validity: Is the assessment
Often, a combination of
Lift materials of weight that method useful for predicting job assessments works well to
must be lifted on the job. performance? optimize results based on your
Drag materials for distances Adverse impact: Do members organizations specific goals
required on the job. of protected groups, including and needs. Assessments can
minorities, women and employees be phased inusing a shorter
Roll barrels.
over age 40, systematically score one first, to screen out the most
Carry materials the distance lower than others? unqualified candidates, and then
they must be carried on the using more complex methods for
job.
Cost: What is the expense of
higher-potential candidates.
developing, administering and
maintaining the assessment?
Evaluation of Assessment Methods on Four Key Criteria

Costs to Develop/
Candidate
Assessment Method Validity Adverse Impact Administer/
Reactions
Maintain
Cognitive ability High High (minorities) Low-high/low/high Somewhat favorable
Personality Low-moderate Low Low-high/low/low More favorable if
job-relevant
Integrity Moderate-high Low Low/low/low Less favorable
Physical fitness Moderate to high High (females and High/high/low More favorable
over age 40)
Biographical data Moderate Low-high for High/low/moderate More favorable if
different types job-relevant
Job knowledge High High (minorities) Low-high/low/high More favorable
Writing High High (minorities) High/low Somewhat favorable
Situational judgment Moderate Moderate High/low/moderate More favorable
(minorities)
Behavioral interviews High Low High/high/low More favorable
Work simulations High Low-high, depends High/high/high More favorable
on what is measured
Assessment centers Moderate to high Low-high, depends High/high/high More favorable
on what is measured
Physical ability Moderate to high High (females and High/high/low More favorable
those over age 40)
9
What is an assessment Validity: The Most Important Consideration

center, and when do Does the assessment predict how individuals will perform on the job?
you need one?
An assessment center Criterion Validity
offers a specific type of Determines the strength of the relationship between scores on an
work simulation, usually assessment and job performance effectiveness, tenure or other
related to executive, important outcomes.
managerial, or supervisory
jobs. Candidates respond
Validity is measured using a correlation coefficient, which ranges
to situations, problems, and from 0.0 to 1.00.
tasks in role-play exercises, Assessment validities typically range from .10 to .40.
in-box exercises that
A minimum validity to target is .25 or above.
ask them to prioritize and
make decisions based
Content Validity
on incoming information,
case exercises, and group Involves providing evidence that an assessment is a direct measure
discussion. Video and voice of work behaviors.
recordings mirror what Based on expert judgment.
candidates see and hear on
the job. More sophisticated
options, such as branching You may find the array of between scores on an assessment
role-plays, can automate possibilities requires you to and important job outcomes. For
personal communication seek advice from an assessment example, assume that a sales job
by providing adaptive expert. Below we explain some requires exceptional customer
responses based on what of the critical information to keep service. If the assessment is a
the candidate does or says. in mind. valid predictor of job performance,
In some cases, assessment candidates who score higher on
Validity customer service will perform
centers can also be used
for non-managerial job Validity means that an better, with higher sales and higher
candidates. Day in the assessment is job-relevant and customer loyalty ratings, and those
life simulations ask job useful for predicting performance. who score lower will not perform as
seekers to handle scenarios Clearly, this is the most important well. To establish criterion validity,
in workflows similar to consideration in selecting an a firm would evaluate the strength
what happens on the job. assessment. The two most of the relationship between
These simulations provide common types of validity human performance on the assessment
robust information for those resource professionals encounter
and job performance on a scale of
making hiring decisions are criterion and content validity. 35
0 to 1.00although scores would
for the candidates. As they Criterion validity involves never reach a perfect 1.00 because
become easier, cheaper, demonstrating a useful relationship so many other factors are at play.
and more flexible, use of
simulations may increase
over time, especially in Tip: Look at Validation Studies
latter stages of a hiring In considering the use of commercially available assessments, HR
process when people professionals should look for how many criterion validity studies
tend to be more willing to have been conducted on the assessments and meta-analysis results,
invest in a longer, in-depth if available. The more successful validation studies a vendor can
commitment. produce from other organizations and the larger the number of
individuals included in validity studies, the more confidence you can
have in the quality of the assessment.
10
Example of Criterion Comparisons of criterion and Looking at the

Validity Studies content validity reveal that Numbers: Effect Size
Administer the assessment criterion validity is stronger and and Adverse Impact
to be validated to a large tends to be viewed as more An effect size can be calculated
group of employees (at least credible. However, content to compare the assessment
300). validation is perfectly acceptable scores between two groups.
for demonstrating the job Effect sizes range from 0,
Collect measures of
relevance of work samples, indicating no difference between
these same employees
simulations and other direct groups, to 1.00 or more,
performance on the job.
measures of work behavior. indicating a large difference.
Correlate the assessment Criterion validity can be used to Effect sizes less than .20 are
scores with the job validate any type of assessment considered small, and those
performance scores, using a and is the only type of validity that near .80 or more are considered
correlation coefficient. can be used when an inference large, signaling a likely adverse
The correlation is the needs to be made about the effect. Understanding effect
strength of the relationship relevance of an assessment sizes is useful because they are
between the assessment for predicting job performance often reported in test manuals
and job performance. instead of using a direct and research reports evaluating
measure of performance. Most assessments.
assessments, including cognitive
After multiple validation studies ability, personality, bio data and
have been conducted for a given quality that they do not enable
integrity, require inferences about
assessment type, summarize the successful validation.
their relevance for predicting
results across studies, and make performance, so criterion The bottom line is that criterion
a stronger statement about the validation is the most common validity studies are so complex
true validity of the assessment. form of validity. that conferring with an assessment
In the table above, we analyzed expert is best if you need to do
validation studies and used them Successful criterion validation
criterion validation work.
to identify low (.20 or less), relies on having performance or
moderate (.20 to .40) and high other such outcome measures
that accurately distinguish Adverse Impact
(.40 to .50) validity.
employees effectiveness on the Adverse impact is a concept
Assessments with low-to-moderate job. Performance ratings that defined by the U.S. government
validities can add value, especially are available in human resource to establish evidence of
when they are used in combination information systems (HRIS) discrimination. It occurs when
with other assessments. For or performance management the results of an assessment or
example, cognitive ability and systems are often of such poor hiring lead to a disproportionately
personality are largely unrelated,
but, cumulatively, they combine to
produce higher validity than when Adverse Impact
used alone. An assessment has adverse impact if it results in hiring
proportionately fewer protected group members than majority group
Content validity involves
members.
demonstrating that an
assessment provides a direct
Calculating Adverse Impact
measure of critical work activities
or behavior. Comprehensive job The 80 Percent Rule:
50 females are assessed and 30 females are hired, or 60 percent.
analysis information and expert
judgment are used to develop 50 males are assessed and 45 males are hired, or 90 percent.
work simulations, ensuring that The ratio of females to males hired is 67 percent.
the assessment closely mirrors A ratio of < 80 percent is evidence of adverse impact.
job requirements.
11
small number of protected group evaluated. The U.S. Office of Federal

Contract Compliance Programs
Assessment Utility
members (e.g., racial minorities,
females, individuals over age 40) uses an alternative approach, one What are the business
being selected. If the proportion to which commercial businesses of outcome benefits of
of protected group members a certain size, with federal contracts, using assessments?
hired is less than 80 percent of must comply. With the exception
Selection Success Gains
the proportion of majority group of South Africa, the concept of
Number or proportion of
members hiredreferred to as the adverse impact is not widely

successful candidates
80 percent ruleadverse impact used outside the United States,
before and after
is suspected. If an assessment but other countries have specific
implementing a valid
laws, regulations and compliance
produces an adverse impact, the assessment
requirements that govern hiring
only way to defend its use is by
practices and employee rights. Business Outcome Gains
showing that it is a valid predictor
of performance and that alternative Adverse impact does not only The financial
measures are not feasible. happen in the assessment process. consequences of using a
Resume screening, reference valid assessment
The 80 percent rule is not the only
checks, interviews and recruitment Revenue and profitability
way adverse impact is defined or
practices can also result in an increases compared to
adverse impact against protected assessment costs
groups. HR teams should not shy
Strategies to Reduce
away from using assessments just
Adverse Impact
to avoid adverse impact, especially strategies include providing test
What To Do given the improved performance preparation sessions for candidates38
and business outcomes that proven and removing test items that have
Recruit more qualified
assessments provide. larger group differences.39 While
candidates from protected
some strategies have shown
groups.
Tradeoffs Between Validity promise, elimination of adverse
Assess a wider set of and Adverse Impact impact entirely is still a challenge
competencies, such as and is something that organizations
All organizations want the best-
contextual performance (e.g., should monitor and manage across
performing workforce, and most also
citizenship, helping others, all talent management processes,
want a diverse workforce that does
expending extra effort), which systems and decisions.
not exclude protected groups.
is predicted by measures that
produce minimal adverse While evidence of validity can Assessment Utility:
impact (e.g., personality and be used to justify and defend Business Benefits
bio data). assessments that produce an Assessment utility addresses
Use assessments with more adverse impact, most organizations the business benefits of using
adverse impact later in want to mitigate adverse impact assessments and is helpful in
the process, although this as much as possible. Some explaining the value of assessment
approach is likely to help assessments that produce high levels to decision-makers who are outside
only when there are many of validity also produce high adverse the human resource team. Two
applicants per position. impact. A number of strategies types of utility are apparent in
may reduce adverse impact assessments: selection success
What Not To Do without reducing validity, including gains and business outcome gains.40
enhanced recruiting of protected
Administer test preparation
groups, assessing a broader array Selection success gains compare
programs. They may increase
of competencies beyond the successful candidates before and
candidate comfort but do not
technical to include motivational and after assessment implementation.
reduce adverse impact.
behavioral factors,36 and adjusting The greater the selection success
Remove test items with large the placement of assessments within gain, the higher the assessment
group differences. the hiring process.37 Less effective utility.
12
Business outcome gains are

more interesting, particularly the
Buy Versus Build?
revenue and margin gains that result Buying Commercial Assessments
from selecting higher-performing
Pros:
employees compared with the Implementation is fast.
assessment costs. The formulas used
Assessment quality has been demonstrated.
to evaluate business outcome gains
are more complicated than simply
Vendors provide hosting, maintenance and updating.
comparing revenue/profits versus Benchmarks provide talent insights.
costs. They consider additional Localized assessments and norms may be available that can be used
factors, including how effectively the globally.
assessment predicts performance, Cons:
the number of applicants and tenure Licensing agreements yield ongoing expense that may be prohibitive.
of the selected group.41 Use across situations may raise security or competitive advantage
When decision-makers see the concerns.
positive impacts of assessment
in terms of key business outcome Building Custom Assessments
metrics, the value of assessment Pros:
Full customization to the organizations jobs and strategy is possible.
usually becomes more apparent and
compelling. They present maximum relevance and potential competitive
advantage.
Costs They have no ongoing licensing fees.
Some assessment methods involve Cons:
higher costs to develop, maintain Ongoing costs exist to host, upgrade and maintain.
and administer than others, although Significant effort and data are needed to localize assessments and
technology has significantly reduced develop appropriate norms.
the large variability in costs. Mobile Organizations need specialized expertise to build, host and maintain
administration should further reduce assessments.
and equalize administration costs. In They are time-consuming.
terms of development, automated
work simulations tend to be the most
costly to develop because they assessments in-house? There who are best matched to the
typically require test experts to work are issues of time, cost and organization and job.
with job experts and IT developers on convenience to consider in each
case. After development, custom
design and automation. Work samples
assessments need to be
that are not automated can be even
Some organizations prefer custom administered and maintained, which
more expensive than automated
assessment content that is uniquely requires ongoing investments
assessments because they involve
designed and validated for their in online hosting, maintaining
equipment, facilities and training
specific situation and embeds databases of test scores, analyzing
evaluators to observe and score
the organizations values, culture and reporting data for decision-
candidates. At the other extreme,
and context. Using a custom making, revising items, and
online multiple-choice assessments of
assessment is most important potentially revalidating in the
innate characteristics can usually be
when the assessment is intended future. If an organizations global
developed independently by testing
to serve as part of a larger, highly assessments have to be adapted
experts and are highly efficient to
branded hiring experience or for use in different countries, the
administer and score.
as both a job preview and an firm should ensure that the content
The buy-versus-build decision is assessment. Some organizations is adjusted and nuanced so
key for any organization. Should believe custom assessment candidates in different cultures can
you license standard assessment provides a competitive advantage understand it. Effectively managing
content or develop and host custom and better identifies candidates a custom assessment program
13
requires specialized expertise An overall great candidate

Candidates React
among in-house staff or external experience.
Positively When:
expert consultants. Helpful information and They are able to
Organizations may choose to feedback. demonstrate job-relevant
use commercially available Assessments relevant to the job. capabilities.
assessments, which assessment
In general, candidates favor Assessment procedures
providers can implement quickly
assessments that measure are administered
and host, update and maintain.
work behavior directly through consistently and fairly.
Commercial assessments should
always come with evidence methods like work samples and Feedback is provided so
of assessment quality across simulations rather than multiple- candidates can determine
organizations. Some commercial choice tests of abstract concepts. their standing and decide
assessment providers offer Cognitive assessments tend whether to proceed.
benchmarking data against which to be viewed more favorably
than personality, integrity and
They are given insight into
organizations can evaluate how their
the nature of the job and
talent stacks up in their industry or bio data assessments.43 Job
organization.
geographical area. Some providers candidates view assessment
have also amassed large volumes of items aimed at diagnosing
data globally, which allows them to personality or clinical disorders
provide localized assessments with much less favorably than items types with excessive words
appropriate norms. Some providers that measure job-relevant were viewed less favorably on
can also customize products to meet personality characteristics mobile devices compared with
a specific organizations needs. like conscientiousness and graphic and image-based items.
agreeableness. As mobile technology expands
In setting up assessment programs, and assessments are designed to
employers should seek the Candidates want information take advantage of more mobile
guidance of assessment experts. about the job, the organization features, such as swiping instead
In many organizations, assessment and how they are doing in the
of clicking, additional research
programs are executed through hiring process, including how
will be needed to fully understand
a combination of in-house staff they could have performed
candidate reactions.
and external experts or vendors. betterbut this is a tricky
SHRM provides comprehensive matter. Employers should
and valuable information about provide as much information as LEGAL ISSUES
available assessments and related is reasonable, but discussing Human resource professionals
topics through the SHRM Testing assessment performance in detail who administer assessments
Center (shrm.org/testing/). can divulge information that could should have a basic
lead to security breaches. And understanding of the legal
Candidate Reactions providing in-depth feedback for issues and requirements that
all candidates would be extremely govern their use. In the United
As many organizations move
toward candidate-centric hiring, expensive. States, two pieces of legislation
candidates reactions have become are especially helpful: Uniform
A final topic relevant to candidate Guidelines on Employee
increasingly important in judging reactions concerns views of
assessments. Candidates who have Selection Procedures and the
mobile assessment. Some studies
a positive hiring experience report Americans with Disabilities Act of
have found no differences in
higher levels of satisfaction and 1990 (ADA).
perceived fairness moving from
are more likely to recommend a PCs to mobile devices,44 but The Uniform Guidelines specify
company to others.42 others have come to the opposite requirements for selection
The most critical factors conclusion.45 Research has systems covered under the Civil
influencing applicants reactions shown that satisfaction increases Rights Acts and under Executive
boil down to whether they believe as screen size increases, Order 11246 (see https://www.
the organization provided: irrespective of the content. Item eeoc.gov/laws/regulations/
14
The Importance of Compliance Federal Contractors

Some managers and others within your organization may believe and Subcontractors:
that using informal selection methods instead of formal assessments Pay Attention to Recent
will avoid legal issues and eliminate the need to comply with federal Regulatory Guidelines for
regulations. This is not the case. If any aspect of the hiring process Hires with Disabilities
clearly produces an adverse impact, an organization will be required Regulatory guidelines
to demonstrate validity if challenged or audited. In practice, well- updated in 2013 reflect
developed and validated formal assessments actually help mitigate increased emphasis on hiring
legal risk and exposure. individuals with disabilities,
If members of your firm view legal requirements as burdensome, let and hiring goals are now in
them know that the procedures in the Uniform Guidelines are, in fact, place for federal contractors
the same procedures an assessment expert would employ if the only and subcontractors, with
goal were to identify the best-qualified candidates for a job. Thus, stipulations for actions
compliance with legal requirements in this case is synonymous with to be taken in the areas
best practices and will yield the best outcomes for any organization. of recruitment, training,
record-keeping and policy
dissemination. The guidelines
for the full text of the Uniform accommodation in assessing that call for voluntary disclosure
Guidelines). These requirements skill. However, if there are ways from job candidates to self-
address the question of whether a to accommodate disabilities both identify as individuals with
procedure is causing an adverse on the job and in the assessment disabilities or protected
impact, and if so, which validation process, an organization must veterans. The guidelines also
requirements should be met do so. Examples of reasonable require workforce surveys
before using the procedure. Two accommodation include: to gather this self-reported
documents can assist in applying information. The data are
Substituting an oral assessment used to create action-
the Uniform Guidelines:
for a written one. oriented programs.
The Society for Industrial and Providing extra time to
Organizational Psychologys
complete an assessment.
Principles for the Validation understand and meet their
and Use of Personnel Selection Offering an assessment in responsibilities under the ADA.
Procedures (see http://siop.org). Braille. One of the most comprehensive
Standards for Educational Providing items in larger print. sources of information is the Job
and Psychological Testing, Accommodation Network (see
Providing an interpreter who
developed jointly by the http://askjan.org), maintained by
can communicate items in sign
American Educational Research the U.S. Department of Labor,
language. which provides free, expert
Association, the American
Organizations increasingly are and confidential guidance on
Psychological Association
held accountable for ensuring workplace accommodations and
and the National Council on
that recruitment and selection disability employment issues.
Measurement in Education (see
http://apa.org/science/programs/ processes are fair and accessible,
testing/standards.aspx). and research is slowly accumulating
on the fairness of selection tools
The Americans with Disabilities and accommodations for people
Act created substantial with disabilities.46 For example,
requirements for providing research on longer exam times
reasonable accommodations to found that they generally improved
applicants with disabilities. If a the performance of all students, not
job requires a skill that a person just those with disabilities.
with a disability is unable to
perform, an organization is not Several sources offer guidance
required to provide reasonable to help HR professionals
15
Validity means that an assessment is job-relevant and useful for predicting performance.
Clearly, this is the most important consideration in selecting an assessment.
CONCLUSION
In recent years, technological advances, competitive demands

for talent and candidate-centric hiring have created powerful
commercial forces that are driving changes in assessment
methods. Many organizations are demanding shorter,
more engaging candidate experiences and less obtrusive
assessment methods to help them target talent. Innovative
assessment techniques are also being developed. These
factors, coupled with continued technological advancement,
will almost certainly lead to revolutionary and rapid changes in
the assessment field.
Always remember, however, that the primary criterion for

evaluating assessments does not change. Managers and
human resource professionals must continue to seek effective
assessments that will result in talented candidates with strong
performance on the jobemployees who bring long-term
value to the organization.
17
The buy-versus-build decision is key for any organization. There are issues of
time, cost and convenience to consider in each case.
Quiderum num faccuptus es ratem res simaion necersperum la simet aut

endanihicium fuga. Ut omniam aut omnimil ius doleste nderio ellabo. Ut oditatus
et molorio tenis molorepel imusam.
- Quotee to come
18
SOURCES AND SUGGESTED

READINGS
JOB ANALYSIS
Brannick, M. T., & Levine, E. L. (2002). Job analysis. Thousand

Oaks, CA: Sage.
This is a user-friendly book about how to conduct a job analysis
and use the results. Topics covered include the building blocks
of job analysis methods, work (task)-oriented methods; worker
knowledge, skills and abilities (KSA) methods; hybrid methods
that combine both task and KSA job analysis; job analysis for
management jobs and teams; job analysis and the law; doing a
job analysis study; and uses of job analysis in job descriptions,
performance appraisals, job evaluation, job design, staffing and
training.
Gael, S. (Ed.). (1988). The job analysis handbook for business,

industry, and government. New York, NY: Wiley.
The author argues that job analysis is the hub of virtually all
human resource administration and management activity and
necessary for the successful functioning of organizations.
The handbook addresses job analysis topics and issues
comprehensively, with material relevant to a wide audience
of practitioners working in business, industry, labor unions,
universities, the military, and federal, state and local
governments.
UNPROCTORED AND MOBILE ASSESSMENT
Beaty, J. C., Dawson, C. R., Fallaw, S. S., & Kantrowitz, T. M.

(2009). Recovering the scientist-practitioner model: How
IOs should respond to UIT. Industrial and Organizational
Psychology: Perspectives on Science and Practice, 2, 58-63.
This commentary raises organizational examples and mitigating
factors associated with unproctored Internet testing (UIT) in
three areas: cheating, ethics and legal concerns. The authors
encourage I/O practitioners to execute a research agenda
associated with UIT.
19
Beaty, J. C., Nye, C., Borneman, devices or applicant demographic three industriesmanufacturing,
M., Kantrowitz. T. M., Drasgow, subgroups. customer service/retail and health
F., & Grauer, E. (2011). Proctored care.
versus unproctored Internet King, D., Ryan, A. M., Kantrowitz,
tests: Are unproctored tests as T. M., Grelle, G., & Dainis, A. (in Landers, R. N., Reddock, C. M.,
predictive of job performance? press). Mobile Internet testing: Cavanaugh, K. J., & Proaps, A. J.
International Journal of Selection An analysis of equivalence, (2014). Talent assessment using
and Assessment, 19, 1-10. individual differences, and mobile devices. In T. Kantrowitz
This paper examines the criterion- reactions. International Journal & C. M. Reddock, Shaping the
related validity of unproctored of Selection and Assessment, 4, future of mobile assessment:
assessments. Results generally 382-394. Research and practice update.
indicate that assessments This paper examines the Symposium presented at the
administered in proctored and equivalence of mobile Internet 29th Annual Conference of
unproctored settings have similar testing (MIT) compared with the Society for Industrial and
validities. testing on personal computers Organizational Psychology,
(PCs) and whether attitudes Honolulu, HI.
Gutierrez, S. L., & Meyer, J. M. and other individual differences The study consisted of 219
(2014). The mobile revolution: influence responses and individuals who completed five
Measurement equivalence and reactions to MIT. Results commercially available and
mobile device administration. In demonstrated equivalence for a previously validated assessments
T. Kantrowitz & C. M. Reddock, supervisory situational judgment from a large human resource
Shaping the future of mobile test across testing modes, consulting firm across two time
assessment: Research and but not for a cognitive ability points three weeks apart. The
practice update. Symposium test. Respondents reported authors found relatively small
presented at the 29th Annual significantly more positive differences in magnitude that
Conference of the Society for reactions when tested on a PC were limited to cognitive-type
Industrial and Organizational versus a mobile device. measures.
Psychology, Honolulu, HI.
This study was conducted in Kinney, T. B., Lawrence, Morelli, N. A., Mahan, R. P.,
partnership with a client organization A., & Change, L. (2014). & Illingworth A. J. (2014).
in the customer service field, with Understanding the mobile Establishing the measurement
the objective of obtaining empirical experience. Data across device equivalence of online selection
support for the organization to and industry. In T. Kantrowitz assessments delivered on
deliver its pre-employment tests via & C. M. Reddock, Shaping the mobile versus non-mobile
mobile devices. future of mobile assessment: devices. International Journal of
Research and practice update. Selection and Assessment, 22,
Illingworth, J., Morelli, N., Scott, Symposium presented at the 124-138.
S., & Boyd, S. (2014). Internet- 29th Annual Conference of This study examines the
based, unproctored assessments the Society for Industrial and measurement equivalence of
on mobile and non-mobile Organizational Psychology, selection assessments delivered
devices: Usage, measurement Honolulu, HI. on mobile and non-mobile
equivalence, and outcomes. This study examined measurement devices. Measurement invariance
Journal of Business and equivalence, pass rate differences tests conducted with multigroup
Psychology, 30, 25-34. and applicant reactions by device confirmatory factor analysis
This study demonstrates that type (smartphone, tablet, PC) suggest that mobile versions of a
the use of mobile and non- for several mobile assessments; cognitive ability-type assessment,
mobile devices did not produce investigated the factors in mobile two bio data assessments, a
any practically significant score users test environment; and multimedia work simulation and a
differences on assessment across conducted analyses across text-based situational judgment
20
test appear to be equivalent to A review of the research literature PERSONALITY ASSESSMENTS

non-mobile versions. However, indicates that general cognitive
mobile device user latent means ability (GCA) predicts supervisor Barrick, M. R., & Mount,
were half a standard deviation ratings and training success M. K. (1991). The big five
lower than their non-mobile as well as objective, rigorously personality dimensions and job
counterparts for the situational content-valid work sample performance: A meta-analysis.
judgment test. performance. The author argues Personnel Psychology, 91, 1-26.
that the findings from this study This study examines the validity
Tippins, N. T. (2009). Internet support classic learning theory of five personality measures
alternatives to traditional over behaviorist theories of for predicting performance
proctored testing: Where learning and performance. in five occupational groups
are we now? Industrial and (professionals, police, managers,
Organizational Psychology: Hunter, J., & Hunter, R. F. (1984). sales, and skilled or semi-
Perspectives on Science and Validity and utility of alternative skilled). Three different kinds
Practice, 2, 2-10. predictors of job performance. of performance measures were
This focal article explores Psychological Bulletin, 96, 72-98. used in this study: job proficiency,
Internet alternatives to traditional, An examination of research on training proficiency and personnel
proctored testing. It provides a predictors of job performance data. One of the personality
comprehensive review of the showed that for entry-level jobs measuresconscientiousness
definition and history of UIT, the cognitive ability was the most was consistently related to all
factors that contribute to the rise valid predictor. For selection three performance measures for
of UIT and the mitigating factors on the basis of current job all of the occupational groups.
that pose risks to organizations performance, the work sample Extraversion was a valid predictor
that use UIT. test was slightly better. The for two occupations involving
authors suggested that using social interactionmanagers and
other predictors in conjunction sales. Openness to experience
COGNITIVE ABILITY TESTS with ability tests might improve and extraversion were valid
validity and reduce adverse predictors of the training
Gottredson, L. S. (1986). Societal impact. proficiency criterion. Overall,
consequences of the g factor the results illustrate the benefits
in employment. Journal of of using the five-factor model
Vocational Behavior, 29, 379- Ree, M. J., Earles, J. A., of personality in assessment
410. & Teachout, M. S. (1994). research.
This article discusses and rebuts Predicting job performance: Not
seven common arguments that much more than g. Journal of
Applied Psychology, 79, 518-524. Costa, P. T., Jr., & McCrae, R. R.
general intelligence (g) is of
The authors investigated the roles (1992). Four ways five factors
little or no practical importance
of general cognitive ability (g) and are basic. Personality and
in employment and illustrates
specific abilities or knowledge Individual Differences, 13, 653-
the effect that differences in
(s) as predictors of work sample 665.
intelligence in a workforce
job performance criteria in seven A considerable number of
may have on the structure and
jobs for U.S. Air Force enlistees. research studies have shown
functioning of whole societies.
Analyses reveal that g was the support for a five-factor model
best predictor and that s added of personality traits. These
Hunter, J. (1986). Cognitive a statistically significant but include conscientiousness,
ability, cognitive aptitudes, practically small amount to this extraversion, agreeableness,
job knowledge, and job prediction. openness to experience and
performance. Journal of emotional stability. The claim
Vocational Behavior, 29, 340- that these factors represent the
362. basic dimensions of personality is
21
based on four lines of reasoning: Raymark, P. H., Schmit, M. J., & An even higher mean validity was
Longitudinal and cross-observer Guion, R. M. (1997). Identifying obtained based on studies using
studies have demonstrated potentially useful personality job analysis explicitly in selection
that all five factors are enduring constructs for employee of personality measures.
dispositions that are manifested selection. Personnel Psychology,
in patterns of behavior; traits 50, 723-736.
related to each of the factors This article describes the
BIO DATA TESTS
have been found in a variety Personality-Related Position
of personality systems and in Requirements Form (PPRF), Hough, L. M. (1984).
the natural language of trait a job analysis form used in Development and evaluation
description; the factors have been making hypotheses about of the accomplishment
found in different age, sex, race what personality predictors record methods of selecting
and language groups, although will be relevant for predicting and promoting professionals.
they may be somewhat differently performance in different jobs. Journal of Applied Psychology,
expressed in different cultures; The Big Five personality factors 69, 135-146.
and evidence of heritability has provided an organizing framework This study describes
suggested that all have some for the PPRF. Subsequent the development of the
biological basis. development resulted in accomplishment record method,
identifying 12 specific sets of which resulted in self-reported
items for facets of each of the Big descriptions of accomplishments
Hough, L. M. (1992). The big five in areas that were highly relevant
personality variablesconstruct Five.
for making selection or promotion
confusion: Description versus decisions. The accomplishments
prediction. Human Performance, Tett, R. P., Jackson, D. N., & were reliably rated using specially
5, 135-155. Rothstein, M. (1991). Personality prepared rating scales and
This article suggests that the five- measures as predictors of job guidelines. To test the validity
factor model of personality traits performance: A meta-analytic of the accomplishment record
(extraversion, conscientiousness, review. Personnel Psychology, method, 329 attorneys prepared
agreeableness, emotional 44, 703-742. accomplishment records according
stability and openness to This study examines past research to specified instructions and also
experience) is not an adequate to a) assess the overall validity completed a battery of more
taxonomy of personality variables of personality measures as typical assessment methods.
for predicting important job predictors of job performance; b) Scores on the accomplishment
performance measures. The investigate moderating effects record inventory were unrelated
author argues that the five of several study characteristics to the traditional assessment
factors are too heterogeneous on personality scale validity; and measures (e.g., aptitude tests,
and incomplete. This paper first c) investigate the predictability grades and honors), but they
discusses the development of of job performance as a function correlated with job performance.
personality and job performance of eight personality measures: The accomplishment record had
models. Comparisons between neuroticism, extraversion, also been found to correlate with
different taxonomies are openness, agreeableness, self-perceptions of success, hard
made, and the author presents conscientiousness, locus of work and self-assurance and with
correlational evidence control, Type A and miscellaneous. the length of time spent practicing
demonstrating the usefulness Consistent with predictions, a profession.
of nine personality measures studies using confirmatory
(affiliation, potency, achievement, research strategies produced
dependability, adjustment, corrected mean personality scale Hough, L. M., Keyes, M. A.,
agreeableness, intelligence, validity that was more than twice & Dunnette, M. D. (1983). An
rugged individualism and locus of as high as that based on studies evaluation of three alternative
control). adopting exploratory strategies. selection measures. Personnel
Psychology, 36, 261-276.
22
The authors used a content- the public and private sectors used for pre-employment
oriented validation strategy to (including selection of blue-collar selection decisions about the
develop three alternative selection and federal government employees trustworthiness of job applicants.
inventories that were designed to and work team members), career
reflect the job content of positions counseling and development, and Ones, D. S., Viswesvaran,
held by attorneys employed job classification. C., & Schmidt, F. L. (1993).
with a large federal agency. Comprehensive meta-analysis of
These inventories and three integrity test validities: Findings
traditional assessment methods
INTEGRITY TESTS
and implications for personnel
were completed by 329 agency selection and theories of job
attorneys as part of a concurrent Camara, W. J., & Schneider, D. L.
(1994). Integrity tests: Facts and performance [Monograph].
validation study. Criterion-related Journal of Applied Psychology,
validities of two traditional unresolved issues. American
Psychologist, 49, 112-119. 78, 679-703.
inventories (a background inventory The authors conducted a
and an interest and opinion This article describes two
independent reports, completed comprehensive examination
inventory) and one alternative of the validity of integrity tests
inventory (an accomplishment by the American Psychological
Association (APA) and the U.S. based on 665 validity coefficients.
record inventory) were statistically Results indicated that integrity
and practically significant. Congress Office of Technology
Assessment, that examined test validities were substantial for
scientific and measurement issues predicting job performance and
Shoenfeldt, L. F. (1999). From concerning integrity testing. Data counterproductive behaviors on
dustbowl empiricism to rational were presented on a variety the job, such as theft, disciplinary
constructs in bio data. Human of tests collected by surveying problems and absenteeism. Results
Resource Management Review, test publishers. The survey data of validity studies conducted
9, 147-167. provided a view of the industrys on applicants using external
This paper presents an attempt to scope (e.g., test audience, user performance measures (i.e.,
use different types of biographical screening, score reporting) that excluding self-reports) indicated
data scales in a concurrent was not available elsewhere. The that integrity tests predicted the
validation to predict service article also addresses unresolved broad criterion of organizationally
orientation. Several types of issues regarding integrity testing. disruptive behaviors better than
scales showed positive validity they predicted employee theft
results with the performance alone. The authors concluded that
measures. The results were Goldberg, L. R., Grenier, R. M., integrity test validities were positive
promising in terms of both the Guion, L. B., Sechrest, L. B., across situations and settings.
prediction and understanding of & Wing, H. (1991). Questions
customer service orientation. used in the prediction of
trustworthiness in pre- Sackett, P. R., & Wanek, J. E.
employment selection decisions: (1996). New developments in
Stokes, G. S., Mumford, M. An APA Task Force report. the use of measures of honesty,
D., & Owens, W. A. (Eds.). Washington, DC: American integrity, conscientiousness,
(1994). Biodata handbook: Psychological Association. dependability, trustworthiness,
Theory, research, and use of This report presents the findings and reliability for personnel
biographical information in of an APA task force that was selection. Personnel Psychology,
selection and performance appointed to review available 49, 787-829.
prediction. Palo Alto, CA: CPP commercial tests used for the This article reviews journal
Books. purpose of assessing the honesty articles, books, book chapters,
This book is a comprehensive and integrity of prospective law review articles, convention
reference that provides information employees. The report focuses papers and dissertations
for understanding and using exclusively on commercially regarding integrity testing
biographical data in selection in published instruments for personnel selection.
23
Developments include an 4) have a panel record and rate and decision-making processes.
examination of professional and answers; 5) administer the process Next, they discussed more
congressional inquiry into this to all candidates; and 6) emphasize recent developments in interview
area of testing, rapid growth job relatedness, fairness and research, such as the use of
of the validity database, new documentation. When the interview the interview as a means of
insight into similarities and was used to hire 149 entry-level assessing person-organization
differences between different production employees, it revealed fit and applicant reactions to the
tests, and links to the Big Five high interrogator reliability and employment interview.
personality dimensions. Inquiries predictive validity as well as
into relationships with other evidence for test fairness and utility.
constructs are reviewed, as are
PHYSICAL FITNESS AND
applicant reactions to these tests. PHYSICAL ABILITIES TESTS
Eder, R. W., & Ferris, G. R.
(Eds.). (1989). The employment Blakley, B. R., Quinones, M. S.,
STRUCTURED INTERVIEWS interview: Theory, research, and Crawford, M. S., & Jago, I. A.
practice. Newbury Park, CA: (1994). The validity of isometric
Campion, M. A., Palmer, D. Sage Publishing. strength tests. Personnel
K., & Campion, J. E. (1997). This book provides a Psychology, 47, 247-274.
A review of structure in the comprehensive treatment of This article examines six studies
selection interview. Personnel theory, research and practice in which isometric strength
Psychology, 50, 655-702. relevant to the employment tests were used as part of
This article reviews the research interview. The topics covered selection procedures. The study
literature on the many ways include the interviewers focuses on the relationship
selection interviews can be decision-making process, between performance on four
structured. Fifteen components applicant strategies and isometric strength tests and both
of structure were identified that employment interview validity. supervisory ratings of physical
may enhance either the content There is also a concluding performance and performance on
or the evaluation process of the commentary summarizing the work simulations. Results indicate
interview. The authors conclude volumes implications for theory that isometric strength tests were
that interviews could be easily building, research methods and valid predictors of both types of
enhanced by using the various effective practice. performance measures across all
components of structure. They of the jobs examined.
recommend that improvement of Judge, T. A., Higgins, C. A,
this popular selection procedure & Cable, D. M. (2000). The Campion, M. A. (1983). Personnel
be a high priority for future employment interview: A selection for physically
research and practice. review of recent research and demanding jobs: Review and
recommendations for future recommendations. Personnel
Campion, M. A., Pursell, E. D., & research. Human Resource Psychology, 36, 527-550.
Brown, B. K. (1988). Structured Management Review, 10, 383- This review article argues that
interviewing: Raising the 406. improvements in personnel
psychometric properties of the This comprehensive review selection systems for physically
employment interview. Personnel examines research on the demanding jobs are needed due
Psychology, 41, 25-42. employment interview process. to EEO considerations, concern
This article proposes an employee- The authors began with a for worker physical well-being
interviewing technique that review of the traditional areas and the lack of alternative
includes the following steps: 1) of interview research: reliability, procedures.
develop questions based on validity, structured interviews,
a job analysis; 2) ask the same interviewer differences, equal
questions of each candidate; 3) employment opportunity (EEO)
use examples and illustrations; issues, impression management
24
Hogan, J. (1991). Physical and-pencil instrument. The results cognitive ability, job experience
abilities. In M. D. Dunnette & L. show that the racial differences and conscientiousness, in the
M. Hough (Eds.), Handbook of in situational judgment test prediction of job performance.
industrial and organizational performance and face validity The SJI was a valid predictor in
psychology (Vol. 2, pp. 753- reactions to the test were three samples and produced
831). Palo Alto, CA: Consulting substantially smaller with the substantial increases in validity in
Psychologists Press. video-based method of testing two samples. Relative to the other
This chapter identifies factors compared with the paper-and- predictors, SJIs correlation with
that affect the ability to perform pencil method. performance, controlling for the
in the workplace, including other predictors, was superior in
physiological responses, training, Motowidlo, S. J., Dunnette, most comparisons. The authors
ability and injury, and workload. It M. D., & Carter G. (1990). An concluded that the SJI should
reviews procedures and methods alternative selection procedure: prove to be a valuable additional
for designing personnel selection A low fidelity simulation. Journal measure in the prediction of
systems for physically demanding of Applied Psychology, 75, 640- job performance, but they also
jobs and emphasizes the unique 647. suggested several additional
physical ability content of job From critical-incident analysis areas of research.
analyses, performance measures and judgments by subject-matter
and assessments. In addition, experts, a low-fidelity simulation
the chapter argues that the need
WORK SAMPLE TESTS
was developed for selecting
to improve development and entry-level managers in the
implementation of fair physical Asher, J. J., & Sciarrino, J. A.
telecommunications industry. (1974). Realistic work sample
ability selection procedures The simulation presented
and the need to understand tests: A review. Personnel
applicants with descriptions Psychology, 27, 519-533.
the relationship between of work situations and five
physical performance and other This study reviews the validity
alternative responses for each evidence for a wide sample of
organizational effectiveness situation. It asked them to select
outcomes are important issues for motor and verbal work sample
one response they would most tests, which were designed to
future research. likely make and one they would measure on-the-job behaviors.
least likely make in addressing Motor tests were shown to have
SITUATIONAL each situation. The results show higher validities than verbal
JUDGMENT TESTS that samples of hypothetical tests when job proficiency was
work behavior can predict the performance measure, but
Chan, D., & Schmitt, N. (1997). performance, without the props, this pattern was reversed when
Video-based versus paper and equipment or role players success in training was the
pencil method of assessment often required by high-fidelity performance measure. These
in situational judgment tests: simulations, such as work-sample validities were somewhat lower
Subgroup differences in test tests or assessment centers. than those for biographical
performance and face validity information but higher than those
perceptions. Journal of Applied Weichmann, D., Schmitt, N., & for other predictors.
Psychology, 82, 143-159. Harvey, V. S. (2001). Incremental
This study examines the effects validity of situational judgment Gaugler, B. B., Rosenthal, D. B.,
of race, reading comprehension, tests. Journal of Applied Thornton, G. C., III, & Bentson,
method of assessment, face Psychology, 86, 410-417. C. (1987). Meta-analyses of
validity perceptions and Using three different samples, assessment center validity.
performance on a situational the authors assess the increases Journal of Applied Psychology,
judgment test. The situational in the validity produced by 72, 493-511.
judgment test was administered situational judgment inventories This paper examines the validity
via a videotape and via a paper- (SJIs), relative to job knowledge, of assessment centers across
25
a large number of studies. Thornton, G. C., III. (1992). ADVERSE IMPACT

Higher validities were found Assessment centers in human
in studies in which ratings of resources management. Boston, Roth, P. L., Bevier, C. A., Bobko,
potential were the performance MA: Addison-Wesley. P., Switzer, F. S., & Tyler, P. (2001).
measures, and lower validities While assessment centers Ethnic group differences in
were found in studies in which have been used for numerous cognitive ability in employment
the assessment center was used human resource management and educational settings: A meta-
to make promotion decisions. functions, all assessment centers analysis. Personnel Psychology,
The authors also found that the are not alike. This book shows 54, 297-330.
validities were higher when a) how assessment for managerial The authors conducted a meta-
the percentage of females who selection/promotion, diagnosis analytic review on factors
were being assessed was high, b) of managerial training needs underlying the one standard
when several evaluation devices and team building among deviation effect size difference
were used, (c) when assessors managerial groups are done observed in previous research
were psychologists rather differently. The author also shows for Black-White differences for
than managers, d) when peer how the assessed dimensions, college application tests (e.g.,
evaluation was used, and e) when observations and participant SAT) and overall analyses of
the study was methodologically feedback all must be tailored to tests of g for job applicants
sound. the specific application of the in corporate settings. They
assessment center method. determined that the one standard
Howard, A. (1983). Work deviation summary of group
samples and simulations Thornton, G. C., III, & Byham, W. differences fails to capture many
in competency evaluation. C. (1982). Assessment centers of the complexities in estimating
Professional Psychology: and managerial performance. ethnic group differences in
Research and Practice, 14, 780- New York, NY: Academic Press. employment settings.
796. This book focuses on
This article argues that the examining the assessment Sackett, P. R., & Ellingson, J. E.
evaluation of professional center experience. It traces the (1997). The effects of forming
competence in psychologyby historical development of multiple multi-predictor composites on
licensing boards, the American assessment procedures with group differences and adverse
Board of Professional Psychology emphasis on those advances impact. Personnel Psychology,
or other boardshas not taken relevant to assessment centers; 50, 707-721.
full advantage of expertise in critiques all of the published It is commonly assumed that
psychological measurement and unpublished research including predictors that
within the profession. The on assessment centers; demonstrate smaller group
advantages and disadvantages integrates assessment center differences with others that
of work samples and simulations procedures into several theories demonstrate larger group
for such purposes are discussed of measurement and human differences will help alleviate the
and compared with those of judgment; and presents new adverse impact observed. The
paper-and-pencil tests. Examples models of job analysis, the purpose of this paper was to
from various professions and nature of managerial work, answer the question, If two or
from psychological research work-sampling assessment more predictors that have smaller
in personnel selection are methods and the process of and larger group differences
presented. A comparison of work human judgment based on the are combined, what will be the
samples with simulations shows assessment center experience. magnitude of group differences
greater advantages for the latter, and, consequently, of adverse
but a combination of exercises in impact? A number of conclusions
an assessment center model is were drawn that clarified the
recommended. extent to which combining
26
predictors with smaller and group norming fared in terms of reactions more consistently than
larger group differences affected tradeoffs between performance procedural justice perceptions.
subsequent adverse impact. and minority representation. Procedural justice perceptions
The results showed that the explained incremental variance in
Sackett, P. R., Schmitt, N., preferred strategy depended on some analyses after the influence
Ellingson, J. E., & Kabin, M. B. the relative value the organization of outcome favorability was
(2001). High stakes testing in placed on performance versus controlled.
employment, credentialing, and minority representation. Results
higher education: Prospects in also showed that the effects Gilliland, S. W. (1995). Fairness
a post affirmative-action world. of different screen-then-select from the applicants perspective:
American Psychologist, 56, 302- selection strategies varied as Reactions to employee selection
318. a result of the selection ratios procedures. International
Cognitively loaded tests of at the screening and selection Journal of Selection and
knowledge, skill and ability often stages, thus precluding simple Assessment, 3, 11-19.
contribute to decisions regarding conclusions about the merits of Applicant reactions to selection
education, jobs, licensure or each selection strategy. procedures were examined in
certification. Users of such terms of the satisfaction and/or
tests often face difficult choices APPLICANT REACTIONS violation of 10 procedural justice
when trying to maximize both rules. The author first collected
the performance and ethnic Bauer, T. N., Maertz, C. P., 237 critical incidents describing
diversity of chosen individuals. Dolen, M. R., & Campion, M. A. fair and unfair treatment during
The authors describe the nature (1996). Longitudinal assessment selection processes from 31
of this quandary, review research of applicant reactions to individuals who had recently
on different strategies to address employment testing and test participated in job search and
it and recommend using selection outcome feedback. Journal of hiring. The critical incidents were
materials that assess the full Applied Psychology, 83, 892- categorized into 10 procedural
range of relevant attributes using 903. justice rules, and the distribution
a format that minimizes verbal This study uses a justice of these incidents was examined
content as much as is consistent framework to examine reactions for different hiring outcomes and
with the outcome one is trying of candidates for an entry-level different selection procedures.
to achieve. Regardless of the accounting job at three time The most common procedural
strategy adopted, however, they periods: before testing, after concerns reflected selection
suggest that it is unreasonable testing but before feedback procedure job relatedness and
to expect that one can maximize on whether they passed or the interpersonal treatment
both the performance and ethnic failed the test, and after-test applicants had received.
diversity of selected individuals. performance feedback. Several
procedural justice measures Hausknecht, J. P., Day, D.
Sackett, P. R., & Roth, L. (1996). (information known about the V., & Thomas, S. C. (2004).
Multi-stage selection strategies. test, chance to perform, treatment Applicant reactions to selection
A Monte Carlo investigation at the test site, consistency procedures: An updated model
of effects on performance of the test administration and and meta-analysis. Personnel
and minority hiring. Personnel job relatedness) predicted Psychology, 57, 639-683.
Psychology, 49, 549-562. applicant evaluations regarding An updated theoretical model of
This study examines the effects the organization, perceptions of applicant reactions to selection
of selection process variables employment testing and applicant procedures is proposed and
on employee performance and test-taking self-efficacy. Test tested in this article. Results
on minority hiring. The authors outcome favorability (passing indicate that applicants who
look at how selection rules that or failing the employment test) held positive perceptions about
did and did not include within- predicted outcomes beyond initial selection were more likely to
27
view the organization favorably to candidates and to selection includes a review of research
and report stronger intentions to procedure validity and utility. In bearing on the accommodation
accept job offers and recommend part one of this two-part study, 110 of tests and testing procedures
the employer to others. The newly hired entry-level managers for examinees with disabilities
average correlation between and 44 recruiting-employment and presents the experiences
applicant perceptions and gender, managers judged simulations, of two large eastern states and
age and ethnic background interviews and cognitive tests their programs of accommodation
was near zero. Face validity and with relatively concrete item types testing.
perceived predictive validity (such as vocabulary, standard
were strong predictors of many written English and mathematical Eyde, L. D., Nester, M. A.,
applicant perceptions, including word problems) as significantly Heaton, S. M., & Nelson, A. V.
procedural justice, distributive more job-related than personality, (1994). Guide for administering
justice, attitudes toward tests bio data and cognitive tests with written employment
and attitudes toward selection. relatively abstract item types examinations to persons
Interviews and work samples (such as quantitative comparisons with disabilities. Washington,
were perceived more favorably and letter sets). A measure of DC: U.S. Office of Personnel
than cognitive ability tests, which new managers cognitive abilities Management.
were perceived more favorably was positively correlated with This paper provides guidance
than personality inventories, the managers perceptions of for administering written
honesty tests, bio data and the job relatedness of selection examinations to persons with
graphology. procedures. In part two, the disabilities. It also includes
reactions of 460 applicants to a information on legal requirements
Ryan, A. M., Greguras, G. J., & range of entry-level to professional for reasonable accommodations
Ployhart, R. E. (1996). Perceived civil service examinations were and offers detailed guidelines
job relatedness of physical found to be positively related to for testing applicants who are
ability testing for firefighters: procedural and distributive justice deaf or hard of hearing or who
Variations in reactions. Human perceptions and willingness to have vision or motor impairments.
Performance, 9, 219-240. recommend the employer to The paper includes a checklist
This research explores others. for physical accessibility of test
perceptions of physical sites and information on effective
abilities testing by examining AMERICANS WITH personal interactions with
the perceived job relatedness DISABILITIES ACT persons with disabilities.
of physical abilities tests.
Experience, self-efficacy and Daley, L., Dolland, M., Kraft, Williams, S. K. (2003, January/
consistency of test administration J., Nester, M. A., & Schneider, February). Tips for minimizing
were found to be related to R. (1988). Employment testing abuses under the Americans
perceptions of job relevance. of persons with disabling with Disabilities Act. Alexandria,
conditions. Alexandria, VA: VA: Society for Human Resource
Smither, J. W., Reilly, R. R., International Personnel Management.
Millsap, R. E., Pearlman, K., & Management Association. This paper provides a discussion
Stoffey, R. (1993). Applicant This is a monograph based of the law, its interpretation
reactions to selection on a symposium presented and implications for use of
procedures. Personnel at both the 1987 International various selection techniques in
Psychology, 46, 49-76. Personnel Management relation to the Americans with
The premise of this research Association Assessment Council Disabilities Act of 1990. It covers
is that applicant reactions to (IPMAAC) Conference and the physical exams, psychological
selection procedures may 1987 International Personnel exams, health questionnaires,
be of practical importance to Management Association (IMPA) physical agility tests, drug and
organizations attractiveness International Conference. It alcohol tests, specific oral and
28
written questions (e.g., about the Uniform guidelines on employee in two top-tier journals (Journal
existence of a disability, about selection procedures. (1978). of Applied Psychology and
ability to perform job-related Federal Register, 43, 38295- Personnel Psychology) from 1980
functions), and specific requests 38315. to 2010 to produce empirical
(e.g., how the applicant would These guidelines incorporate a effect size benchmarks for 20
perform job-related functions single set of principles designed common research domains.
with or without reasonable to assist employers, labor Results indicate that the usual
accommodation). organizations, employment interpretation and classification of
agencies, and licensing and effect sizes in statistical methods
certification boards to comply books as small, medium and large
LEGAL GUIDANCE with requirements of federal law bear almost no resemblance to
prohibiting employment practices findings in the field.
Azar, B. (1994). Could policing that discriminate on the grounds
test use improve assessments? of race, color, religion, sex and
APA Monitor, 16. Cohen, J. (1988). Statistical
national origin. power for the behavioral
This article discusses the
American Psychological sciences (2nd ed.). Hillsdale, NJ:
Associations Standards for GENERAL REFERENCES Erlbaum.
Educational and Psychological This book is a guide to power
Testing. It discusses issues Borman, W. C., & Motowidlo, analysis in research planning
surrounding the enforcement S. J. (1993). Expanding the that provides users of applied
of the standards as well as the criterion domain to include statistics with the tools they need
possibility of developing more elements of contextual for more effective analysis.
specific standards. performance. In N. Schmitt
& W. Borman and Associates Finch, D. M., Edwards, B. D., &
Society for Industrial and (Eds.), Personnel selection in Wallace, J. C. (2009). Multistage
Organizational Psychology. organizations (pp. 71-98). San selection strategies: Simulating
(2003). Principles for the Francisco, CA: Jossey-Bass. the effects on adverse impact
validation and use of personnel In this chapter, the authors and expected performance for
selection procedures (4th ed.). argue that selection criteria various predictor combinations.
Bowling Green, OH: Author. should embrace a domain Journal of Applied Psychology,
This document outlines of organizational behavior 94, 318-340.
principles adopted by the broader than just technical The authors use Monte Carlo
Society for Industrial and task performance. Specifically, simulation to explore the trade-
Organizational Psychology (SIOP) performance measures should off between expected mean
of the American Psychological be expanded to also include performance and minority hiring
Association for the validation and contextual activities, such as in multistage selection strategies
use of personnel selection and helping co-workers when needed and to identify those strategies
assessment procedures, including and putting in extra effort to get most effective in balancing the
performance appraisal. The the job done. trade-off. The results show that
principles specify SIOP policy on an increase in minority hiring
conducting validation research Bosco, F. A., Aguinis, H., Singh, was associated with a decrease
using principles of good practice K., Field, J. G., & Pierce, C. A. in predicted performance for
in the choice, development and (2014). Correlational effect size many scenarios, although the
evaluation of personnel selection benchmarks. Journal of Applied results also show that certain
procedures. Psychology, 100, 431-449. multistage strategies are much
The authors examine a large more effective than others for
number (147,328) of correlations managing the performance and
from published research in adverse impact trade-offs.
applied psychology reported
29
Heneman, H. G., III, & Judge, provided the best prediction of Ryan, A. M., & Tippins, N. T.
T. A. (in press). Staffing job-specific and general technical (2004). Attracting and selecting:
organizations (5th ed.). task proficiency, whereas the What psychological research
Middleton, WI: Mendota House, temperament/personality tells us. Human Resource
and New York, NY: McGraw-Hill. measures best predicted giving Management, 43, 305-318.
The book is filled with up-to-date extra effort, supporting peers and This article reviews research
research, useful examples and exhibiting personal discipline. on which selection tools work,
best business practices in the Scores from the interest inventory which recruitment strategies
staffing area. In addition, each correlated more highly with work, how selection-tool use
chapter concludes with in-depth technical task proficiency than relates to workforce diversity,
applications (cases and exercises) with demonstrating effort and and which staffing and recruiting
that enhance skills and provide peer support. processes lead to positive
practice in key staffing activities applicant perceptions. The goal
and related decision-making. Murphy, K. (1996). Individual is to assist the reader not only in
differences and behavior in understanding the present gaps
Kehoe, J. (2000). Managing organizations. San Francisco, between research and practice in
selection in changing CA: Jossey-Bass. recruitment and selection but also
organizations. San Francisco, This book integrates existing in developing skills for employing
CA: Jossey-Bass. research and stimulates new ways research in HR practice.
This book provides managers of thinking about how individual
and HR practitioners with differences affect peoples Rynes, S. L., Colbert, A. E.,
practical guidance on making behaviors and experiences in & Brown, K. G. (2002). HR
decisions about employee organizations. It provides insights professionals beliefs about
staffing. The chapters are into what influences behavior on effective human resources
authored by leading researchers the job. practices: Correspondence
and practitioners with extensive between research and practice.
experience in the staffing area Potosky, D., & Bobko, P. Human Resource Management,
and offer proven strategies for (2004). Selection testing 41, 149-174.
the design and management of via the Internet: Practical In this study, 5,000 human resource
effective selection processes in considerations and exploratory professionals were surveyed
organizations. empirical findings. Personnel regarding the extent to which they
Psychology, 57, 1003-1034. agreed with various HR research
McHenry, J. J., Hough, L. M., This article presents equivalence findings. Responses suggested
Toquam, J. L., Hanson, M. information and practical lessons that large discrepancies existed
A., & Ashworth, S. A. (1990). concerning selection testing via between research findings and
Project A validation results: The the Internet. The authors identify practitioners beliefs in some areas,
relationships between predictor several issues associated with especially staffing. In particular,
and criterion domains. Personnel measurement and validity, the practitioners placed far less faith
Psychology, 43, 335-353. role of individual characteristics, in intelligence and personality
In this study, a predictor respondents reactions tests as predictors of employee
battery of cognitive ability, and behaviors, and other performance than HR research
perceptual-psychomotor ability, considerations regarding Internet would recommend.
temperament/ personality, interest test administration. They also
and job outcome preference report results from an exploratory Schmitt, N., & Borman, W. C.
measures was administered to study of the correlation between (1995). Personnel selection in
enlisted soldiers in nine Army paper-and-pencil and Internet- organizations. San Francisco,
occupations. Scores from the administered cognitively oriented CA: Jossey-Bass.
cognitive and perceptual- selection tests (including timed In this book, experts highlight the
psychomotor ability tests and untimed, proctored tests). personnel selection issues that
30
will receive increasing attention

in the years ahead. The book
examines specific topics such
as recruitment and retention,
structured-versus-unstructured
interviews, the ethics and
effectiveness of computerized
psychological testing, perceptions
of selection fairness, productivity,
turnover, and absenteeism.
Schmidt, F. L., & Hunter, J.

E. (1998). The validity and
utility of selection methods in
personnel psychology: Practical
and theoretical implications of
85 years of research findings.
Psychological Bulletin, 124, 262-
274.
This article summarizes the
practical and theoretical
implications of 85 years of
research in personnel selection.
It presents the validity of
19 selection procedures for
predicting job and training
performance and the validity of
paired combinations of general
mental ability (GMA) and the 18
other selection procedures.
31
Technology will continue to spark revolutionary changes in assessment methods

and practices.
Quiderum num faccuptus es ratem res simaion necersperum la simet aut
endanihicium fuga. Ut omniam aut omnimil ius doleste nderio ellabo. Ut oditatus
et molorio tenis molorepel imusam.
- Quotee to come
32
ENDNOTES
1. CEB. (2016). Competing for critical talent [Webinar].

2. CEB. (2012). Q4 global labor market survey.
3. CEB. (2010). Candidate rules of engagement survey.
4. CEB Corporate Leadership Council. Creating enterprise
leaders.
5. Bauer, T. N., Maertz, C. P., Dolen, M. R., & Campion, M. A.
(1996). Longitudinal assessment of applicant reactions to
employment testing and test outcome feedback. Journal of
Applied Psychology, 83, 892-903.
6. Kantrowitz, T. M. (2014). Global assessment trends report 2014.
Arlington, VA: CEB.
7. Sullivan, J. (2014). The power has shifted to the candidate, so
current recruiting practices will stop working. ERE Recruiting
Intelligence. Retrieved from http://www.eremedia.com/ere/
the-power-has-shifted-to-the-candidate-so-current-recruiting-
practices-will-stop-working/
8. Beaty, J. C., Dawson, C. R., Fallaw, S. S., & Kantrowitz, T. M.
(2009). Recovering the scientist-practitioner model: How
IOs should respond to UIT. Industrial and Organizational
Psychology: Perspectives on Science and Practice, 2, 58-63.
9. Tippins, N. T. (2009). Internet alternatives to traditional
proctored testing: Where are we now? Industrial and
Organizational Psychology: Perspectives on Science and
Practice, 2, 2-10
10. Ibid.
11. Golubovich, J., & Boyce, A. (2013). Hiring tests: Trends in mobile
test usage. In N. Morelli, Mobile devices in talent assessment:
Where are we now? Symposium presented at the 28th annual
conference of the Society for Industrial and Organizational
Psychology, Houston, TX.
33
12. Illingworth, J., Morelli, N., (Ed.). (1982). The g factor Development and evaluation
Scott, S., & Boyd, S. (2014). in employment. Journal of of the accomplishment
Internet-based, unproctored Vocational Behavior, 29(3). record methods of selecting
assessments on mobile and and promoting professionals.
19. Raymark, M. J., Schmit, M.
non-mobile devices: Usage, Journal of Applied Psychology,
J., & Guion, R. M. (1997).
measurement equivalence, and 69, 135-146.
Identifying potentially useful
outcomes. Journal of Business
personality constructs for 28. Hough, L. M., Keyes, M. A.,
and Psychology, 30, 25-34.
employee selection. Personnel & Dunnette, M. D. (1983). An
13. Morelli N., Mahan, R. P., Psychology, 50, 723-736. evaluation of three alternative
& Illingworth A. J. (2014). selection measures. Personnel
20. Costa, P. T., Jr., & McCrae, R. R.
Establishing the measurement Psychology, 36, 261-276.
(1992). Four ways five factors
equivalence of online selection
are basic. Personality and 29. Hunter, J. (1986). Cognitive
assessments delivered on
Individual Differences, 13, 653- ability, cognitive aptitudes,
mobile versus non-mobile
665. job knowledge, and job
devices. International Journal
performance. Journal of
of Selection and Assessment, 21. Hough, L. M. (1992). The big five
Vocational Behavior, 29, 340-
22, 124-138. personality variables-construct
362.
confusion: Description
14. Ibid.
versus prediction. Human 30. Motowidlo, S. J., Dunnette,
15. Gael, S. (Ed.). (1988). The Performance, 5, 135-155. M. D., & Carter G. (1990). An
job analysis handbook for alternative selection procedure:
22. Tett, R. P., & Christiansen, N.
business, industry, and A low fidelity simulation.
D. (2007). Personality tests at
government. New York: Wiley; Journal of Applied Psychology,
the crossroads: A response to
Brannick, M. T., & Levine, E. L. 75, 640-647.
Morgeson, Campion, Dipboye,
(2002). Job analysis. Thousand
Hollenbeck, Murphy, and 31. Kantrowitz, T. M. (2014). Global
Oaks, CA: Sage.
Schmitt (2007). Personnel assessment trends report 2014.
16. LoVerde, M. A., & Schmidt, C. Psychology, 60(4), 967-993. Alpharetta, GA: SHL.
(2016, April). Overview of the
23. Ibid. 32. Asher, J. J., & Sciarrino, J. A.
CEB leadership study (CLS).
(1974). Realistic work sample
In M. A. LoVerde, Overview 24. Ones, D. S., Viswesvaran,
tests: A review. Personnel
and selected findings from a C., & Schmidt, F. L. (1993).
Psychology, 27, 519-533.
multi-organizational, multi-level Comprehensive meta-analysis
leadership study. Symposium of integrity test validities: 33. Hogan, J. (1991). Physical
conducted at the 31st annual Findings and implications abilities. In M. D. Dunnette & L.
conference of the Society for for personnel selection and M. Hough (Eds.), Handbook of
Industrial and Organizational theories of job performance. industrial and organizational
Psychology, Anaheim, CA. Journal of Applied Psychology, psychology (Vol. 2, pp. 753-
78, 531-537. 831). Palo Alto, CA: Consulting
17. Hunter, J. (1986). Cognitive
Psychologists Press.
ability, cognitive aptitudes, 25. Stokes, G. S., Mumford, M. D.,
job knowledge, and job & Owens, W. A. (Eds.). (1994). 34. Hausknecht, J. P., Day, D.
performance. Journal of Biodata handbook. Palo Alto, V., & Thomas, S. C. (2004).
Vocational Behavior, 29, 340- CA: CPP Books. Applicant reactions to selection
362. procedures: An updated model
26. Shoenfeldt, L. F. (1999). From
and meta-analysis. Personnel
18. Ree, M. J., Earles, J. A., dustbowl empiricism to rational
Psychology, 57, 639-683;
& Teachout, M. S. (1994). constructs in biodata. Human
Ryan, A. M., Greguras, G. J., &
Predicting job performance: Resource Management Review,
Ployhart, R. E. (1996). Perceived
Not much more than g. Journal 9, 147-167.
job relatedness of physical
of Applied Psychology, 79,
27. Hough, L. M. (1984). ability testing for firefighters:
518-524; Gottredson, L. S.
34
Variations in reactions. Human 38. Sackett, P. R., Schmitt, N., Organizational Psychology,
Performance, 9, 219-240; Ellingson, J. E., & Kabin, M. B. Honolulu, HI.
Schmidt, F. L., & Hunter, J. E. (2001). High stakes testing in
45. Gutierrez, S. L., & Meyer, J. M.
(1998). The validity and utility of employment, credentialing, and
(2014). The mobile revolution:
selection methods in personnel higher education: Prospects in
Measurement equivalence and
psychology: Practical and a post affirmative-action world.
mobile device administration. In
theoretical implications of 85 American Psychologist, 56,
T. Kantrowitz & C. M. Reddock,
years of research findings. 302-318.
Shaping the future of mobile
Psychological Bulletin, 124,
39. Ibid. assessment: Research and
262-274; Roth, P. L., Bevier,
practice update. Symposium
C. A., Bobko, P., Switzer, F. 40. Heneman, H. G., III, &
presented at the 29th Annual
S., & Tyler, P. (2001). Ethnic Judge, T. A. (2006). Staffing
Conference of the Society for
group differences in cognitive organizations (5th ed.).
Industrial and Organizational
ability in employment and Middleton, WI: Mendota House,
Psychology, Honolulu, HI.
educational settings: A meta- and New York, NY: McGraw-
analysis. Personnel Psychology, Hill. 46. Sireci, S. G., Scarpati, S.
54, 297-330; Barrick, M. R., & E., & Li, S. (2005). Test
Mount, M. K. (1991). The big five 41. Ibid.
accommodations for students
personality dimensions and job with disabilities: An analysis
42. Smither, J. W., Reilly, R. R.,
performance: A meta-analysis. of the interaction hypothesis.
Millsap, R. E., Pearlman, K., &
Personnel Psychology, 91, 1-26. Review of Educational
Stoffey, R. (1993). Applicant
reactions to selection Research, 75(4), 457-490.
35. Heneman, H. G., III, & Judge, T.
A. (2006). Staffing organizations procedures. Personnel
(5th ed.). Middleton, WI: Psychology, 46, 49-76.
Mendota House, and New York, 43. Gilliland, S. W. (1995).
NY: McGraw-Hill.
Fairness from the applicants
36. Borman, W. C., & Motowidlo, perspective: Reactions
S. J. (1993). Expanding the to employee selection
criterion domain to include procedures. International
elements of contextual Journal of Selection and
performance. In N. Schmitt, Assessment, 3, 11-19; CEB
W. Borman, and Associates Recruiting Leadership Council.
(Eds.), Personnel selection (2013). Selecting high-quality
in organizations (pp. 71-98). hires for todays new work
San Francisco: Jossey-Bass; environment; CEB Recruiting
Hattrup, K., Rock, J., & Scalia, Leadership Council. (2013).
C. (1997). The effects of Candidates as customers:
various conceptualizations of Optimizing the candidate
job performance on adverse experience.
impact, minority hiring, and 44. Kinney, T. B., Lawrence,
predicted performance. Journal
A., & Change, L. (2014).
of Applied Psychology, 82,
Understanding the mobile
656-664.
experience: Data across device
37. Sackett, P. R., & Roth, L. and industry. In T. Kantrowitz
(1996). Multi-stage selection & C. M. Reddock, Shaping the
strategies. A Monte Carlo future of mobile assessment:
investigation of effects on Research and practice update.
performance and minority Symposium presented at the
hiring. Personnel Psychology, 29th Annual Conference of
49, 549-562. the Society for Industrial and
35
The future of HR is here
(and its SHRM-certified)
Todays workplace demands that HR professionals deliver innovative thinking and measurable
business results. The SHRM-CP and SHRM-SCP, the new standard in HR certification, uses
competencies to test not only what you know, but how you apply that knowledge in the workplace.
Join the more than 90,000 professionals whove earned their SHRM Certification,
and demonstrate that you are the future of HR.
Get Certified Now:

16-0179
shrmcertification.org/foundation
About the SHRM Foundation
At the SHRM Foundation, we are a catalyst for thought leadership. We help predict where the workforce is
headed because weve been studying its evolution for nearly 50 years. Our mission is to offer unmatched human
capital knowledge for the benefit of professional organization leaders with a total focus on studying and reporting
the management practices that work. Supporting the Foundation is a chance to contribute to an ongoing study
about the direction of human resources in society. The Foundation is governed by a volunteer board of directors,
comprising distinguished HR academic and practice leaders. Contributions to the SHRM Foundation are tax-
deductible. The SHRM Foundation is a 501(c)(3) nonprofit affiliate of the Society for Human Resource Management
(SHRM). For more information, contact the SHRM Foundation at 703.535.6020. Online at shrmfoundation.org.

January 2017 Monthly Article - Choosing Effective Talent Assessments To Strengthen Your Organization

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

January 2017 Monthly Article - Choosing Effective Talent Assessments To Strengthen Your Organization

Uploaded by

Copyright:

Available Formats

EPG

Choosing Effective Talent

Organizational success is largely driven by the acquisition of the best talent

This report is part of the SHRM Foundations Effective Practice Guidelines

The SHRM Foundation champions workforce and workplace transformation

We encourage you to learn more. Please visit shrmfoundation.org to

Mark Schmit, Ph.D., SHRM-SCP

REVIEWERS PROJECT MANAGER

Jim Link, SHRM-SCP

ABOUT THE AUTHORS

Tracy Kantrowitz, Ph.D., is research director at CEB Talent Assessment. In

Tracy was previously chair of the Professional Practice Committee for

Dr. Kantrowitz holds a Ph.D. in industrial/organizational psychology from the

Research reveals that careful talent assessment results in significant

CHOOSING EFFECTIVE TALENT

THE BUSINESS CASE FOR EFFECTIVE TALENT ASSESSMENT

Questions Leaders Are Asking Organization Challenges

Each position has many applicants,

How can we identify effective Only 12 percent of todays

Given the proven bottom-line value of attracting and retaining

Although the term assessment can include a wide variety

THE ASSESSMENT LANDSCAPE services.5 Approximately two- Assessment Trends

that their organizations are analytics are introducing

Test Equivalence Is a Key Issue Technological Advances

for a job. To ensure candidate as much as by their measurement

report, we will look at the science Investigator Job: Sample Competencies

Work behavior is the tip of

combination to provide a measure of job performance,18 particularly Example of an Integrity

automated scoring algorithms are Example of a Job

writing demands, such as attorney,

Structured Behavioral Interview Question and Sample Rating Criteria

Low Moderate High

Evaluation of Assessment Methods on Four Key Criteria

What is an assessment Validity: The Most Important Consideration

Example of Criterion Comparisons of criterion and Looking at the

small number of protected group evaluated. The U.S. Office of Federal

members hiredreferred to as the adverse impact is not widely

Business outcome gains are

requires specialized expertise An overall great candidate

The Importance of Compliance Federal Contractors

In recent years, technological advances, competitive demands

Always remember, however, that the primary criterion for

Quiderum num faccuptus es ratem res simaion necersperum la simet aut

SOURCES AND SUGGESTED

Brannick, M. T., & Levine, E. L. (2002). Job analysis. Thousand

Gael, S. (Ed.). (1988). The job analysis handbook for business,

UNPROCTORED AND MOBILE ASSESSMENT

Beaty, J. C., Dawson, C. R., Fallaw, S. S., & Kantrowitz, T. M.

test appear to be equivalent to A review of the research literature PERSONALITY ASSESSMENTS

a large number of studies. Thornton, G. C., III. (1992). ADVERSE IMPACT

will receive increasing attention

Schmidt, F. L., & Hunter, J.

Technology will continue to spark revolutionary changes in assessment methods

1. CEB. (2016). Competing for critical talent [Webinar].

Get Certified Now:

You might also like