HIRING MANAGER USA STAFFING ASSESSMENT GUIDE

USA Staffing Assessment Questionnaire Development Guidelines

This Assessment Guide provides information and guidance to support the candidate evaluation process with the new USA Staffing tool. USA Staffing is replacing Resumix as the Department of the Navy staffing and referral system. USA Staffing uses task based statements in assessment questionnaires that applicants address during the application process. These questionnaires are used to identify both qualified and best qualified candidates. Questionnaires are developed through the job analysis and assessment development process. Although your HRSC specialist will guide you through the process, two short 10 minute Computer Based Training sessions on job analysis and assessment are available on line to introduce hiring managers to the job analysis and assessment process. Overview of Job Analysis http://www.public.navy.mil/donhr/Employment/HiringReform1/Overview%20of%20Job%20Analys is.htm Overview of Assessment Development http://www.public.navy.mil/donhr/Employment/HiringReform1/Overview%20of%20Employment% 20Assessment.htm Background on Assessment Assessments in federal hiring gather information to distinguish qualifications amongst applicants. Assessing applicants using valid, reliable tools has been a part of the hiring process for many years, is a procedure specified in the Presidents hiring reform directives and is required by federal regulation. This fact sheet defines assessment in the new hiring process, explains its importance and highlights the process for creating an assessment plan. Assessments Defined Assessments are measurement tools that allow for fair and consistent methods to determine which applicants are best qualified for a position. They use questions to determine the experience, education and competencies of an applicant. Types of assessments include questionnaires, structured interviews, written tests and work samples. They are called by various names occupational questionnaires, rating schedules, crediting plans, etc. When utilizing USA Staffing, an assessment is commonly accomplished through an occupational questionnaire.

June 2011, Page 1 of 6

DON USA Staffing Assessment Guide

HIRING MANAGER USA STAFFING ASSESSMENT GUIDE

Importance of an Effective Assessment An effective assessment is important because it helps ensure that all high quality candidates are considered by the hiring manager and that new hires are well-suited to excel on the job. High Quality Candidates Assessments systematically gather valuable information about applicants and make consistent, accurate, and merit-based distinctions among them. Therefore, better quality candidates are forwarded for consideration. Well-Suited New Hires When the assessment tool assesses candidates based on the competencies needed to perform the position well, hiring managers will likely select new hires whose skill set meets the job requirements. Well-suited new hires are typically better performers on the job and are more satisfied with their position. Satisfied, superior performers will better help the Department of the Navy meet its mission. Process for Creating an Effective Assessment An effective assessment is based on a thorough job analysis which identifies the most critical competencies or knowledges skills and abilities (KSAs) that the position requires. When the assessment tool is well-aligned to competencies and KSAs critical for the position, hiring managers will tend to select employees whose skill set meets the job requirements. The following steps outline the process for hiring managers to create an effective assessment: Collaboratively Develop an Assessment Strategy Partner with a human resources professional to confirm which knowledge, skills and abilities (KSAs) identified through the job analysis are critical for the position, set targets for the total number of KSAs needed for the assessment and determine if weights for each KSA will be used. Develop the Minimum Qualification Statement Use the job analysis information to draft a statement of directly-related experiences that an applicant should have to meet the basic qualifications for the job. Write Task Statements Develop a list of relevant tasks that measure the extent of the applicants experience performing tasks associated with each KSA identified in the job analysis. Additional more detailed information on assessment development is contained in this USA Staffing Assessment Questionnaire Guide.

June 2011, Page 2 of 6

DON USA Staffing Assessment Guide

HIRING MANAGER USA STAFFING ASSESSMENT GUIDE

USA Staffing Assessment Questionnaire Development Guidelines

Your HRSC Specialist should provide you with a draft job analysis and assessment document, when reviewing the content of these documents, please use the guidelines listed below. Developing Minimum Qualifications Questions

Write specialized experience descriptions that clearly and accurately depict experience gained at
the next lower grade level based on relevant job analysis information (e.g., the qualifications and position classification standards, relevant position descriptions, including those at the next lower grade level)

Write specialized experience descriptions that are verifiable on the basis of a resume or other

required supporting documentation. Avoid words such as should, may or such as. Instead use words such as need to have

Ensure distinctions between grade levels, as appropriate, are clearly depicted Ensure education requirements are completely articulated. Note that we cannot add educational
requirements outside those prescribed by OPM.

Verify there is a suitable response option for qualified and unqualified applicants Identify selective placement or screen out factors required
Identifying Ranking Factors

Identify four to eight1 critical competencies or knowledge, skills or abilities to assess based on a
careful review of available job analysis information, including the position classification standards and relevant position descriptions

Verify that the ranking factors:

Are important Are required at entry Will distinguish superior performers from average performers Are not redundant with other factors being assessed Are measurable using a training and experience-based questionnaire Are verifiable on the basis of the applicants resume or other supporting documentation

The number of competencies assessed will depend on a number of factors including the complexity of the position and the exact purpose of the questionnaire. Four to eight is recommended as a good target range June 2011, Page 3 of 6 DON USA Staffing Assessment Guide

HIRING MANAGER USA STAFFING ASSESSMENT GUIDE

Writing Questions to Measure Each Ranking Factor

Develop four to eight2 task-based questions to measure each ranking factor Write items following the below formula:
Perform what? (precise action verb) + To whom or what? (direct object) + To produce what? why? or how? (qualifying statement) Ex: Proofread + outgoing correspondence + to address formatting, grammatical, or typographical errors and consistency

Write questions in behavioral, observable and verifiable terms Write questions that are clear and succinct Avoid subjective terms or adjectives (e.g., effective, all) or adverbs (e.g., accurately) Avoid agency-specific terminology and acronyms were the applicant pool might not have

experience with or knowledge of the term. For merit promotion with a limited AOC, its acceptable to add agency specific information if the applicant pool has experience with the system or term.

Write questions that are likely to create meaningful distinction among applicants Write questions that reflect a range of proficiency or experience in order to differentiate
applicants

Write questions that reflect a single task or behavior (i.e., applicant could have had experience
with one task but not the other)

Write questions that make sense with the rating scale being used
Additional Information

Consider each of the questions on page 3 Specific Questions to Consider When Developing &
Reviewing Assessment Questionnaire Items for Automated Delivery See the Sample Task Questions on page 4 for tasks associated with typical ranking factors

The number of items for each factor will depend on a number of variables (e.g., number of competencies covered, complexity of competences, types of questions and rating scales). Four to eight items for each competency being measured is recommended as a good target range. A target range for a full questionnaire is 15 to 30 items. June 2011, Page 4 of 6 DON USA Staffing Assessment Guide

HIRING MANAGER USA STAFFING ASSESSMENT GUIDE

Specific Questions to Consider When Developing & Reviewing Assessment Questionnaire Items for Automated Delivery
Is each question specific and clear?

Does each question contain a precise, unambiguous action verb, a direct object and a qualifying statement? Is the wording of the question too broad or ambiguous in any way? (i.e., Will every applicant understand it in
the same way? Is there a chance that the applicant could misinterpret the item? Or interpret it to his/her advantage?)

Is the question written as clearly and succinctly as possible, such that the applicant can read it once and
respond accurately?

Particularly for entry-level positions, is each question written in a way that is specific enough to measure the
competencies needed at the time of hire but general enough so that qualified applicants are not excluded based on details specific to the position?

Particularly for positions being advertised externally (outside the agency), does the question avoid using any
agency-specific jargon or terminology that applicants outside of the organization may not understand? Can each question be measured and verified?

Can the job-relatedness of each item be demonstrated? That is, is there documentation that substantiates the
relationship between the tasks performed on the job, the competencies (also called factors) being assessed, and the items being used to measure each competency?

Is the question written in behavioral, observable, and verifiable terms (i.e., will the applicants previous
supervisor, teacher or colleague be able to verify the applicants response?)

Does each question measure the competency or factor it is intended to measure?

Are all items collectively effective?

Does each question make sense with regard to the rating scale that is being used? Is the question conveying one specific task? If the item mentions two tasks, are both tasks always performed

in conjunction with one another (i.e., the applicant couldnt have experience performing one task and not the other)? (e.g., accurately, highly, timely)? If a subjective term (e.g., complex) must be used, are qualifying examples included?

Is the question free of subjective terms or adjectives (e.g., effective, adequate, complex, all, some) or adverbs Will each question help to differentiate among applicants (or, for example, will all applicants be able to rate
the item high)?

Are all spelling, typographical or grammatical errors corrected and acronyms spelled out? When reviewed collectively, do the questions capture the most important aspects of each respective
competency (factor)?

When reviewed collectively, is the number of questions reasonable given the competencies being assessed?
Can any items be removed without compromising the validity of the assessment questionnaire?
June 2011, Page 5 of 6 DON USA Staffing Assessment Guide

HIRING MANAGER USA STAFFING ASSESSMENT GUIDE

Sample Task Questions Knowledge of proper storage procedures and working with classified documents
1. Maintain property records to account for equipment 2. Review supply documentation to ensure completeness and verify funding and approval 3. Compile and analyze supply management data and statistics for report preparation

Knowledge of supply principles, concepts and methodologies

1. Write supply program requirements for inventory management data processing systems 2. Interpret supply management regulations and directives to determine inventory management requirements 3. Conduct supply management surveys to provide data for policy implementation 4. Manage the withdrawal and turn-in of property in accordance with established supply principles 5. Use automated systems to maintain records of supply items in inventory 6. Analyze new or proposed legislation and regulations to evaluate the impact on supply program operations 7. Arrange for the transfer of accountable property 8. Recommend and implement supply management policies and procedures to ensure operational accountability of property

Oral communication
1. Coordinate with supply personnel and organizations to ensure compliance with property accountability procedures and policies 2. Facilitate discussion during inventory management conferences 3. Exchange information with personnel in supply, production, maintenance and organization to resolve inventory management problems

Written communication
1. Write accountable property Standard Operating Procedures (SOPs) to ensure continuity of effort during personnel changeover 2. Prepare reports or documents on equipment accountability 3. Write briefings or training materials to present to property custodians

Skills in conducting organizational visits to ensure compliance

1. Provide technical guidance to others on supply systems, policy, procedures or guidelines 2. Review supply program investigative and audit reports to ensure compliance with applicable laws 3. Evaluate organizational supply operations to ensure compliance with policy, guidance and/or directives

Ability to plan, organize and meet deadlines

1. Establish and maintain inventory schedules to meet established timeframes 2. Respond to inquiries on the status of requisitions and shipments 3. Advise management on the potential benefits and uses of automation to improve supply programs operation

June 2011, Page 6 of 6

DON USA Staffing Assessment Guide