Evaluation Plan FRIT 8435

Program Evaluation- Final Report 1
Program Evaluation- Final Report

Angela Kennedy, Jesse Scott, and Sam Wainford
Georgia Southern University
FRIT 8435 Program Evaluation
Dr. Carlson
04/23/2012
Program Evaluation Final Report, April 23, 2012
Group 3: Angela Kennedy, Jesse Scott, Philis (Sam) Wainford

Page 2 of 47

Table of Contents
I. Executive Summary ............................................................................................................................................................ 4
Purpose and Scope .............................................................................................................................................................. 4
Audience .................................................................................................................................................................................. 4
Data Collection and Analysis ........................................................................................................................................... 4
Findings ................................................................................................................................................................................... 5
Recommendations ............................................................................................................................................................... 7
II. Introduction .......................................................................................................................................................................... 8
Purpose of the Evaluation ................................................................................................................................................ 8
Limitations of the evaluation .......................................................................................................................................... 9
Audience for the evaluation report ............................................................................................................................... 9
III. Focus of the Evaluation .................................................................................................................................................. 9
Description of the evaluation object ............................................................................................................................. 9
Evaluative questions used to focus the study ........................................................................................................... 10
Information needed to complete the evaluation .................................................................................................... 10
IV. Brief Overview of the Evaluation Plan and Procedures .................................................................................. 11
Sources of information ..................................................................................................................................................... 11
Methods of data collection ............................................................................................................................................. 11
Purposive sampling rationale ....................................................................................................................................... 12
Analysis Procedures .......................................................................................................................................................... 12
V. Summary of Evaluation Findings ............................................................................................................................... 13
Benefits of Using ADP ....................................................................................................................................................... 13
Summary of Quantitative Results ................................................................................................................................ 13
Summary of Qualitative results .................................................................................................................................... 16
VI. Interpretation of Evaluation Findings .................................................................................................................... 18
Efficiency ............................................................................................................................................................................... 19

Page 3 of 47

Effectiveness ........................................................................................................................................................................ 20
Standards and Security Compliance ........................................................................................................................... 24
Satisfaction .......................................................................................................................................................................... 25
Organizational Fit to business processes ............................................................................................................... 29
VII. Conclusions and recommendations ....................................................................................................................... 31
Criteria and standards used to judge the quality in use of ADP ....................................................................... 31
Efficiency/ productivity ................................................................................................................................................... 32
Effectiveness: ....................................................................................................................................................................... 33
Satisfaction .......................................................................................................................................................................... 34
Judgments ............................................................................................................................................................................. 36
Strengths ............................................................................................................................................................................... 36
Weaknesses .......................................................................................................................................................................... 36
Recommendations ............................................................................................................................................................. 37
System Deficiencies ...................................................................................................................................................... 37
Organizational issues .................................................................................................................................................. 38
VIII. Minority reports or rejoinders ............................................................................................................................... 38
IX. References .......................................................................................................................................................................... 39
X. Appendices .......................................................................................................................................................................... 41
Appendix A- Evaluation Plan ....................................................................................................................................... 41
Appendix B- ADP Help Desk Data .............................................................................................................................. 42
Appendix C- ADP Quality in Use Survey .................................................................................................................. 43
Appendix D- System Usability Scale Results and Explanation ....................................................................... 44
Appendix E- Questionnaire Data: Ratings of aspects of the ADP portal .................................................... 45
Appendix F- HiSoftware Cynthia Says - Web Accessibility and W3C Reports ................................. 46
Appendix G- Path for logging time starting from the Welcome Screen...................................................... 47

Page 4 of 47

I. Executive Summary
Purpose and Scope
ADP has been the system of record for USG's payroll, benefits, and time tracking for all employees
since August, 2009. Its interface is utilized by virtually all of Georgia Southern's staff and faculty.
The Georgia Southern Human Resources web site summarizes the program model:

"The University System of Georgia has contracted with ADP to implement an Enterprise Human
Resources System and a Time & Labor System to provide all its employees greater access to
managing their personal information, reviewing select benefits, record time entries to log time
worked and leave time taken and run the payroll processes." --Georgia Southern University, ADP
FAQ.

In the context of the program model described above, the scope and purpose of this evaluation
focused on judging the quality in use of the ADP user interface as perceived by employees at
Georgia Southern University. Evaluation questions emerged from the quality in use criteria of
efficiency, effectiveness, and satisfaction.

Based on the results of this evaluation, ADP's user interface does not currently facilitate Georgia
Southern employees' ability to successfully utilize the necessary functionality for completing
required tasks in a timely manner. Employees do not perceive the interface as efficient or effective,
and employees are not satisfied with the quality in use of the system.
Audience
This evaluation took place concurrently with the USGs 90 day evaluation of ADP and seeks to
inform the USG level evaluation committee and key stakeholders at Georgia Southern on the
benefits ADP and specific deficiencies in the ADP user interface that could be addressed during the
USG level fit/gap analysis in order to improve usability and employee acceptance of the system.
Data Collection and Analysis
Qualitative data were collected by means of purposeful sampling at Georgia Southern University
through interviews with key stakeholders, a focus group from a large department that represented
a typical subgroup, and through an online survey emailed to Information Technology staff who
represent informed experts. Nvivo9 software was employed to identify common themes and key
words in the transcripts of the focus group, interviews, and the open comments volunteered in the
online survey.

The online survey also contained quantitative components consisting of a rating scale on various
aspects of the system, as well as a standard System Usability Scale (SUS) based on a Likert scale that
has been in use internationally for over a decade. The resulting SUS score is a single number
ranging from 0 to 100, which is an indicator of overall usability of the system.

Eighteen months of help desk tickets for ADP issues opened by Georgia Southern users were
reviewed and weighed against the qualitative data. Commonly used screens were submitted to the
W3C Markup validation services and ADA Section 508 compliance tests were performed to check
for accessibility for people with disabilities.


Page 5 of 47

Findings
The findings of the evaluation are categorized below based on the three evaluation questions that
emerged from the quality in use criteria of efficiency, effectiveness, and satisfaction.

Efficiency: Is ADP's user interface logical and intuitive, allowing all Georgia Southern
employees to navigate the interface and efficiently complete required tasks? No.

1. Navigation: Navigation is not logical or self-evident. The NVivo analysis yielded 22
references to navigation issues, second only to functionality at 32 hits. Survey respondents
rated navigation the lowest of all aspects of the system, with an overall rating of 2.19 on a
scale of 1 to 5. The average rating by managers was 1.74 indicating that additional task
requirements in the system correlate with lower ratings.

2. Help tools and documentation: External documentation is detailed, extensive, and for the
most part user-friendly. Not all documents are up-to-date and screen shots of menus and
layout do not consistently match the current user interface.

3. Error Handling: Employees most often struggle with the leave request and approval
process and report generation. While the system often generates errors, prompts are
cryptic and unhelpful. Survey respondents rated the leave request and approval process at
2.29 overall on a scale of 1 to 5. Biweekly and monthly employees rated the leave request
and approval process the lowest, below all other aspects of the system.

Effectiveness: Does ADPs user interface respond and function as expected in predictable
and consistent ways, allowing all Georgia Southern employees to successfully complete
time-critical tasks? No.

1. Reliability
Common system functions are unreliable and inconsistent, often impeding
successful completion of time-critical tasks. The NVivo analysis resulted in 32
references to functional issues, which was the greatest of all classifications.
Functionality problems exist in the leave request process, including inaccurate
recording of leave time charged.

An escalating trend in help desk tickets for account lockouts over 18 months
combined with employee comments about the process revealed defects in the
password reset functionality, where the system asks security questions that the user
did not answer during setup.

Other significant issues identified through comments include session persistence
anomalies, system hangs in response to mouse clicks, inconsistent notification of
actions, a lack of adequate functionality for retrieving reports and general
availability of the system.

Significant business process fit issues and other organizational problems exist that
directly impact managers perceptions of the functionality of the system.


Page 6 of 47

2. Accessibility: The most commonly used screens in ADP failed all accessibility and
validation tests for standards-based code. The programming code shows no evidence of
meeting even minimal accessibility standards for people with disabilities. The W3C Markup
Validator identified 33 errors on the ADP Portal login page and 92 errors on the welcome
page. Notably, none of the web pages contain a DOCTYPE (Document Type Declaration) as
the first line of code, triggering quirks mode in browsers, known to cause web page
display problems.

3. Security & browser requirements: The ADP Portal uses SSL (Secure Socket Layer) on
standard port 443 to transmit data over the Internet; however, an insecure version of Java
is required to run on the client computer. ADP only supports Java 1.6.06, which has
documented security vulnerabilities.

Satisfaction: Do Georgia Southerns employees have positive attitudes toward ADP, and
are they satisfied interacting with the ADP user interface for completing required tasks?
No.

1. Employee Attitudes
Employee attitudes toward interacting with the ADP user interface are overwhelmingly
negative. The most frequently used adjectives describe the interface as: cumbersome,
confusing, clunky, slow, awful, complex, antiquated, unreliable, and inconsistent.

The survey respondents and the interview and focus group participants freely
expressed their frustrations and dissatisfaction with the user interface with managers
being most negative.

Significant organizational issues in HR that have not yet been resolved at the
institutional level directly impact managers negative perceptions of the quality in use of
the system.

2. User confidence
Employees and stakeholders did not express confidence in the efficiency or functionality
of the system. Positive comments were usually stated with mitigation, such as better
than paper, but not much.

Interviews with key stakeholders revealed a lack of confidence in system notifications of
actions, the accuracy of leave time accruals, and some managers mentioned witnessing
the system fail to record time punches for hourly employees.

Managers also criticized system availability, routinely receiving system busy errors
during specific times in the payroll cycle; as well as the system being unexpectedly
offline for maintenance.

3. System Usability Scale SUS Score
The overall SUS score for ADP is 38.23, which is a remarkably low score when compared
to the statistics for thousands of SUS surveys comprising over ten years worth of data.
A score of 70 is considered borderline and 80 is considered usable.


Page 7 of 47

Fewer than 6% of systems scored below 50, and there were no scores below 30. The
SUS score based on ratings by Georgia Southern managers was 30.38, which is cause for
serious concern. (It should be noted that there are significant organizational policy and
procedure issues that affect managers opinion of the system.)

Clearly, the 48 employees who volunteered to complete the survey are not satisfied with
the usability of the system, with managers scoring it very poorly.

Recommendations
1. The password retrieval/reset functionality should be modified to present only the security
questions that the user answered during the setup process.

2. The Leave request and approval functionality should be modified to accurately calculate the
hours, based on the users role and shift. For example, it should not allow the accrual of
weekend hours for monthly employees.

3. Navigation should be simplified based on user role in the system, and access to the time card
should be one click away from the welcome screen.

4. The wording of instructions on the welcome screen should be carefully considered, logical and
written in step-by-step style.

5. In order to address platform and browser compatibility, slow loading pages, display problems,
and accessibility, the programming code for the most commonly used screens should be cleaned
up to pass the W3C validation for standards-based code, and Level A Accessibility specifications
should be achieved.

6. The system should be able to run using the updated, current, secure version of Java, or no Java
at all. Requiring the insecure Java 1.6.06 for a payroll system containing Class III data is
unacceptable.

7. Session persistence should be addressed so that users who are actively working in the system
are not logged out automatically. Prompts to remain logged in should display in focus in front of
other windows the user has opened.

8. Cryptic error handling prompts and poorly named buttons and links should be corrected and
renamed. Clear instructions for commonly used functions should be visible on screen.

9. System availability and responsiveness needs improvement. It appears that this issue is being
addressed in the forthcoming upgrade scheduled for April 28, 2012.

10. External user documentation and instructional material should be updated to match current
menus, functions, and page layouts.

11. The concurrent job time transfer tool for hourly employees to transfer hours to a different
department should be made user friendly with department and supervisor names visible as
opposed to the long string of numbers that currently displays.

Page 8 of 47

12. HR policies and procedures should be changed to better align with the ADP model. Procedures
should be put in place to allow HR to collect all information needed for the hiring process before
the new hire begins work. New employees should be in the system and able to add e-time and
clock in on their first day of work.

13. HR policies and procedures for the termination and transfer of employees need to be clearly
stated and enforced. This process should be able to be easily handled by managers within ADP
without requiring input of information that is already in the system.

14. The deadline for managers to approve time should be extended to 9 AM Tuesday following the
end of the pay period.

II. Introduction

The majority of the University System of Georgia (USG) institutions implemented the state-
mandated Automatic Data Processing (ADP) payroll system during fiscal year 2008-09. This
decision was based on the theory that shared services would reduce costs and increase services to
customers by leveraging economies of scale (University System of Georgia Shared Services, p.2).
However, after a system implementation fraught with problems
1
and rushed into production on a
much abbreviated schedule in 2009
2
, USGs ADP users overall perception is unfavorable. Users do
not consider ADPs services best-of-breed technology. There is a perception that ADPs
implementation has increased the likelihood of an institutional audit. Some consider the current
ADP and SSC business model less efficient than the previously decentralized model
3
. Finally, there
is a perception that ADPs quality assurance and hosting center contribute to user frustrations
around system performance and availability
4

.
Purpose of the Evaluation

Preliminary discussions with major stakeholders at Georgia Southern indicated ADP's interface is
an aspect of the program in need of further evaluation. These stakeholders also agreed that the
results from evaluating this component of the ADP system would provide important insight to
ADP's usability, as related to stakeholder requirements. After considering stakeholder input, access
to the ADP software, and the time allotted for this evaluation, the decision was made to focus on the
quality in use of ADPs user interface as defined by the categories of efficiency or productivity,
effectiveness, and user satisfaction and through the analysis of perceptions and experiences of
various end users during normal operations and support of the product at Georgia Southern. The
decision to evaluate only the quality in use of ADPs user interface was made primarily due to time
and budget constraints.

1
State Accounting Office and Georgia Department of Audits and Accounts, pp. 94-96
2
Georgia Southern University Faculty Senate Meeting Minutes, pp. 4, 5
3
University System of Georgia Shared Services Governing Council meeting presentation, slide 5
4
University System of Georgia Shared Services Governing Council meeting presentation, slide 55

Page 9 of 47

Limitations of the evaluation

Several methods of purposive sampling were used while conducting this evaluation, including:
reviewing documents, interface inspection, interviews with key stakeholders, a focus group
discussion, and a survey. Quantitative data for the entire campus population was reviewed in the
form of Help Desk tickets. In addition to the decision to limit the scope of this evaluation to quality
in use, the following limitations were also necessary to complete the evaluation on schedule.

1. Although ADP is used by several of USGs institutions, this evaluation focuses exclusively on
the opinions and attitudes of ADP users at Georgia Southern.
2. Methods employed within this evaluation were limited to inspection and inquiry-based
evaluation methods, rather than methods such as automated testing and analytical
modeling.
3. Due to time and budget constraints, automated usability testing could not be performed.
4. Lack of access to the backend ADP systems, infrastructure, error logs, and equipment
needed to capture performance metrics for software testing during use precluded the
capture of objective measures of system functionality and responsiveness.
5. The online survey instrument sent to Information Technology staff was voluntary. It is
possible that only those with strong feelings about the system chose to respond.

Audience for the evaluation report

The findings from this evaluation, including specific usability issues that may be targets for
modification in short term contract negotiations, will be shared with key stakeholders, including a
member of a USG-level ADP evaluation committee. This committee was formed out of rising
concerns about the ADP system and is charged with assessing ADP's current business model and
making recommendations to the Chancellor. Recommendations by this committee will be
considered when re-negotiating USG's short and long-term contracts with ADP.

III. Focus of the Evaluation
Description of the evaluation object
"The University System of Georgia has contracted with ADP to implement an Enterprise Human
Resources System and a Time & Labor System to provide all its employees greater access to
managing their personal information, reviewing select benefits, record time entries to log time
worked and leave time taken and run the payroll processes." --Georgia Southern University, ADP
FAQ.

On its website, ADP states, ADP is committed to WORLD CLASS SERVICE
5
. As a provider of
business outsourcing solutions to 570,000+ clients and a worldwide presence
6

5

this claim appears
to be legitimate. However, after winning a competitive bid process with USG selecting ADP to
http://www.adp.com/about-us/success-factors/world-class-service.aspx
6
http://www.adp.com/media/press-releases/2012-press-releases/adp-sees-strong-
demand-from-mid-sized-organizations-seeking-a--rapidly-deployed-payroll-solution.aspx

Page 10 of 47

provide a consolidated payroll system as part of the initial launch of its Shared Services Initiative, it
became evident that the payroll system chosen did not meet all of the unique payroll needs
associated with higher education (Hinton, p.95). For instance, the ADP system does not allow an
employee to concurrently be assigned to more than one payroll group at a time. This is especially
problematic if an employee is salaried for one role and hourly in the other role, because salaried
employees are paid monthly and hourly employees are paid bi-weekly. This is a fairly common
occurrence for graduate assistants who may work in another setting, such as in Food Services
(Wainford, p. 4, 5).
Feedback from several of USG's institutions in an October 2011 audit survey indicates there are
user concerns about ADPs user interface. Two major concerns are users' perception that this
product doesn't offer enough functionality and navigating the interface is quite time consuming
7

.
This was confirmed by initial conversations with key stakeholders, including upper-level IT
management at Georgia Southern. Additionally, there is significant concern around the lack of
sophistication of the user interface and its ability to handle daily processes. For these reasons,
along with the limitations described, the decision was made to analyze the quality in use of ADPs
user interface.
The big question that this evaluation answers is: "Does ADP's user interface facilitate Georgia
required tasks in a timely manner?"

Evaluative questions used to focus the study
The following questions were used to develop the categories for quality in use and to focus the
scope of the evaluation.

1. Is ADP's user interface logical and intuitive, allowing all Georgia Southern employees to
navigate the interface and efficiently complete required tasks?
2. Does ADPs user interface respond and function as expected in predictable and consistent
ways, allowing all Georgia Southern employees to successfully complete time-critical tasks?
3. Do Georgia Southerns employees have positive attitudes toward ADP, and are they satisfied
interacting with the ADP user interface for completing required tasks?

Information needed to complete the evaluation
After determining the focus for the evaluation, the following information concerning Georgia
Southern ADP users perceptions was needed in order to accurately assess the quality in use of the
ADP user interface.

1. User perceptions about the benefits of using ADP.
2. User perceptions of the interfaces functionality.
3. User perceptions concerning the adequacy of training.
4. User perceptions of issues with using the interface.
5. Actual issues, as defined by help desk tickets.
6. The degree to which the ADP user interface meets W3C and ADA software guidelines.
7. Overall user perception of the ADP user interface.

7
University System of Georgia Shared Services Governing Council audit background meeting
presentation, slides 15-19


Page 11 of 47

IV. Brief Overview of the Evaluation Plan and Procedures
Sources of information

This evaluation is descriptive, cross-sectional with elements of a case study. Sources for qualitative
information include key stakeholders and employees at Georgia Southern, as well as Information
Technology Services staff who use the system and provide support for system users.

Existing documents and reports were studied including multiple publications and websites
addressing quality of use, usability and software standards. Shared Services Operating Board
meeting minutes, reports, published instructional material, and help desk data were reviewed.

A sampling of critical ADP web pages, such as the login page, portal landing page, and e-time were
submitted to the W3C validation service to determine whether these interfaces follow
specifications for good coding practices for web pages, and HiSoftwares Cynthia Says Section 508
Accessibility tool was used to determine whether the most frequently used screens are in
compliance with ADA Section 508 Accessibility Standards for those with disabilities.

Methods of data collection

Eight (8) semi-structured interviews with key stakeholders such as VPs, directors, business leaders,
and lead IT support staff were conducted on site by a member of the evaluation team. The purpose
of the interview and the evaluation was explained; and individuals were asked open-ended
questions and encouraged to speak freely. The interviews were guided by a series of questions
about the benefits of ADP and the actual functionality of the system in terms of completing critical
tasks. Individuals were also asked to describe their impressions of the usability and functionality of
the ADP portal based on their own experiences using the system, and about their experiences
during the initial implementation phase. For each interview, the leading questions were asked in
the sequence described above. Continuous communication with key stakeholders was maintained
throughout the evaluation, including informal follow-up meetings and personal communications
through telephone conversations and the exchange of email messages.

A large department that employs 450 to 500 personnel was chosen as a typical subgroup for a small
focus group session, which included three department business managers and the IT director for
the department. The same questions used to guide the interviews were used to focus the discussion.

Information Technology staff members, who are considered informed experts, received an online
questionnaire to assess employee satisfaction with various aspects of the system such as logging in,
editing the time card, requesting and/or approving time off, as well as navigation and
responsiveness of the system. The survey was voluntary and included an unlimited open comment
area, and a short answer question asking respondents to describe ADP in three words or less.
Neither the open comment nor the short answer required a response in order to submit the survey.
All other questions were designed to yield quantitative results and responses were required to
submit.


Page 12 of 47

The online survey also included a quantitative component in the form of a standard System
Usability Scale (SUS), which has been in use internationally since 1996, and yields an overall
usability rating on a scale of 0 to 100. To ensure validity of the results, user logins were required to
guarantee that only one response per employee was submitted. For confidentiality, respondents
usernames were removed from the database prior to viewing and analysis of the data by the
evaluation team. See Appendix D for full data, computation, and explanation of the SUS score. A
copy of the survey is also provided in Appendix C.

Purposive sampling rationale

Due to time and budget constraints, purposive sampling was used to obtain data. The rationale for
purposive sampling selection is listed below.

In-depth interviews with key stakeholders (IT VPs, directors, and lead IT support staff)
high technical ability (use system under best of conditions)
extensive previous experience with similar user interfaces
ability to distinguish between system vs. user issues
high level view of system benefits
in-depth knowledge of implementation and deployment
familiarity with organizational culture, policies and procedures
ability to provide existing data, assist with analysis and interpretation, and use results

Typical subgroup (Department Focus Group)
Large department of 450 to 500 employees, managing all types of employees in ADP
ADP manager/approvers
monthly full time
bi-weekly full time
hourly punch time
temporary
High employee turnover, likely to see benefits, problems and anomalies during use of the
system
In-depth analysis likely to differentiate usability issues from organizational issues

Online survey of informed experts (ITS Staff)
high technical ability (use system under best of conditions)
extensive previous experience with similar user interfaces
ability to distinguish between system vs. user issues
various roles in the system from core user to punch time
in-depth knowledge of support and deployment procedures

Analysis Procedures

Triangulation of the same construct using multiple sources of data was employed to increase
validity of the results of the evaluation. The qualitative analysis software, NVivo9, was used to
classify and code all qualitative data by importing transcripts from the focus group, interviews and
open comments submitted via the online survey into the software. The NVivo9 classification nodes
that emerged from the data, and that were used to code common themes across all qualitative data
included appearance, benefits of ADP, compatibility, defects/anomalies, fit for business process,

Page 13 of 47

functionality, implementation, java, navigation, organizational issues, overall dissatisfaction,
passwords, responsiveness, and training.
Data were then consolidated and analyzed by category and weighed against quantitative data to
isolate specific areas of concern regarding the ADP user interface and to differentiate any usability
issues from organizational issues that affect usability but are not system related. See Appendix A
for the complete evaluation plan.

V. Summary of Evaluation Findings

Benefits of Using ADP

Many key stakeholders made positive comments about the benefits of ADP, most notably that
people are getting paid, and ADP is an improvement over the paper-based process that was in place
prior to the implementation. A number of managers stated ADP is more accurate and takes less
time to keep track of employee hours and leave time than the previous payroll system. Managers
also commented that it is easier to verify employee clock-in and clock-out, especially for student
workers, eliminating the need for chasing students down with paper time sheets. The voluntary
survey revealed employee appreciation for the ability to access pay statements online and receive
W2s electronically rather than waiting for hard copies in the mail.

Summary of Quantitative Results

Quantitative data analyzed during the evaluation includes ratings of satisfaction levels on a scale of
1 to 5 for various aspects of the ADP portal, the overall usability score resulting from the System
Usability Scale (SUS) rankings, help desk tickets submitted over an 18 month period, and pass/fail
reports from the W3C Markup Validator and the ADA Section 508 Accessibility checker.

There were 48 responses to the online questionnaire submitted by Information Technology staff
members, providing data on ratings for various aspects of the system as well as the overall SUS
score. Questionnaire respondents were asked to rate their satisfaction with various aspects of the
portal on a scale from 1 to 5, including the login process, use of e-time, navigating the menu system,
requesting and/or approving leave time, and responsiveness of the system. The highest average
rating was 3.50 from bi-weekly employees. The lowest average was 1.75 as rated by
manager/approvers. The averages over all roles for the various system components range from
2.19 to 2.75 on the 1 to 5 scale. A summary of the ratings by ADP user role is shown below. See
Appendix E for all data.


Page 14 of 47

The System Usability Scale

The System Usability Scale (SUS) was developed by John Brooke for Digital Equipment Corporation
and has been in use internationally since 1996. It consists of 10 questions using a Likert rating
scale, and yields a single number grade ranging from 0 to 100 representing a composite measure of
overall usability. Based on the 48 responses from Information Technology staff members, the SUS
score for ADP is 38.23. See Appendix D for all SUS data and details about how the score is calculated
from the Likert scale ratings. Further analysis and interpretation of the SUS score by employee role
in ADP is presented in the Interpretation of Findings section of this evaluation.

ADP Help Desk Data

A cross-tab report for ADP trouble tickets at Georgia Southern was provided by the ADP lead
system support administrator (See Appendix B). This report detailed tickets that were submitted
between July 2010 and February 2012 and included 18 full months of data by ticket type for August
2010 through January 2012 for a total of 5,467 tickets. Data for July 2010 and February 2012 were
incomplete and were therefore excluded from the findings. Account Locks comprise more than half
of the total tickets submitted as indicated in the chart below.


Page 15 of 47

W3C Validation and Accessibility Compliance

A sampling of critical ADP web pages, such as the login page, portal landing page welcome screen,
and e-time were submitted to the W3C Markup Validation Service to determine whether these
interfaces follow specifications for good coding practices for web pages, and HiSoftwares Cynthia
Says Section 508 Accessibility tool was used to determine whether the most frequently used
screens are in compliance with ADA Section 508 Accessibility Standards for those with disabilities.

The W3C Markup Validation Service found 33 validation errors and 11 warnings for the ADP Portal
login screen. The programming code for the ADP welcome screen generated 92 validation errors
and 23 warnings, and the validation report for enterprise e-time included 5 errors and 9 warnings.
Based on this sample of the three main screens, ADP fails to pass the test for valid, standards-based
HTML code.

HiSoftware Cynthia Says - Web Content Accessibility Report found that the login screen is not in
compliance with ADA Section 508 accessibility standards for people with disabilities, failing to
provide the most basic of accommodations such as text equivalents of non-text elements, alternate
descriptions for images, or links to required browser plug-ins.


Page 16 of 47

See Appendix F for the full reports with details of all errors found. A discussion about the
significance of these errors is provided in the Interpretation of Evaluation Findings section of this
report.

Summary of Qualitative results

The qualitative data obtained during the evaluation consist of transcripts from the interviews and
the focus group, and open-ended comments from the online survey. Additionally, the evaluation
team reviewed the external instructional documentation, and inspected internal help files.

Focus Group, Interviews, and Survey Comments

NVivo9 software was used to classify common themes in the various qualitative data. A count of
employee comments by classification is shown below.

Theme Count Theme Count
Appearance 6 Java 8
Benefits of ADP 12 Navigation 22
Compatibility 4 Organizational issues 20
Defects/anomalies 11 Overall dissatisfaction 8
Fit for business process 18 Passwords 10
Functionality 32 Responsiveness 15
Implementation 8 Training 9

The chart below illustrates the greatest areas of concern according to the count of employee
comments by theme classification, and corroborates the Shared Services Governing Council audit
findings on functionality and navigation issues. Over half of the comments that employees chose to
share and stakeholders chose to discuss could be filtered into just 4 of the 14 themes involving
functionality problems, portal navigation, organizational issues or the fit for business processes. It
is interesting to note that some of the lesser concerns indicated by the lower counts emerge as
pivotal when juxtaposed with the quantitative findings of the evaluation. These key points will be
illuminated in the Interpretation of Evaluation Findings section of this report.


Page 17 of 47

Instructional material and help files

A review of the external documentation and instructional material provided on the Georgia
Southern Human Resources website revealed an extensive amount of user friendly content.
Although the material is thorough in terms of providing screen captures to illustrate various ADP
procedures for every user role in the system, not all of the graphics match the current menu options
or web page layouts, and some of the instructions need updating. The Shared Services Center (SSC)
is currently advertising for a position responsible for developing and maintaining technical
documents and end-user business procedures for SSC personnel and USG institution
administrators, so it is evident that the issue of up-to-date documentation has been identified and is
in the process of being addressed. Therefore, the evaluation team saw no need to further inspect
external documentation.

On-screen instructions for navigating to various system functions are available on the ADP Portal
welcome page. As demonstrated by the screen capture below, the instructions provided do not
match the menu options available for the particular user logged in.


Page 18 of 47

Based on an interview with Georgia Southerns lead ADP support administrator, all USG institutions
are greeted with the same welcome screen, which is updated by ADP, Inc. None of the individual
institutions have the access rights to update the content. Additionally, access to menu options is
based on the logged-in users role in the system. While the available menu options are variable by
role, the instructions displayed above are static, regardless of user role.

VI. Interpretation of Evaluation Findings

It should be noted that on March 29, 2012, preliminary reports on both the quantitative and
qualitative survey data were shared with the key stakeholders and a member of the Shared Services
Governing Board in the event this information could prove useful to the USG level Evaluation
Committee conducting the 90 day evaluation of ADP. Currently, a fit/gap analysis has begun and
some of the findings presented in this report are being addressed at the USG level, resulting in
incremental modifications of the ADP user interface to simplify navigation. In a personal
communication on April 18, 2012, Vice President of Information Technology, Steve Burrell, put it
this way: We continue to work with ADP to enhance various aspects of their product(s). The
changes you're seeing are a result of discussions intended to yield improvements to the user
experience. He continued, ADP's ability to respond to our request to improve the user experience
will be a key issue in our determination as to whether it stays or goes.

Page 19 of 47

In order to present the interpretation of evaluation results in a logical manner, the information
below is categorized based on the three evaluation questions that emerged from the quality in use
criteria of efficiency, effectiveness, and satisfaction.

Efficiency
Is ADP's user interface logical and intuitive, allowing all Georgia Southern employees to

Based on the quantitative data from the online survey, navigation scored the lowest overall rating
for any aspect of the system. Respondents were asked to rate the ADP Portal navigation and menu
system on a scale of 1 to 5. The rating was 2.19 when averaged across all user roles in the system.
For managers who have additional required tasks to perform, the average rating was just 1.75,
lower than that of any other employee role. This is a preliminary indicator that perceptions of
quality in use inversely correspond with frequency of system navigation. Stated another way, the
more frequently users navigate the system, the less likely their perception of quality in use will be
positive.

Everyone interviewed commented negatively about the menu system and navigation, and the
survey also yielded numerous comments expressing frustrations with the lack of efficiency in
navigating the system as illustrated by a sample of the comments below:

It takes way too many clicks to find what you need.

As a new employee in an IT department, I had to be shown how to navigate ADP. It has
been the most complex HR system I have ever used.

Its not intuitive not straight forward. Always have to figure out something to find
anything. As a manager, I have to go to Employee [instead of Manager] to see the time and
attendance for my direct reports.

I can use it because I am a technical person. How does a regular person manage to get
around in ADP?

It doesnt flow well. Things are not where you would expect them to be. For instance, how
would you know to go to Actions to request time off? There are no instructions.

Why can't I go directly to time card after logging in or at least only one click away?

When stepping through the navigation process from the welcome screen, a member of the
evaluation team counted six (6) clicks to navigate to the monthly employee time, confirming
comments about the amount of time it takes to navigate the system. See Appendix G for screen
prints tracing this path. User comments concerning the logical placement of items on the screen
are also confirmed by the appearance of the one-size-fits-all welcome screen instructions for
navigating to e-time. They are unclear, reference menu items that do not exist for all users, and do
not accurately describe the navigation path.


Page 20 of 47

Effectiveness

Does ADPs user interface respond and function as expected in a predictable and consistent
ways, allowing all Georgia Southern employees to successfully complete time-critical tasks?

The stakeholder and employee feedback regarding functionality, reliability and consistency was
negative overall. Interviews with key stakeholders revealed a lack of confidence in system
notifications of actions, the accuracy of leave time accruals, and some managers mentioned
witnessing the system fail to record time punches for hourly employees. A few comments have
been selected to illustrate the most frequent complaints that point to key issues in the system that
impede effectiveness.

If I request time off and it is approved, and then something comes up and I dont take the
time off (which happens a lot) I cant go back and fix it. If it passes the day, you cannot fix it.
You have to use a paper work-around.

The procedure for requesting time off in the system is not clear. If the employee wants a
week off starting on a Wednesday and the leave spans a weekend, the system charges 8
hours of leave for each weekend day an additional 16 hours. Two separate leave requests
are required, one from Wednesday through Friday, and a second for Monday and Tuesday.
There are no instructions for this procedure in the portal. When this happens, the original
request has to be canceled by the employee and resubmitted. The manager may or may not
get notification of all these activities, so may or may not approve all requests and
cancellations as a result.

Not all functions behave in a consistent manner. When an employee requests time off, the
system is supposed to send the manager an email notification, but that doesnt always
happen. So if the manager has not yet approved the leave request and the employee notices
that the vacation hours are not showing up on the e-time timecard and manually enters
them, the system charges the vacation the hours twice when the employee approves their
time (as required) at the end of the month. It will take 16 hours out of an 8 hour work day.
The system should not allow this to happen, or it should at least prompt the user with a
warning. Once the month rolls over to the next pay period and Payroll approves the time,
there is no way to fix it and the employee effectively loses an additional 8 hours of leave
time.

The vacation requests dont span the weekend correctly either. It does have some good
accounting reports if you can figure out how to run them. It only gives you the option of
previous, current, or next pay periods. If you want anything else, you have to input a date
range. There is no list of ranges provided, so you have to go look at a calendar to find the
dates when the pay periods starts and ends.

I requested time off over a weekend and it didnt figure the right number of hours. It took
off for the weekend. My manager had to reject the request and I had to resubmit two
requests one for the time before the weekend and one for the time afterwards.


Page 21 of 47

The leave request and approval process received the lowest satisfaction ratings from monthly and
biweekly employees weighed against all other aspects of the system based on data collected in the
online survey. The average rating by monthly employees was 2.24 on a 1 to 5 scale, and biweekly
employees rated it at 2.50. Managers rate the leave approval system at 2.30, but managers rated all
aspects of the system very low.

There were a few functional anomalies mentioned that are worth quoting:

when you log out, it asks you to log in again - must give your username and password again
to log off.

If you select an employee that you have hired and they have worked at another institution
in the USG, the system brings up the whole homepage of the other school school colors,
logo, and all.

Yes, we get knocked out when we are actively working in the system. We work in ADP all
day long. It happens every 5 or 10 minutes. I know many systems will log you off if you are
idle and not actively using the system. But we are in there entering data.

Overall availability and reliability of the system came up in the interviews with key stakeholders
and in the focus group. These issues were more of a concern for those who use the system often or
on a continuous basis. Key stakeholders expected 97 to 98% up time, and based on their comments
ADP is not meeting the mark. Unscheduled down times and unexpected maintenance are
problematic. One of the key issues identified by managers was unacceptable system performance
especially around critical payroll processing times.

On Fridays before payroll runs, the system bogs down. The connection times out and we
have log off and try again.

Sometimes when we login it just whirls and whirls when we are trying to work in the
current pay period.

Very difficult to use due to wait times

too long to load time card info

Systems seem to be unavailable frequently and often slow

Password reset functionality
When evaluating the trends for the help desk tickets over 18 months, the overall count of tickets is
trending downward, while the Account Lockout issues have increased quite a bit over time, as
illustrated in the chart shown below.


Page 22 of 47

This anomaly prompted the question: Why are Account Lockout problems increasing? Using the 5
Whys technique to explore cause and effect, the evaluation team asked the following questions:

Q. Why is the user locked out of ADP?
A. The user could not remember his or her password.

Q. Why couldnt the user remember the password?
A. The user could not retrieve or reset the forgotten password.

Q. Why couldnt the user retrieve or reset the password?
A. The user could not access the reset function.

Q. Why couldnt the user access the reset function?
A. The user could not answer the security question required to access the reset function.

Q. Why couldnt the user answer the security question?
A. Either the user forgot the answer to the security question or the system did not present the security
question that was set up when creating the password.

Because password security questions are typically standardized to include answers that are factual
and do not change over time, such as the city of birth or the name of the first pet, it is improbable
that all these account lockouts are caused by forgotten answers. For instance, in January of 2012,
there were 258 users who submitted tickets for account lockouts. It is unlikely that 258 individuals
forgot answers to standard questions like city of birth, a relatives name, or name of first pet.


Page 23 of 47

The evaluation team then explored the transcripts of qualitative data using NVivo9s keyword
search functionality to retrieve 8 mentions of the keyword password. The transcripts from the
focus group and an interview with one of the key stakeholders contained a similar description of
problems with the password reset process. After finding several instances of password reset
issues within the transcripts, a member of the evaluation team tested the functionality and
experienced the same strange system behavior when attempting to reset the password. The
following comments further describe the issues encountered.

When an employee sets up their ADP account for the first time and creates a password, the
system asks the user to choose from a list of security questions to answer. If you forget your
password, it is not supposed to ask you security questions that you did not answer, but it
does. It does this all the time. We had a student who forgot her password and the system
asked for her fathers name. The student doesnt know who her father is and said she would
never have answered that security question because she doesnt know his name! The
system asks security questions that the user did not answer when they first set up their
passwords and selected security questions.

We had another student who tried to reset her password, and the system changed her
username during the forgot password process.

One of the key stakeholders interviewed mentioned the same security question problem when
trying to change his password. He said that he wrote it off to being forgetful, thinking he simply
couldnt remember what questions he had chosen to answer during setup. He acknowledged that
the security questions presented did not appear to be questions he had answered during setup.

An evaluation team member who is employed at Georgia Southern confirmed the same system
behavior when attempting to change the password: I received questions from the system that I
would never have answered because I dont know the answers. Those questions simply can never
apply to me. During the password change process, I persisted and eventually after 3 or 4 questions
I received a security question that I had answered; and by inputting the appropriate response I was
finally able to change the password.

From the triangulation of transcripts, help tickets, and the evaluation teams inspection, it appears
that the system is not exclusively retrieving security questions answered by the user when the
password reset process is initiated. Instead, every security question is presented, regardless of
whether the question was answered at setup, until the user answers one correctly, at which point,
the user is allowed to change the password. This would quite naturally be confusing and
disconcerting to the user who might just abandon attempts to answer a security question if the
questions presented are unfamiliar and have no obviously correct answer.

The system should present only the questions that the user chose to answer during the setup
process, not just randomly select from every available security question whether they were
answered or not. This is clearly an issue with the ADP software that needs to be corrected. Unless
there is a software defect, any web-based or software system using security questions as a means
for password resets only presents the security questions answered by the user at setup. Otherwise,
the security question process is not fulfilling its purpose to allow users to reset their passwords via
a system process versus engaging technical support. Since many applications do lock out a user if
an incorrect password or security question response is entered more than a set number of times,


Page 24 of 47

most people would discontinue attempts to reset their password and submit a trouble ticket after
receiving two or three unrecognizable security questions. This has likely contributed to the rising
trend in the volume of Account Lockout trouble tickets.

The system requires users to change their passwords periodically and prompts users to do so by
presenting the Change Password screen. Users must know their current password to make the
change. Even if employees simply forget their passwords or expect their password to be
synchronized with their Georgia Southern network password and trigger an account lockout by
typing the wrong password, they should be able to easily perform the password retrieval or reset
by answering the appropriate security questions to regain access to the system. As the ADP
password reset process currently performs, it does not complete this function as expected.

The evaluation team suggests that this odd system response to password reset requests is actually a
software defect and is directly contributing to the escalating trend in help desk tickets submitted
for account lockouts. Therefore, this issue should be submitted to ADP, Inc. via its established issue
resolution process for further investigation.

Standards and Security Compliance

W3C Standards & Quirks

ADP failed the W3C validation check and the Accessibility compliance check. For example, there
were 33 validation errors on the ADP login home page, and 92 errors on the portal welcome page.
None of the ADP web pages tested using the W3C Validator contained a DOCTYPE (Document Type
Declaration), which is required as the first line of code in standards-based HTML documents. A web
page that does not include a DOCTYPE at all will render in quirks mode. This can lead to
compatibility issues, because, In the latest versions of major browsers, standards mode is turned
on by the presence of a DOCTYPE declaration. Lack of a DOCTYPE can lead to different rendering
from browser to browser (W3C). When no DOCTYPE declaration is present, as is the case with the
ADP portal, web browsers do not know which of the many versions of HTML or XHTML to use to
render the page, which in turn triggers quirks mode and potential web page display problems. See
Appendix F for the full reports detailing all of the errors and accessibility compliance problems.

Accessibility

The ADP portal web pages are not access-compliant for users with disabilities. They are out of
compliance with even the most basic of the ADA Section 508 Accessibility and World Wide Web
Consortium Web Content Accessibility Guidelines (WCAG) 2.0 Level A standards, without any
evidence of addressing these standards in the programming code.

ADP, Inc. is based in Canada, and the equivalent laws regarding Accessibility fall under the
Accessibility for Ontarians with Disabilities Act, 2005 (AODA) Section 14. ADP claims to be
committed to meeting obligations under the AODA: ADP Canadas mission is to provide accessible
services to people with disabilities in Ontario and ensuring that people with disabilities have the
same opportunity to access our goods and services and allowing them to benefit from the same
services, in the same place and in a similar way as other customers. There are no statements to be
found on the ADP website specific to software development and Accessibility.

Page 25 of 47

Security

The ADP portal is using SSL (Secure Socket Layer) for all pages within the application, providing for
secure transmission of data over the Internet via the standard secure port 443.

However, numerous comments from the interviews and the online survey pointed out that ADP is
using an insecure version of Java. The Director of Technical Services explained the Java issue this
way:

ADP should update the software more often. It is using an old, insecure version of Java. We
had to sandbox it in a virtual application, so that the Java would run and not conflict with
other applications that use the current version of Java. We shouldnt have to do that. Lots
of software companies manage to stay current and be ready for the next Operating System.
If we hadnt been running SCCM for workstation management to push out the virtualized
app across campus, it would have been a lot worse. How can you have a payroll system that
uses a vulnerable version of Java?

Understandably, IT professionals who responded to the survey expressed a lack of confidence in
system security, best illustrated by the following two comments:

I still wonder why the biggest payroll company in the nation doesn't rely upon an updated
version of java.

ADP's continued reliance on outdated Java SDKs not only puts the university system at risk
for security breaches based on Java 1.6.06. It also points to a company that doesn't wish to
keep their software current and secure, two points that I think should be important to our
payroll management system provider.

Satisfaction

Do Georgia Southerns employees have positive attitudes toward ADP, and are they satisfied
interacting with the ADP user interface for completing required tasks?

Aside from the accuracy and convenience of time tracking and expressions that ADP is better than
paper emerging from the interviews with high-level key stakeholders, the vast majority of
employees expressed negative attitudes toward ADP.

During the interviews, four high level Information Technology stakeholders referred to the difficult
implementation and ridiculously short nine-month project schedule mandated by the USG. It was
apparent that the implementation and its unrealistic timeline were particularly painful for those
charged with bringing the system online in August of 2009. Comments included significant
concerns about university policies, processes, and procedures that had to be changed on the fly to
fit the ADP business model, the exorbitant amount of time and effort involved, as well as anxieties
regarding the impact that the change had and continues to have on employees.


Page 26 of 47

These high level stakeholders suspect that the stress of the initial implementation is at least
partially responsible for the persistent negative attitudes that exist today among employees.
However, these suspicions are not borne out by the employee comments received in the survey.

Of all the employee comments collected by the survey, only one cited the implementation stating,
My overall impression of ADP is that it was a broken software package that was never properly
implemented. All of the other voluntary comments were overwhelmingly focused on current
perceptions about the quality in use of the ADP product as it stands today.

While it appears that employees in general have recovered from the stressful implementation, it is
pertinent to note that the implementation schedule did not allow time to adequately train
employees or core users on the use of the system or to properly fit HR policies and procedures to
the system requirements. As the following comments corroborate, many of these organizational
issues remain a source of frustration, especially for managers and undoubtedly influenced their
opinions about the quality in use of ADP.

Another issue was lack of training. We did not have enough time to properly train people
on how to use eTime, allow them to test and asked questions and then address those
questions. It was just kind of - BAM, here it is.

Training on e-time is overwhelming.

We got a two hour training session the week before the system went live so we could
support our users. It couldnt have been at a worse time of year right before school starts.

I can call the same person with a question and get different answers to the same question
on different days.

We keep trying to get this fixed, and HR tells us to fill out the hiring proposal one way, then
a week later they tell us to do it a different way. We are just trying to do it whatever way
they want, but they keep changing.

Not all the issues are the fault of ADP. Some issues were a very short implementation. This
did not allow enough time for the core users to get comfortable and competent with the
system before the campus had to use it.

One of the most common complaints from managers is that new employees are not in the system in
time, and they have to manually record time for the first two weeks and then submit a pay
adjustment form to have the hours added to the next paycheck. As seen in the user comments
below, there are a number of reasons that might account for this problem, and all of them stem
from organizational policy and procedural issues. There is a good bit of contention surrounding the
issue.

When people don't get paid, more often than not, it is due to people who missed deadlines.
If HR doesn't have something, they cannot key in that person. Yet the department blames
HR or ADP, when in fact it was their own fault.

But onboarding for students and temps is only held for a few hours two days out of the
week. We are doing hiring proposals daily. HR sends back the hiring slips to us but they

Page 27 of 47

dont circle any missing information on the form, like if the SS card was missing.

Theres a number of issues that cause that to happen, not terminated, a missing SS card, or
not in ADP in time. For every new employee we have to manually enter time for the first
two weeks because they are not in the system.

Never, in two years time, have any of my new hires been added to the system until two
weeks after they start work, and often just a day before the deadline to approve their time.
This is the case regardless of when the paperwork is completed and all the required
information submitted to HR. Ive done it up to two months ahead of time, and HR still cant
get them in the system on time.

Help desk data reveals 281 tickets for missing employee ID, and 291 for missing employees, and
1114 e-time issues. According to the ADP lead administrator at Georgia Southern, users do not
always know which trouble ticket option to choose when employees cannot clock in for whatever
reason: If a person does not have an employee ID typically it means that they have not been
processed by HR. The departments typically just email adp@georgiasouthern.edu directly when
that happens. For the Unable to use eTime option, 99% of the tickets are that the employee did not
add eTime when they registered.

Regardless of the reasons for new employees inability to clock time, there is much confusion,
frustration, and finger pointing from all sides of the issue, but to date, nothing has been done to
correct the issue. These problems stem from institutional policy issues that need to be addressed,
particularly if improving manager acceptance of ADP is a long-term goal.

Most often used adjectives

Perhaps the most revealing reflection of employee attitudes was prompted by the survey question
asking employees to describe ADP in three words or less. The word cloud below presents a picture
of dissatisfaction, showing the more frequently used adjectives and themes in progressively larger
type. However, it should be cautioned that the online survey instrument sent to Information
Technology staff was voluntary. It is possible that only those with strong feelings about the system
chose to respond.


Page 28 of 47

Overall Usability and SUS scores

A brief explanation of the System Usability Scale (SUS) is in order. In a 2008 paper titled An
Empirical Evaluation of the System Usability Scale, Bangor, Kortum and Miller present the results of
analyses of over 10 years worth of SUS data, including 2,324 surveys from 206 studies. They found
that the overall mean SUS score was 70.14. On a per study basis, the mean was 69.69 ranging from
30 to 93.93.

Significantly, they found that fewer than 6% fall below 50 and none fall below 30. They concluded
that systems with a score above 70 are passable, better systems score near 80, and superior
products score better than 90. The authors divided marginal scores into high or low marginal,
stating This break occurs roughly at the beginning of the second quartile range. Products with
scores less than 50 should be cause for significant concern and are judged to be unacceptable (p.
592).

The overall usability score for ADP resulting from the SUS survey was 38.23, which falls into the
unacceptable range with cause for significant concern. The SUS scores broken down by user roles in
ADP are shown below. While the system usability score of 65 as rated by biweekly staff approaches
the passable mark, monthly employees and managers rated the system significantly lower at 41.60
and 30.38 respectively. Considering that the analysis of ten years worth of data from thousands of

Page 29 of 47

surveys found no SUS score below 30, the mean score of 30.38 for the 20 managers who responded
to the ADP Quality in Use Survey indicates serious deficiencies (See Appendix D).

It is clear that in the context of their roles managers find interacting with the ADP interface entirely
unsatisfactory. This finding corroborates the interpretation of the satisfaction ratings on the
various aspects of the ADP portal. Increased use of the system for completing required tasks
correlates to lower satisfaction levels.

Organizational Fit to business processes

Although the findings discussed below are essentially out of scope for this evaluation, these
environmental variables directly affect user perceptions and acceptance of the ADP system and
specifically, the quality in use of the ADP user interface. Many of the problems users cited can be
attributed, at least in part, to the lack of compatibility or fit between ADPs product and Georgia
Southerns unique organizational issues and existing business processes at the university. The
NVivo analysis of interviews and survey comments found 18 comments about the fit and 20
comments referring to organizational issues at Georgia Southern that will need to be addressed by
policy and procedural changes on campus for ADPs business model to work more effectively. ADP
cannot fix the majority of these problems because they are not system related. To differentiate
them from ADP user interface issues requiring attention from ADP, Inc., the most significant of
these findings are categorized into three themes and summarized by the following employee
comments.

1. Moving paper-based processes into an automated system.
It is worth noting that there is a trade-off when an organization decides to move from a paper-
based system to an automated, off-the-shelf solution, and it is a decision worthy of careful
consideration. While automated processes certainly increase efficiency, they cannot compete with
paper-based systems in terms of flexibility. So, to work effectively within the confines of an off-the-
shelf solution, an organization must undertake the task of standardizing its processes, regardless of
how cumbersome this may be. Otherwise there are bound to be issues, as the following comments
illustrate.

There are over 800 benefits programs with all the USG institutions that require
customizations. Benefits are handled in a different system by ADP and batch updates are
made to the ADP database. That is the main problem with all the customizations.


Page 30 of 47

The previous system was a paper based and had too many processes that did not fit.

We put old processes into a new system. The process fit is not good.

The following comments demonstrate the differing views concerning the alignment of Georgia
Southerns policies to best function in the framework of ADPs system.

Pointing out issues that are not ADP, is lack of change in business processes. We have a
new house and everyone is trying to make the old furniture fit in it. As a campus we lack the
desire to change, what seems like any, process. People are still, 2 years later, trying to make
their old processes work.

The system required the University to change procedures to match those of the system
rather than the system functioning to support the University. The system seems to govern
the University's payroll and time accounting function rather than supporting the needs of
the University.

2. Institutional policies that impede employees ability to effectively manage their work loads.
Another repeated theme concerned very basic payroll functions: adding new employees to the
system and insuring time worked is loaded and approved prior to payroll processing.
Unfortunately, these are not handled efficiently due to institutional policies. Below are comments
describing some of the larger issues encountered.

Getting people in the system is the worst problem. We hire a lot of temporary people and
we always do the paperwork way ahead of time. For instance, if we are going to start a new
employee for spring term in the middle of January, we complete the PAF (Personnel Action
Form) during the first week of December. And they are still not in the system when they
start work. So they have to enter their hours on paper timecards and then we key it all in
two weeks behind. Then we have to fill out a payroll adjustment form for all these new
employees to get the first two weeks of work added to the next pay check. That means they
dont get paid for a month. That is really hard on hourly workers who dont make that much
to begin with. So no, they are not getting paid on time.

I know it seems like HR should be able to add them in that amount of time, but often there
are policies in place that prevent them from entering them any sooner. The president has
made the decision that we cannot ask for the Social Security number until their first day on
the job when they go through onboarding. So even though you completed your paperwork
over a month early, HRs hands are tied. We need to revisit this policy and come up with a
procedure to obtain the necessary information before the employees starts work

...workers dont leave until 1 or 2 AM on Saturday, those hours fall into the next pay period.
Pay periods all start on Saturday. We have to approve time by 9 AM on Monday for people
to get paid. The entire USG has that same deadline. We work every Saturday to approve
everyones time because one hour on Monday morning is not enough to approve time for
450 to 500 employees. The deadlines are not realistic.

3. Private sector versus public sector organizational structures.
While ADP has a proven track record in the private sector, public universities are not set up the
same way as corporations. In general, businesses can better fit a mold, with the same benefits and

Page 31 of 47

rules that everyone follows. The various USG institutions are by nature structurally and culturally
different from the corporate business model as the following comments illustrate.

ADP was not ready to handle the complexities of the Higher Ed. environment.

Some issues we have due to the fact ADP has not worked with a lot of schools. The idea of
having two jobs is a foreign concept to ADP. It is unfortunate that the majority of our issues
stem from students having two jobs.

It is also a problem if the student works in two departments on campus. We train them on
how to transfer the hours over to our department before they clock in and transfer back to
the other department and manager when they clock out, but the process is not clear in ADP
and they often forget to do it. ADP just shows them a list of really long department numbers
to choose from. We are constantly doing budget reallocations. Getting ADPs official time
clocks wont solve this problem.

User perceptions of the overall usability are affected by the organizational fit issues that were not
addressed in the planning stages of the initial implementation, and have yet to be addressed.

The flexibility of the previous paper-based system used for tracking time also allowed for flexibility
in the hiring policies and onboarding procedures. To achieve manager acceptance of ADP, managers
must understand that leeway is no longer possible due to the additional time and effort required for
completing the process of adding new employees to the system and must be trained on its use. In
addition, procedures for termination and transfer need better management as well as integration
into the system.

Many of the quality in use problems that are brought to bear on managers and new employees can
be directly traced back to HR policies and procedures that have not changed to fit the ADP business
model and system requirements, which could partially account for the lower ratings by managers
and their lack of confidence in the system. For the ADP model to work effectively for managers, it is
essential that new hourly employees and student workers are in the system and trained to use it on
their first day of employment so that they can clock their time, thereby avoiding the multitude of
manual work-arounds that are now being used to compensate for the resistance to change.

The decision to address the previously discussed issues in an effort to better fit the ADP business
model rests at the institutional level, and the issue of a better overall long term fit rests with the
USG level evaluation committee.

VII. Conclusions and recommendations

Criteria and standards used to judge the quality in use of ADP

The criteria and standards used to judge the quality in use of ADP are mapped to evaluation
questions to provide the reader with a clear picture of the major findings of the evaluation.
Judgments regarding the quality in use of ADPs user interface fall into the three major categories of
efficiency or productivity, effectiveness, and satisfaction.


Page 32 of 47

The evaluation criteria and standards shown in the following tables are based on the needs and
requirements of stakeholders and end users at Georgia Southern. It is assumed all standards are
applied under normal conditions where no network or user input errors exist. It is worth noting
that a significant proportion of so-called human errors can often be attributed to inconsistencies
and ambiguities in a poorly designed user interface or dialog, or to functionality which is not closely
matched to the users task needs causing them to struggle with badly thought-out functionality
(Bevan, 1999, p.7).

Efficiency/ productivity
The resources expended in relation to the accuracy and completeness with which users achieve
goals [ISO 9241-11:1998] (Ref NIST 7432, p.5)

Question: Is ADP's user interface logical and intuitive, allowing all Georgia Southern employees to
navigate the interface and efficiently complete required tasks? No.
.
Criteria Standards Findings

1. Portal
navigation

1. Navigation is logical and self-
evident. Clear navigation
mechanisms are provided with on-
screen instructions, links, and tabs.
Menus are descriptive, prominently
displayed, and easy to follow.
(Bevan, p.7).

Navigation is not logical or self-evident based
on employee comments from interviews, the
focus group, and survey comments. The
NVivo analysis yielded 22 references to
navigation issues, second only to
functionality at 32 hits. Survey respondents
rated navigation the lowest of all aspects of
the system, with an overall rating of 2.19 on a
scale of 1 to 5. The average rating by
managers was 1.74 indicating that additional
task requirements in the system correlate
with lower ratings.

2. Help tools and
documentation

2. Help tools and external
documentation are detailed, up-to-
date, and written in user-friendly
language. Information is provided
for using the navigation tools and
about application content. ISO 9241-
11s Part 13: User Guidance. (Bevan,
p.7).

External documentation is detailed,
extensive, and for the most part user-
friendly. Not all documents are up-to-date
and screen shots of menus and layout do not
consistently match the current user interface.

3. Error
handling

3. ADPs user interface has intuitive
and comprehensive error-handling
mechanisms that include prompts
and feedback to address common
user errors and reinforce learning.
ISO 9241-11s Part 13: User
Guidance. (Bevan, p.7).

Based on observation and employee
comments, common error handling
mechanisms are not intuitive and provide
little (if any) instruction to reinforce learning.
Employees most often struggle with the leave
request and approval process and report
generation. While the system often generates
errors, prompts are cryptic and unhelpful.
Survey respondents rated the leave request

Page 33 of 47

and approval process at 2.29 overall on a
scale of 1 to 5. Biweekly and monthly
employees rated the leave request and
approval process the lowest, below all other
aspects of the system.

Effectiveness:
The accuracy and completeness with which users achieve specified goals [ISO 9241-11:1998] (Ref
NIST 7432, p.5)

Question: Does ADPs User Interface respond and function as expected in a predictable and
consistent ways, allowing all Georgia Southern employees to successfully complete time-critical
tasks? No.


1. Reliability
of common
system
functions

1. The system is available and
processes user input in consistent and
predictable ways. Functions, such as
mouse-click reliability, application
start-up time, logging in, session-
persistence, updating passwords and
personal information, recording and
approving time, requesting and
approving time off, notification of
actions, and retrieving reports within
the context of the user's role are
intuitive, consistent, and accurate,
ensuring user control of time-sensitive
content. Parts 10 and 17 of ISO 9241,
Dialogue principles and Form dialogue
principles (Bevan, p. 6-7).

Common system functions are unreliable and
inconsistent, often impeding successful
completion of time-critical tasks. The NVivo
analysis resulted in 32 references to
functional issues, which was the greatest of
all classifications. Interviews, the focus group,
and survey comments revealed major
functionality problems in the leave request
process, including inaccurate recording of
leave time charged. An escalating trend in
Help Desk tickets for account lockouts over
18 months combined with employee
comments about the process revealed defects
in the password reset functionality. Other
significant issues identified through
comments include session persistence
anomalies, system hangs in response to
mouse clicks, inconsistent notification of
actions, a lack of adequate functionality for
retrieving reports and general availability of
the system. Significant business process fit
issues and other organizational problems
exist that directly impact manager
perceptions of the functionality of the system.

2.
Accessibility

2. The ADP user interface conforms to
minimum ADA Section 508's
Accessibility Standards and W3C
specifications for valid markup,
including text equivalents for non-text
elements, scripts that allow

The most commonly used screens in ADP
failed all accessibility and validation tests.
The programming code shows no evidence of
any effort to meet minimal accessibility
standards, such as alternate descriptions or
text equivalents for non-text elements, or any

Page 34 of 47

accessibility, providing frame titles,
enabling users to skip repetitive
navigation links, and ensuring plug-ins
and applets meet accessibility
requirements. Screens regularly
utilized by all ADP users properly
display in the browser and are
accessible to those with disabilities.
(http://www.section508.gov/index.cf
m?fuseAction=stdsdoc#Web and U.S.
Department of Health and Human
Services, p. 22).
of the other standards specified in ADA
Section 508. The W3C Markup Validator
identified 33 errors on the ADP Portal login
page and 92 errors on the welcome page.
Notably, none of the web pages contain a
DOCTYPE (Document Type Declaration) as
the first line of code, triggering quirks mode
in browsers, known to cause web page
display problems.

3. Security
and
OS/browser
requirements,
product
maintenance

3. The ADP user interface is secure
using SSL protocol. Ideally, it is also
platform and browser-agnostic. If it is
not platform and browser-agnostic, it
performs with no system conflicts on
ADP supported operating systems,
browsers, plugins, and secure java
versions facilitating product support
and delivery to all Georgia Southern
employees. (NIST 7432, p.26).

The ADP Portal uses SSL (Secure Socket
Layer) on standard port 443 to transmit data
over the Internet; however, an insecure
version of Java is required to run on the client
computer. ADP only supports Java 1.6.06,
which has documented security
vulnerabilities. At this writing, the current
version of ADP requires IE6 or above. A
forthcoming update scheduled for April 28
th

expands compatibility to IE 8 and Firefox
browsers. MAC users report problems
accessing the portal, which will continue to be
problematic. The insecure Java version is an
unacceptable security issue that puts
institutional payroll data at risk of
compromise.

Satisfaction
Freedom from discomfort, and positive attitudes towards the use of the product [ISO 9241-
11:1998] (Ref NIST 7432, p.5)

Question: Do Georgia Southerns employees have positive attitudes toward ADP, and are they
satisfied interacting with the ADP user interface for completing required tasks? No.


1. User attitude
toward ADPs user
interface

1. Within the context of their
roles, users express positive
attitudes toward using ADPs
user interface.

Employee attitudes toward interacting with
the ADP user interface are overwhelmingly
negative. The most frequently used adjectives
describe the interface as: cumbersome,
confusing, clunky, slow, awful, complex,
antiquated, unreliable, and inconsistent.
Nearly everyone who completed the survey
volunteered to comment, although comments
were not required. Note that the survey was

Page 35 of 47

voluntary and it is possible that only those
with strong feelings about the system chose
to respond. The interviews and focus group
participants freely expressed their
frustrations and dissatisfaction with the user
interface with managers being most negative.
Significant organizational issues that have not
yet been resolved at the institutional level
directly impact managers negative
perceptions of the quality in use of the
system.

2. User confidence
interacting with
ADPs User
Interface

2. Users are confident that a
given task can be completed
efficiently and successfully each
time the action is taken, as
defined by ISO DIS 20282-1s
ease of operation of everyday
products. (Bevan, p.4)

Employees and stakeholders did not express
confidence in the efficiency or functionality of
the system. Positive comments were usually
stated with mitigation, such as better than
paper, but not much. Interviews with key
stakeholders revealed a lack of confidence in
system notifications of actions, the accuracy
of leave time accruals, and some managers
mentioned witnessing the system fail to
record time punches for hourly employees.
Managers also criticized system availability,
routinely receiving system busy errors during
specific times in the payroll cycle; as well as
the system being unexpectedly offline for
maintenance.

3. SUS Score on
the overall
usability of the
system as an
indicator of user
satisfaction with
ADPs user
interface

3. While an overall average score
in the 70s is generally
considered acceptable, an overall
average score above 80 on the
System Usability Scale would
indicate a high level of usability
and that employees are satisfied
with the appearance,
functionality and usability of the
system for completing all
required payroll and benefits
tasks. (Bangor, Kortum, & Miller,
p.115).

The overall SUS score for ADP is 38.23, which
is a remarkably low score when compared to
the statistics for thousands of SUS surveys
comprising over ten years worth of data.
Fewer than 6% of systems scored below 50,
and there were no scores below 30. The SUS
score based on ratings by Georgia Southern
managers was 30.38, which is cause for
serious concern. (It should be noted that
there are significant organizational policy and
procedure issues that affect managers
opinion of the system.) Clearly, the 48
employees who volunteered to complete the
survey are not satisfied with the usability of
the system, with managers scoring it very
poorly. Note that because the survey was
voluntary, it is possible that only those with
strong feelings about the system chose to
respond.


Page 36 of 47

Judgments

Based on the results of this evaluation, ADP's user interface does not currently facilitate Georgia
required tasks in a timely manner. Employees do not perceive the interface as efficient or effective,
and employees are not satisfied with the quality in use of the system.

Strengths

While the tone of this report may present a negative picture overall and does point out numerous
specific deficiencies, it could be argued that what was not said during the collection of qualitative
data implies that many of ADPs benefits are taken for granted.

Only organizational policy and procedural issues stand in the way of employees getting paid on
time. As long as new employees are added to the system in a timely manner, the efficiencies of
electronic time card approval have been realized. Likewise, the inefficiencies inherent in the sign-
off process for the paper-based time sheets of the past have long been forgotten.

Online access to pay advices, tax forms, and the ability to sign up for direct deposit are features that
perform consistently. It goes without saying that convenience of the web portal is a huge benefit.

The program model or theory of shared services is solid, and ADP has a credible record of
successfully providing comprehensive benefits and payroll management services in the private
sector.

Weaknesses

Based on the negative evaluation in all three categories of efficiency, effectiveness and satisfaction
that persists more than two years after the go-live date in August of 2009, the question that begs an
answer is why basic system functionality and usability problems have not previously been
identified and resolved by ADP. It appears that ADP does not provide the level of support needed to
finish the job.

For instance, leave report errors and leave request functionality that do not work consistently were
first brought to the table in July of 2009, yet the functionality of the leave request feature remains
unreliable.

The systems inability to handle concurrent jobs continues to be problematic both in terms of
organizational fit to the ADP model, as well as the functionality of the concurrent job time-transfer
system, which is not user-friendly and is difficult to use.

System responsiveness is unpredictable and availability is unreliable.


Page 37 of 47

The lack of standards-based programming code and provisions for accessibility cause the system to
be error prone. The outdated, insecure version of Java required to access the system not only
causes the system to hang, but also places payroll data at risk of compromise.

It appears that numerous logic errors exist in the softwares programming. Interface features and
navigation are poorly designed causing employees to struggle with locating and using the various
tools and functionality available in the system.

Recommendations

During this evaluation, a USG level decision has been made to upgrade the e-time platform. The
Shared Services website has posted this announcement: eTime Upgrade - In the winter of 2012,
the SSC will begin the process of upgrading the eTime application to release 6.2. The upgrade gives
institution employees and practitioners access to more robust functionality and the ability to report
on earnings, leave and vacation through ADP Reporter.

The evaluation team supports the e-time upgrade to release 6.2, provided there is adequate
planning to improve the backend functionality of the system and to address institutional level
policy and procedural issues that negatively affect the fit of the ADP business model and
employee perceptions of the system. Adequate time for training on the new interface and updated
user documentation should also be part of the plan.

The evaluation team recommends that the following items be addressed in the short term in order
to achieve the biggest bang for the buck in terms of quality in use, and as the most straight
forward path to improvement in user perceptions and acceptance of the system.

System Deficiencies

1. The password retrieval/reset functionality should be modified to present only the security
questions that the user answered during the setup process.

2. The Leave request and approval functionality should be modified to accurately calculate the
hours charged based on the users role and shift. For example, it should not allow the
accrual of weekend hours for monthly employees.

3. Navigation should be simplified based on user role in the system, and access to the time
card should be one click away from the welcome screen.

4. The wording of instructions on the welcome screen should be carefully considered, logical
and written in step-by-step style.

5. In order to address platform and browser compatibility, slow loading pages, display
problems, and accessibility, the programming code for the most commonly used screens
should be cleaned up to pass W3C validation for standards-based code, and Level A
Accessibility specifications should be achieved.

6. The system should be able to run using the updated, current, secure version of Java, or no
Java at all. Requiring the insecure Java 1.6.06 for a payroll system containing Class III data is

Page 38 of 47

unacceptable.

7. Session persistence should be addressed so that users who are actively working in the
system are not logged out automatically. Prompts to remain logged in should display in
focus in front of other windows the user has opened.

8. Cryptic error handling prompts and poorly named buttons and links should be corrected
and renamed. Clear instructions for commonly used functions should be visible on screen.

9. System availability and responsiveness need improvement. It appears that this issue is
being addressed in the upcoming upgrade scheduled for April 28, 2012.

10. External user documentation and instructional material should be updated to match current
menus, functions, and page layouts.

11. The concurrent job time transfer tool for hourly employees to transfer hours to a different
department should be made user friendly with department and supervisor names visible as
opposed to the long string of numbers that currently displays.

Organizational issues

1. HR policies and procedures should be changed to better align with the ADP model.
Procedures should be put in place to allow HR to collect all information needed for the
hiring process before the new hire begins work. New employees should be in the system
and able to add e-time and clock in on their first day of work.

2. HR policies and procedures for the termination and transfer of employees need to be clearly
stated and enforced. This process should be easily handled by managers within ADP
without requiring input of information that is already in the system.

3. The deadline for managers to approve time should be extended to 9 AM Tuesday following
the end of the pay period.

VIII. Minority reports or rejoinders

None


Page 39 of 47

IX.References

Accessibility for Ontarians with Disabilities Act, 2005. Retrieved on April 17, 2012 from
http://www.e-
laws.gov.on.ca/html/source/regs/english/2011/elaws_src_regs_r11191_e.htm#BK15

ADP, Inc. http://www.adp.com

Bangor, A., Kortum, P., & Miller, J. (2009). Determining what individual SUS Scores mean: Adding
an adjective rating scale. Journal of Usability Studies, 4(2), 114-123.
http://www.upassoc.org/upa_publications/jus/2009may/JUS_Bangor_May2009.pdf

Bevan, N. International Standards for HCI Reference: International Standards for HCI.
Updated May 2006. Based on chapter in Encyclopedia of Human Computer Interaction. Idea
Group Publishing, 2006. http://nigelbevan.com/papers/International_standards_HCI.pdf

Bevan, N. (1999). Quality in Use: Meeting User Needs for Quality. Journal of System and Software,
1999. http://www.usabilitynet.org/papers/qiuse.pdf

Georgia Southern University, ADP FAQ. http://jobs.georgiasouthern.edu/facultystaff/adp/faq

Georgia Southern University Faculty Senate Meeting Minutes, (September 23, 2009). Problem with
ORP Deductions and Contributions (pp. 4, 5). See statement from President Grube. First
retrieved by Sam Wainford on November 10, 2009 from the Faculty Senate Archive of
Agendas, Minutes, and Librarian's Reports:
https://sharepoint.georgiasouthern.edu/OP/facultysenate/Lists/Archives/Faculty%20Sen
ate%20Archives.aspx
New link retrieved on February 6, 2012 requires Georgia Southern login:
https://inside.georgiasouthern.edu/President/facultysenate/Lists/Archives/Faculty%20Se
nate%20Archives.aspx

Hinton, R. (2011). Single audit report of the state of Georgia for the fiscal year ended June 30, 2010.
http://www.open.ga.gov/reports/sgd-10-swsa.pdf

Ivory, M. Y. (2001). An Empirical Foundation for Automated Web Interface Evaluation. PhD
Dissertation, UC Berkeley Computer Science Division, 2001, Chapters 2 and 5.
http://webtango.berkeley.edu/papers/thesis/thesis.pdf

NIST Reference: NISTIR 7432 Common Industry Specification for Usability Requirements,
Information Access Division, Information Technology Laboratory. National Institute of
Standards and Technology, Technology Administration, U.S. Department of Commerce. June
28, 2007. http://zing.ncsl.nist.gov/iusr/documents/CISU-R-IR7432.pdf

Section 508 Standards Guide 1194.22 Web-based intranet and internet information and
applications. http://www.section508.gov/index.cfm?fuseAction=stdsdoc#Web


Page 40 of 47

State Accounting Office and Georgia Department of Audits and Accounts, (March 7, 2011). State of
Georgia Single Audit Report: Financial Statement Findings and Questioned Costs For Year
Ended June 30, 2010 (pp. 94-96). See Board of Regents Funding Control Number FS-472-10-
01 Accounting Controls (Overall), Inadequate Internal Controls over Payroll Consolidation
and Outsourcing Project. Retrieved February 2, 2012:
http://www.open.ga.gov/reports/sgd-10-swsa.pdf

University System of Georgia Shared Services, (n.d.). Payroll Consolidation: A Shared Services
Approach White Paper, p.2. Retrieved on January 26, 2012:
http://www.atlm.edu/irpa/sharedservices/documents/white_paper.pdf

University System of Georgia Shared Services Governing Council audit meeting presentation,
(October 31, 2011). Audit background-Shared Services.

University System of Georgia Shared Services Governing Council audit meeting presentation,
(October 31, 2011). SSC Governing Council-October 31, 2011.

U.S. Department of Health and Human Services. Research-based Web and Usability Guidelines.
Updated August 15, 2006. http://www.usability.gov/guidelines/guidelines_book.pdf

W3C Markup Validator

W3C, (August 8, 2010). Serving HTML & XHTML: standards vs quirks modes. Retrieved April 17, 2012
from http://www.w3.org/International/articles/serving-xhtml/Overview.en.php#quirks

Wainford, S. (November 15, 2009). Unintended Consequences Resulting from the ADP Conversion at
Georgia Southern. Unpublished report on the September 2, 2009 ADP Core Users Meeting,
Deal Hall Conference Room, Georgia Southern University.

Program Evaluation Final Report
April 23, 2012
Page 41 of 47

X. Appendices

Appendix A- Evaluation Plan
Evaluation Plan Quality in Use of the ADP User Interface
Angela Kennedy, Jesse Scott, Philis (Sam) Wainford
March 26, 2012

Page 1 of 9

The Object of Evaluation: The Automatic Data Processing (ADP) User Interface

"The University System of Georgia has contracted with ADP to implement an Enterprise Human Resources System and a Time
& Labor System to provide all its employees greater access to managing their personal information, reviewing select benefits,
record time entries to log time worked and leave time taken and run the payroll processes." --Georgia Southern University,
ADP FAQ.

The Big Question

In the context of the program model described above, the scope of this evaluation will be limited to the quality in use of the
ADP User Interface at Georgia Southern. The big question to be answered is: "Does ADP's User Interface facilitate Georgia
Southern employees' ability to successfully utilize the necessary functionality for completing required tasks in a timely manner?"

Shared Evaluation Plan Sources and Methodology

In order to consolidate redundant data, a listing of evaluation-specific items that are used as sources and methodologies for all
evaluation questions is included below, followed by question-specific information in table format.

Design
o Descriptive
o Cross-sectional
o Possible Case Study

Information Sources
o Qualitative
Georgia Southern employees
Client and key stakeholders
IT Staff who use and maintain ADPs User Interface
Evaluation team

March 26, 2012

Page 2 of 9

o Existing Documents
Multiple publications and websites addressing quality of use, usability and software standards
Shared Services Operating Board meeting minutes, reports, and Help Desk data

Purposive Sampling Rationale
o Key Stakeholders (IT VPs, directors, and lead IT support staff)
High technical ability (use system under best of conditions)
Extensive previous experience with similar User Interfaces
Ability to distinguish between system vs. user issues
High level view of system benefits
In-depth knowledge of implementation and deployment
Familiarity with organizational culture, policies and procedures
Ability to provide existing data, assist with analysis and interpretation, and use results
o Typical subgroup (Departmental Focus Group)
Large department of 450 to 500 employees that manages all types of employees in ADP
ADP manager/approvers
Monthly full time
Bi-weekly full time
Hourly punch time
Temporary
High employee turnover, likely to see benefits, problems and anomalies during use of the system
In-depth analysis likely to differentiate usability issues from organizational issues
o Survey Informed Experts (ITS Staff)
High technical ability (use system under best of conditions)
Extensive previous experience with similar User Interfaces
Ability to distinguish between system vs. user issues
Various roles in the system from core user to punch time
In-depth knowledge of support and deployment procedures

March 26, 2012

Page 3 of 9

Analysis Procedures and Triangulation Strategy
o Reduction and synthesis to discover major themes in transcripts of focus group and interviews, including IT
support and maintenance functions
o Identify and extract usability issues (if any) as distinguished from organizational issues that affect usability, but
are not system related
o Compare, analyze, and synthesize Help Desk metrics and major themes
o Reduction and synthesis of survey responses to identify any recurring themes through keyword analysis and
pattern matching
o Triangulate the same construct using multiple sources of data to increase validity

Reporting Procedures
o Audience
GSUs Business and IT leadership, including a member of the USG level Evaluation Committee. This USG level
committee is charged with assessing ADP's current business model and making recommendations to the
Chancellor. These recommendations will be considered when re-negotiating USG's short and long-term
contracts with ADP.
o Format
Evaluation Report
PowerPoint presentation for Shared Services decision makers
o Schedule
3/26/12 through 4/16/2012: Analysis and interpretation of data
4/23/2012: Evaluation Report due
4/30/2012: PowerPoint presentation for Georgia Southern VPIT and Shared Services Evaluation
Committee Member
Evaluation Plan Worksheets

The three tables on the following pages are categorized by evaluation question. They detail the information needed, additional
information sources, methods and schedules for collecting data, and sampling, collection and analysis procedures.

March 26, 2012

Page 4 of 9

1. Is ADP's User Interface logical and intuitive, allowing all Georgia Southern employees to
Evaluation Plan Worksheet
Information
Required
Additional
Information
Sources
Method of
Collection
Sampling Collection
Procedures
Schedule Analysis Procedure
Quality of the
portals
navigation
features
Quality of the
portals error
handling
capabilities
Quality of the
ADP User
Interfaces help
tools and
documentation
Existing Help
Desk data.

Existing
instructional
documents on
the use of the
system.

Interviews Purposive Sampling:
Key Stakeholders (IT
VPs, directors, and lead
IT support staff)
Interviews with
key stakeholders
onsite.

Interviews
conducted
2/28/12
through
3/23/12
Triangulate Help Desk metrics,
survey results, and major
themes found in interviews
and focus group

Focus Group Purposive Sampling:
Typical subgroup
(Departmental focus
group)
A Departmental
Focus Group
conducted onsite
2/28/12 Triangulate Help Desk metrics,
and focus group

Identify and extract usability
issues (if any) as distinguished
from organizational issues that
affect usability, but are not
system related

Survey Survey informed
experts (ITS Staff)

An online survey
that includes the
System Usability
Scale
Survey-
3/26/12
through
4/16/12
Reduction and synthesis of
survey responses to identify
any recurring themes through
keyword analysis and pattern
matching
Help Desk
metrics
Quantitative data for
the entire campus
population
Cross-tab report
of existing Help
Desk data over
18 months, by
problem type by
month
Received
2/10/12
and focus group

March 26, 2012

Page 5 of 9

Document
Review &
Interface
inspection
Purposive Sampling of
OS and browser
requirements
Documentation
available on the
ADP & Georgia
Southern HR
websites
3/26/12
through
4/9/12
Ensure that requirements are
met

Interpretation Procedures Content
To what extent is navigation logical and self- evident with clear navigation mechanisms provided, including
prominently displayed menu labels, on-screen instructions, links, and tabs?

To what extent are help tools and external documentation detailed, up-to-date, and written in user-friendly
language? Do these tools provide information for using the navigation tools and about application content?

To what extent does ADPs User Interface have intuitive and comprehensive error-handling mechanisms
that include prompts and feedback to address common user errors and reinforce learning?

While out of scope for the evaluation, are there any organizational policies, issues or benefits that should be
listed in the report as influences on the overall fit to the ADP business model?
Address the question: Is ADP's
User Interface logical and
intuitive, allowing all Georgia
Southern employees to navigate
the interface and efficiently
complete required tasks?

Help answer the big question:
Does ADP's User Interface
facilitate Georgia Southern
employees' ability to
successfully utilize the
necessary functionality for
completing required tasks in a
timely manner?

March 26, 2012

Page 6 of 9

2. Does ADPs User Interface respond and function as expected in a predictable and consistent
way, allowing all Georgia Southern employees to successfully complete time-critical tasks?
Information
Required
Additional
Information
Sources
Method of
Collection
Sampling Collection
Procedures
Reliability of
common system
functions
Degree of
accessibility
compliance and
W3C
specification
compliance
Information
concerning
security,
OS/browser
requirements,
and product
maintenance
Existing Help
Desk data
W3C check
and
HiSoftwares
Cynthia Says
Section 508
Accessibility
Report.

IT support staff)
Interviews with
key stakeholders
onsite

Interview
Director of
Technical
Services on
deployment and,
support
Interviews
conducted
2/28/12
through
3/23/12
and focus group

Verify security, OS/browser,
java compliance, including IT
support and maintenance
functions
Typical subgroup
(Departmental focus
group)
A Departmental
Focus Group
conducted onsite
2/28/12 Triangulate Help Desk metrics,
and focus group

experts (ITS Staff)

An online survey
that includes the
System Usability
Scale
Survey-
3/26/12
through
4/16/12
matching
Help Desk
metrics
Quantitative data for
the entire campus
population
Cross-tab report
of existing Help
Desk data over
18 months, by
problem type by
month

Received
2/10/12
and focus group

March 26, 2012

Page 7 of 9

W3C and
Accessibility
check
critical ADP web pages,
such as login page,
portal landing page, e-
time
Submit web page
HTML code for
W3C check and
HiSoftwares
Cynthia Says
Section 508
Accessibility
Report
2/23/12
through
2/24/12
Pass/Fail with full report of
any Section 508 Accessibility
and W3C errors found for the
most frequently used screens,
including explanation of
significance in terms of
usability

Document
Review &
Interface
inspection
OS and browser
requirements
Documentation
available on the
ADP & Georgia
southern HR
website
3/26/12
though
4/9/12
Ensure that requirements are
met
To what extent is the system available and does the system process user input in consistent and
predictable ways?

Within the context of the users role, do functions, such as mouse-click reliability, application start-up
time, logging in, session-persistence, updating passwords and personal information, recording and
approving time, requesting and approving time off, notification of actions, and retrieving reports,
behave intuitively, consistently, and accurately, while ensuring user control of time-sensitive content?

Does the ADP User Interface conform to minimum ADA Section 508's Accessibility Standards and W3C
specifications for valid markup, including text equivalents for non-text elements, scripts that allow
accessibility, providing frame titles, enable users to skip repetitive navigation links, and ensure plug-ins
and applets meet accessibility requirements? Do screens regularly utilized by all ADP users properly
display in the browser and are they accessible to those with disabilities?

Is the ADP User Interface secure using SSL protocol? Is the system platform and browser-agnostic?
Does the interface facilitate product support and delivery to all Georgia Southern employees by
performing with no system conflicts on ADP supported operating systems, browsers, plugins, and
secure java versions?
Address the question: Does
ADPs User Interface respond
and function as expected in a
predictable and consistent way,
allowing all Georgia Southern
employees to successfully
complete time-critical tasks?

timely manner?

March 26, 2012

Page 8 of 9

3. Do Georgia Southerns employees have positive attitudes toward ADP, and are they satisfied
interacting with the ADP User Interface for completing required tasks?
Information
Required
Additional
Information
Sources
Method of
Collection
Sampling Collection
Procedures
Data concerning
user attitude
toward ADPs
User Interface
Data concerning
user confidence
when interacting
with ADPs User
Interface.
SUS Score -
Overall level of
user satisfaction
with ADPs User
Interface.
Bangor, A.,
Kortum, P., &
Miller, J.
(2009).
Determining
what
individual
SUS Scores
mean: Adding
an adjective
rating scale.
Journal of
Usability
Studies, 4(2),
114-123

IT support staff)
Interviews with
key stakeholders

Interviews
conducted
2/28/12
through
3/23/12
Triangulate survey results, and
major themes found in
interviews and focus group
transcripts

Typical subgroup
(Departmental focus
group)
A Departmental
Focus Group
conducted onsite
2/28/12 Triangulate survey results, and
major themes found in
interviews and focus group
transcripts
experts (ITS Staff)

An online survey
that includes the
System Usability
Scale
Survey-
3/26/12
through
4/16/12
matching

Identify and extract usability
issues (if any) as distinguished
from organizational issues that
affect usability, but are not
system related

Compute and compare SUS
score results against 80%
standard from usability
surveys (based on proven
Likert scale)

March 26, 2012

Page 9 of 9

Within the context of their roles, to what extent do users express positive attitudes toward using ADPs
User Interface?

To what extent are users confident that a given task can be completed efficiently and successfully each
time the action is taken?

Is there a high overall satisfaction with the ADP User Interface, as defined by the System Usability Scale
score of 80 or above?

While out of scope for the evaluation, are there any organizational policies, issues or benefits that
should be listed in the report as influences on the overall fit to the ADP business model?
Address the question: Do
Georgia Southerns employees
have positive attitudes toward
ADP, and are they satisfied
interacting with the ADP User
Interface for completing
required tasks?

timely manner?

April 23, 2012
Page 42 of 47

Appendix B- ADP Help Desk Data
A
c
c
o
u
n
t

L
o
c
k
s
E
m
a
i
l

A
d
d
r
e
s
s

C
h
a
n
g
e
G
r
o
u
p
s
e
t

C
h
a
n
g
e
s
e
T
i
m
e

I
s
s
u
e
s
M
i
s
s
i
n
g

E
m
p
l
o
y
e
e

I
D
M
i
s
s
i
n
g

E
m
p
l
o
y
e
e
s
M
a
n
a
g
e
r

C
h
a
n
g
e
O
t
h
e
r
S
c
h
e
d
u
l
e

C
h
a
n
g
e
T
o
t
a
l
s
February-12 78 - - 25 26 9 15 4 6 163
January-12 258 8 - 40 5 10 31 9 2 363
December-11 112 5 - 19 2 3 18 15 1 175
November-11 170 6 - 22 7 3 33 11 3 255
October-11 172 2 - 44 6 12 26 17 9 288
September-11 237 7 1 90 28 21 48 25 3 460
August-11 214 3 - 81 48 18 56 9 5 434
July-11 133 2 - 26 6 12 50 11 7 247
June-11 154 1 - 46 2 12 27 3 5 250
May-11 165 2 - 41 2 8 25 7 1 251
April-11 139 6 1 70 6 10 25 12 - 269
March-11 110 1 4 47 22 8 23 13 - 228
February-11 165 5 - 60 26 27 21 27 2 333
January-11 184 6 2 49 16 28 59 5 - 349
December-10 64 - - 39 - 2 15 13 - 133
November-10 119 3 1 75 1 16 16 15 - 246
October-10 117 3 1 107 30 23 27 50 3 361
September-10 122 2 4 130 43 47 25 22 3 398
August-10 177 4 2 92 5 24 57 62 4 427
July-10 31 4 2 11 - 3 7 77 - 135
2921 70 18 1114 281 296 604 407 54 5765
ADP Help Desk Tickets
Georgia Southern University
Received February 10, 2012
April 23, 2012
Page 43 of 47

Appendix C- ADP Quality in Use Survey
ADP Qual i ty i n Use Survey
Greetings, colleagues!
This survey is part of an evaluation of the ADP Portal interface functionality and usability.
There are two parts to this survey. The first section is a standard System Usability Scale; and the
second part contains few questions on your satisfaction with specific aspects of ADP Portal.
Please submit this survey only once. Although we are recording usernames to ensure validity, your
responses will remain confidential and anonymous. For rating scale questions, only aggregate
information will be used in the final evaluation report. A few open comments might be selected for
the report; however, no identifiable information whatsoever will be associated with any of these
comments.
Thank you for taking a few minutes to complete this survey. We appreciate your feedback!
Your username (samwai nford@georgi asouthern.edu) will be recorded when you submit this
form. Not samwai nford? Sign out
* Required
Your Role in ADP
First, please select your role in ADP. This will allow filtering of responses based on the roles and
their various functions in the system.
Pl ease sel ect your *hi ghest* rol e i n the ADP portal . *
If you are an employee who also approves time for other employees, please choose
Manager/Approver. If you are an undergraduate student worker or temporary employee, please
choose Punch Time.
Choose highest role...
Part 1: System Usability Scale
The System Usability Scale was developed Digital Equipment Corporation and has been in use for
over a decade. It yields a single number grade ranging from 0 to 100 representing a composite
measure of the overall usability of the system.
Please record your immediate response to each of the 10 items, rather than thinking about items for
a long time. If you feel that you cannot respond to a particular item, please check the center point of
the scale.
1) I thi nk that I woul d l i ke to use thi s system frequentl y. *
1 2 3 4 5
Strongly disagree Strongly agree
2) I found the system unnecessari l y compl ex. *
1 2 3 4 5
3) I thought the system was easy to use. *
1 2 3 4 5
4) I thi nk that most peopl e woul d need the support of a techni cal person to be abl e to use
thi s system. *
1 2 3 4 5
5) I found the vari ous functi ons i n thi s system were wel l i ntegrated. *
1 2 3 4 5
6) I thought there was too much i nconsi stency i n thi s system. *
1 2 3 4 5
7) I woul d i magi ne that most peopl e woul d l earn to use thi s system very qui ckl y. *
1 2 3 4 5
8) I found the system very cumbersome to use. *
1 2 3 4 5
9) I fel t very confi dent usi ng the system. *
1 2 3 4 5
10) I needed to l earn a l ot of thi ngs before I coul d get goi ng wi th thi s system. *
1 2 3 4 5
Part 2: Satisfaction with Specifics
Please rate your level of satisfaction using ADP and e-time.
1 =very dissatisfied, 2 =dissatisfied, 3 =neutral or N/A, 4 =satisfied, and 5 =very satisfied
Pl ease rate your sati sfacti on wi th the l ogi n process. *
Consider the login process and the password retrieval and reset process.
1 2 3 4 5
Very dissatisfied Very satisfied
Pl ease rate your sati sfacti on usi ng e-ti me or edi ti ng the ti mecard. *
Consider any of the e-time functions that you use, e-time display in the browser, and reliability of
accepting and recording your input.
1 2 3 4 5
Pl ease rate the ADP portal navi gati on and menu system. *
Consider whether navigation is self-evident and logical, and whether it is easy to find the functions
and tools that you need.
1 2 3 4 5
Pl ease rate your sati sfacti on usi ng the ADP Portal to request and/or approve ti me off. *
Consider the leave request and approval process in ADP as well as its accuracy.
1 2 3 4 5
Pl ease rate the ADP portal and e-ti me avai l abi l i ty and responsi veness. *
Consider whether the system is available when needed, load time in the browser, and response to
mouse clicks.
1 2 3 4 5
Pl ease descri be ADP i n 3 words or l ess.
Pl ease share your opi ni on of your overal l experi ence usi ng ADP or on any speci fi cs that
may or may not be addressed i n thi s survey.
Please comment freely.
Send me a copy of my responses.
Submit
Powered by Google Docs
Report Abuse - Terms of Service - Additional Terms
April 23, 2012
Page 44 of 47

Appendix D- System Usability Scale Results and Explanation
April 23, 2012
Page 45 of 47

Appendix E- Questionnaire Data: Ratings of aspects of the ADP portal
Questionaire Data: Rating Scale from 1 to 5 for various aspects of the the ADP portal.
Lowest rating by role is highlighted.
Please select your
*highest* role in the
ADP portal.
Please rate your
satisfaction with
the login process.
Please rate your
satisfaction using
e-time or editing
the timecard.
Please rate the
ADP portal
navigation and
menu system.
Please rate your
satisfaction using
the ADP Portal to
request and/or
approve time off.
Please rate the
ADP portal and e-
time availability
and
responsiveness.
Totals Overall Average 2.69 2.50 2.19 2.29 2.75
Admin/Core User 3.00 3.00 3.00 3.00 2.00
Employee: Biweekly 3.50 3.50 3.50 2.50 3.50
Employee: Monthly 2.76 2.60 2.40 2.24 3.04
Manager/Approver 2.50 2.25 1.75 2.30 2.35
Admin/Core User 3 3 3 3 2
Employee: Biweekly 4 4 3 3 3
Employee: Biweekly 3 3 4 2 4
Sub total 3.5 3.5 3.5 2.5 3.5
Employee: Monthly 2 4 1 2 2
Employee: Monthly 4 1 4 1 1 Employee: Monthly 4 1 4 1 1
Sub total 2.76 2.6 2.4 2.24 3.04 25
Manager/Approver 2 1 3 1 3
Manager/Approver 3 2 1 1 1 g pp
Manager/Approver 4 2 2 4 2 Manager/Approver 4 2 2 4 2
Sub total 2.5 2.25 1.75 2.3 2.35 20
Overall Ratings 2.69 2.50 2.19 2.29 2.75
April 23, 2012
Page 46 of 47

Appendix F- HiSoftware Cynthia Says - Web Accessibility and W3C Reports
HiSoftware Cynthia Says - Web Content Accessibility Report
Powered by HiSoftware Content Quality Technology. If you have a question about this
output please email support@hisoftware.com
Verified File Name: https://portal.adp.com/public/index.htm
Date and Time: 2/23/2012 7:47:57 AM
Failed Automated Verification
Emulated Browser: MS Internet Explorer 6.0
HiSoftware can help you meet all of your accessibility
needs and more. Our industry leading enterprise content compliance soultions provide
you with an automated, full-featured monitoring, auditing and testing solution to
ensure your ever-changing Web content is always compliant with the latest standards
for accessibility, privacy and confidentiality, site quality and data and information
security. Visit www.hisoftware.com to find out more about how HiSoftware solutions
can help you meet your Web compliance goals and request a trial copy.
Read The Accessibility Handbook today! Download Now
The level of detail setting for the report is to show all detail.
Verification Checklist
Checkpoints Passed
508 Standards, Section 1194.22 Yes No Other
A. 508 Standards, Section 1194.22, (a) A text equivalent for every non-text element shall be provided (e.g., via "alt", "longdesc", or in element content).
Rule: 1.1.1 - All IMG elements are required to contain either the alt or the longdesc attribute.
Warning - IMG Element found at Line: 133, Column: 58 contains the 'alt' attribute with an empty value. Please verify that this image is only used for spacing or design
and has no meaning.
Failure - IMG Element at Line: 138, Column: 71
and has no meaning.
and has no meaning.
and has no meaning.
and has no meaning.
and has no meaning.
and has no meaning.
Rule: 1.1.2 - All INPUT elements are required to contain the alt attribute or use a LABEL.
No invalid INPUT elements found in document
Rule: 1.1.3 - All OBJECT elements are required to contain element content.
No OBJECT elements found in document body.
Rule: 1.1.4 - All APPLET elements are required to contain both element content and the alt attribute.
No APPLET elements found in document body.
Rule: 1.1.6 - All IFRAME elements are required to contain element content.
No IFRAME elements found in document body.
Rule: 1.1.7 - All Anchor elements found within MAP elements are required to contain the alt attribute.
No MAP elements found in document body.
Rule: 1.1.8 - All AREA elements are required to contain the alt attribute.
No AREA elements found in document body.
Rule: 1.1.9 - When EMBED Elements are used, the NOEMBED element is required in the document.
Failure - EMBED Element(s) are found in document and the NOEMBED element is not.
No
B. 508 Standards, Section 1194.22, (b) Equivalent alternatives for any multimedia presentation shall be synchronized with the presentation.
Rule: 1.4.1 - Identify all OBJECT Elements that have a multimedia MIME type as the type attribute value.
Rule: 1.4.2 - Identify all OBJECT Elements that have a 'data' attribute value with a multimedia file extension.
Rule: 1.4.3 - Identify all EMBED Elements that have a 'src' attribute value with a multimedia file extension.
Note: EMBED Element found at Line: 140, Column: 21 appears to have '.swf' as a file extension in the 'src' attribute.

C. 508 Standards, Section 1194.22, (c) Web pages shall be designed so that all information conveyed with color is also available without color, for example from context
or markup.

D. 508 Standards, Section 1194.22, (d) Documents shall be organized so they are readable without requiring an associated style sheet.
Note: Document uses external stylesheets, inline style information, or header style information.

E. 508 Standards, Section 1194.22, (e) Redundant text links shall be provided for each active region of a server-side image map.
Rule: 1.2.1 - Locate any IMG element that contains the 'ismap' attribute.
No IMG elements found in document body that contain the 'ismap' attribute.
Rule: 1.2.2 - Locate any INPUT element that contains the 'ismap' attribute.
No INPUT elements found in document body that contain the 'ismap' attribute.
N/A
F. 508 Standards, Section 1194.22, (f) Client-side image maps shall be provided instead of server-side image maps except where the regions cannot be defined with an
available geometric shape.
Rule: 9.1.1 - No IMG element should contain the 'ismap' attribute.
No server-side image map IMG elements found in document body.
Rule: 9.1.2 - No INPUT element should contain the 'ismap' attribute.
No server-side image map INPUT elements found in document body.
N/A
G. 508 Standards, Section 1194.22, (g) Row and column headers shall be identified for data tables.
Rule: 5.1.1 - Identify all Data TABLE elements.
Note: Data TABLE Element found at Line: 128, Column: 15.

H. 508 Standards, Section 1194.22, (h) Markup shall be used to associate data cells and header cells for data tables that have two or more logical levels of row or
column headers.
Rule: 5.2.1 - Identify all Data TABLE elements.

Cynthia Says Report http://www.cynthiasays.com/mynewtester/cynthia.exe
1 of 5 2/23/2012 7:48 AM
Checkpoints Passed
I. 508 Standards, Section 1194.22, (i) Frames shall be titled with text that facilitates frame identification and navigation.
Document is not a FRAMESET Page.
N/A
J. 508 Standards, Section 1194.22, (j) Pages shall be designed to avoid causing the screen to flicker with a frequency greater than 2 Hz and lower than 55 Hz.
Rule: 7.1.1 - Documents are required not to contain the BLINK element.
No BLINK elements found in document body.
Rule: 7.1.2 - Documents are required not to contain the MARQUEE element.
No MARQUEE elements found in document body.
Note: SCRIPT element(s) found in document body, a visual verification must be done to ensure the script does not cause the screen to flicker.

K. 508 Standards, Section 1194.22, (k) A text-only page, with equivalent information or functionality, shall be provided to make a web site comply with the provisions of
this part, when compliance cannot be accomplished in any other way. The content of the text-only page shall be updated whenever the primary page changes.

(k) Option A - Check for the string 'Text Version' within the document. N/V
(k) Option B - Check for a Global Text Version Link within the document. N/V
(k) Option C - Check for an Accessibility Policy Link within the document. N/V
L. 508 Standards, Section 1194.22, (l) When pages utilize scripting languages to display content, or to create interface elements, the information provided by the script
shall be identified with functional text that can be read by assistive technology.
Rule: 6.3.1 - Anchor elements are required not to use javascript for the link target when the NOSCRIPT element is not present in the document. These elements will not cause a
failure of the checkpoint if the NOSCRIPT element is found, however, they will be identified.
Warning: Anchor Element found at Line: 188, Column: 25 contains 'javascript:' in the 'href' attribute and the NOSCRIPT element is not present within the document body.
Rule: 6.3.2 - AREA elements are required not to use javascript for the link target when the NOSCRIPT element is not present in the document. These elements will not cause a
failure of the checkpoint if the NOSCRIPT element is found, however, they will be identified.
No AREA Elements found in document body.
Rule: 6.3.3 - Locate elements that use HTML event handlers.
Note: This rule has not been selected to be verified for this checkpoint.
Rule: 6.3.4 - When SCRIPT Elements are used, the NOSCRIPT element is required in the document.
Warning - SCRIPT Element(s) are found in document and the NOSCRIPT element is not.

M. 508 Standards, Section 1194.22, (m) When a web page requires that an applet, plug-in or other application be present on the client system to interpret page
content, the page must provide a link to a plug-in or applet that complies with 1194.21(a) through (l).
Rule: 6.3.5 - All OBJECT elements are required to contain element content.
Rule: 6.3.6 - All APPLET elements are required to contain both element content and the alt attribute.
No APPLET elements found in document body.
Rule: 6.3.7 - When EMBED Elements are used, the NOEMBED element is required in the document.
Failure - EMBED Element(s) are found in document and the NOEMBED element is not.
Rule: 6.3.8 - All pages that have links to files that require a special reader or plug-in are required to contain the specified text indicating a link to the reader or plug-in.
No
N. 508 Standards, Section 1194.22, (n) When electronic forms are designed to be completed on-line, the form shall allow people using assistive technology to access the
information, field elements, and functionality required for completion and submission of the form, including all directions and cues.
N/A
O. 508 Standards, Section 1194.22, (o) A method shall be provided that permits users to skip repetitive navigation links.
Rule: (o).1 - All pages are required to contain a bookmark link to skip navigation that has the specified text in either the link text or the 'title' attribute value.
Skip Navigation Text:

P. 508 Standards, Section 1194.22, (p) When a timed response is required, the user shall be alerted and given sufficient time to indicate more time is required.
Checkpoint Result Legend: Yes = Passed Automated Verification, No = Failed Automated Verification, Warning = Failed Automated Verification, however, configured not to cause page to fail (Priority 2 or 3 only), N/ V = Not selected for
verification, N/ A = No related elements were found in document (Visual only), No Value = Visual Checkpoint
HiSoftware Alt Text Quality Report
Verified File Name: https://portal.adp.com/public/index.htm
Date and Time: 2/23/2012 7:47:57 AM
Failed Automated Verification
Emulated Browser: MS Internet Explorer 6.0
Verification Checklist
Checkpoints Passed
Yes No Other
1.1 Validate that the alt text does not use the word image When users add alternative text to an image they tend to add the word "Image" when it really says
nothing about the image, but describes the object versus the meaning of the object. This check will fail a page for the use of the word image in the alternative text.
Image Alternative Text should not contain the word "Image"
Failure - img Element at Line: 138, Column: 71 - The element does not use the attribute: alt.
No
1.2 Validate that the alt text does not contain the text: .jpg, .gif, .bmp, .jpeg Many content creation tools will automatically add alternative text when you add
an image to your content. The text is generally the image name. Validate that: .jpg, .gif, .bmp, .jpeg, are not found in the alt text.
Image alternative text should not contain : .jpg, .gif, .bmp, .jpeg
No
1.6 Validate that the alt text does not contain the text "image" Many content creation tools will automatically add alternative text when you add an image to
your content. The text is generally the image name or the word image with a number associated, like image001. This checkpoint will fail a page if the string image is
found in the alternative text.
Image alternative text should not contain the text "image"
No
2 of 5 2/23/2012 7:48 AM
Checkpoints Passed
2.1 Validate that Alternative Text is greater than 7 and less than 81 characters in length Short alternative text may not be valid, warn the report user if
alternative text was found that is less than seven characters in length. Additionally alternative text should not be larger than 80 characters, if the alt text is greater the
long description attribute should be used. This check validates that the alt attribute does not exceed 80 characters in length.
The alternative text failed the minimum/maximum allowed characters check
Note - img Element at Line: 133, Column: 58 - The alt attribute is 0 characters.
Note - img Element at Line: 138, Column: 71 - The element does not use the attribute: alt.
Warning
2.2 Validate that Alternative Text is not used to repeat words Alternative text should not be used to simply hide words with the hope of increasing your ranking
on search engines. If you repeat a word more than 5 times your page may not be indexed.
The alternative text failed the maximum allowed repeated words check
No
Checkpoint Result Legend: Yes = Passed Automated Verification, No = Failed Automated Verification, Warning = Automated Verification Warning, N/ V = Not Verified, N/ A = No related elements were found in document, No Value = Visual
Checkpoint
Report generated by the HiSoftware Company Cynthia Agent. Powered by the AccMonitor Compliance Server HiSoftware, Cynthia Says, AccMonitor Compliance Server,
Cynthia Agent are all trademarks of HiSoftware Inc. (www.hisoftware.com 603.578.1870 or 1.888.272.2484)
Source Listing
1: <ht ml l ang="en">
2: <head>
3: <t i t l e>Sel f Ser vi ce Por t al </ t i t l e>
4: <META ht t p- equi v=Cont ent - Type cont ent ="t ext / ht ml ; char set =wi ndows- 1252">
5: <META ht t p- equi v=expi r es cont ent =0>
6: <META ht t p- equi v=Pr agma cont ent =no- cache>
7: <META cont ent ="payr ol l , human r esour ce, benef i t s, HR, paycheck, ADP, payr ol l ser vi ces, benef i t s admi ni st r at i on, human r esour ces, adp payr ol l , HR sof t war e, pay
8: <META cont ent ="ADP, t he payr ol l l eader , of f er s benef i t admi ni st r at i on, human r esour ce and r et i r ement ser vi ces f or busi nesses of any si ze. " name=des
9: <META cont ent =document name=r esour ce- t ype>
10: <META cont ent =gl obal name=di st r i but i on>
11: <META cont ent ="al l cont ent s copyr i ght 2010 Aut omat i c Dat a Pr ocessi ng, al l r i ght s r eser ved. " name=copyr i ght >
12: <l i nk hr ef ="t heme/ adppor t al 1. css" r el ="st yl esheet " t ype="t ext / css">
13: <scr i pt l anguage="j avascr i pt ">
14: <! - -
15:
16: <! - - hi er menu scr i pt - - >
17:
18: i f ( wi ndow. event + "" == "undef i ned") {
19: event = nul l ;
20: }
21:
22: f unct i on HM_f _PopUp( ) {
23: r et ur n f al se
24: };
25:
26: f unct i on HM_f _PopDown( ) {
27: r et ur n f al se
28: };
29:
30: popUp = HM_f _PopUp;
31: popDown = HM_f _PopDown;
32:
33: / / - - >
34: </ scr i pt >
35: <! - - <SCRI PT LANGUAGE="J avaScr i pt 1. 2" SRC=". . / i ncl udes/ HM_Loader . j s" TYPE=' t ext / j avascr i pt ' ></ SCRI PT>- - >
36: <SCRI PT LANGUAGE="J avaScr i pt ">
37:
38:
39: / / FUNCTI ON TO OPEN A CONSTRAI NED WI NDOW
40: f unct i on openWi n( URL) {
41: Wi ndow = wi ndow. open( URL, "wi ndow", "wi dt h=575, hei ght =455, st at us=no, scr ol l bar s=yes, r esi zabl e=yes, l ocat i on=0, menubar =0, t ool bar =no") ;
42: Wi ndow. f ocus( ) ;
43: }
44:
45: / / st op hi di ng me - - >
46:
3 of 5 2/23/2012 7:48 AM
47: f unct i on cl eanupCooki es( ) {
48: Del et e_Cooki e( "A1", "/ ", ". adp. com" ) ;
53: Del et e_Cooki e( "B1", "/ ", ". adp. com" ) ;
54: Del et e_Cooki e( "R1", "/ ", ". adp. com" ) ;
55: Del et e_Cooki e( "PORTLETREDI RECTCOOKI E", "/ ", ". adp. com" ) ;
56: Del et e_Cooki e( "RELOGI NI MPERSONATECOOKI E", "/ ", ". adp. com" ) ;
57: Del et e_Cooki e( "J SESSI ONI D", "/ ", "" ) ;
58: }
59:
60: f unct i on get Cooki e( name) {
61: var st ar t = document . cooki e. i ndexOf ( name+"=") ;
62: var l en = st ar t +name. l engt h+1;
63: i f ( ( ! st ar t ) && ( name ! = document . cooki e. subst r i ng( 0, name. l engt h) ) ) r et ur n nul l ;
64: i f ( st ar t == - 1) r et ur n nul l ;
65: var end = document . cooki e. i ndexOf ( "; ", l en) ;
66: i f ( end == - 1) end = document . cooki e. l engt h;
67: r et ur n unescape( document . cooki e. subst r i ng( l en, end) ) ;
68: }
69:
70: f unct i on Get _Cooki e( check_name ) {
71: var a_al l _cooki es = document . cooki e. spl i t ( ' ; ' ) ;
72: var a_t emp_cooki e = ' ' ;
73: var cooki e_name = ' ' ;
74: var cooki e_val ue = ' ' ;
75: var b_cooki e_f ound = f al se;
76:
77: f or ( i = 0; i < a_al l _cooki es. l engt h; i ++ )
78: {
79: a_t emp_cooki e = a_al l _cooki es[ i ] . spl i t ( ' =' ) ;
80:
81:
82: cooki e_name = a_t emp_cooki e[ 0] . r epl ace( / ^\ s+| \ s+$/ g, ' ' ) ;
83:
84: i f ( cooki e_name == check_name )
85: {
86: b_cooki e_f ound = t r ue;
87: i f ( a_t emp_cooki e. l engt h > 1 )
88: {
89: cooki e_val ue = unescape( a_t emp_cooki e[ 1] . r epl ace( / ^\ s+| \ s+$/ g, ' ' ) ) ;
90: }
91: r et ur n cooki e_val ue;
92: br eak;
93: }
94: a_t emp_cooki e = nul l ;
95: cooki e_name = ' ' ;
96: }
97: i f ( ! b_cooki e_f ound )
98: {
99: r et ur n nul l ;
100: }
101: }
102:
103:
104: f unct i on Set _Cooki e( name, val ue, pat h, domai n ) {
105: i f ( Get _Cooki e( name ) ) {
106: document . cooki e = name + "=" + val ue +( ( pat h ) ? "; pat h=" + pat h : "") + ( ( domai n ) ? "; domai n=" + domai n : "" ) + "; ";
107: }
108: }
109:
110: f unct i on Del et e_Cooki e( name, pat h, domai n ) {
111: i f ( Get _Cooki e( name ) ) {
112: document . cooki e = name + "=" +( ( pat h ) ? "; pat h=" + pat h : "") + ( ( domai n ) ? "; domai n=" + domai n : "" ) + "; expi
113: }
114: }
115:
116: cl eanupCooki es( ) ;
117:
118: </ SCRI PT>
119: </ head>
120:
121: <body bgCol or =#f f f f f f l ef t Mar gi n=0 t opMar gi n=0 mar gi nwi dt h="0" mar gi nhei ght ="0">
122: <t abl e wi dt h="100%" hei ght ="100%" bor der ="0" cel l paddi ng="0" cel l spaci ng="0">
123: <t r >
124: <t d al i gn="cent er " val i gn="mi ddl e">
125: <t abl e wi dt h="740" bor der ="1" cel l paddi ng="0" cel l spaci ng="0" bor der col or ="#000000">
126: <t r >
127: <t d>
128: <t abl e wi dt h="100%" hei ght ="45" bor der ="0" cel l paddi ng="0" cel l spaci ng="0" cl ass="pagehead" dwcopyt ype="CopyTabl eRow">
129: <t r >
130: <t d wi dt h="50" nowr ap><a hr ef ="ht t p: / / www. adp. com/ i ndex. ht ml " ></ a></ t d>
131: <t d al i gn="l ef t " nowr ap ><st r ong><f ont col or ="#666666" si ze="3" f ace="Ar i al , Hel vet i ca, sans- ser i f "></ f ont ></ st r ong></ t d>
132: <t d al i gn="r i ght " nowr ap cl ass="l ef t nav" ></ t d>
133: <t d wi dt h="20" val i gn="bot t om" nowr ap> </ t d>
134: </ t r >
135: </ t abl e>
136: <t abl e cel l Spaci ng="0" cel l Paddi ng="0" wi dt h="100%" bor der =" 0">
137: <t r >
138: <t d wi dt h="2" vAl i gn="t op" nowr ap bgCol or ="#CC0000"></ t d>
139: <t d wi dt h="4" vAl i gn="t op" nowr ap bgCol or ="#CC0000">
140: <embed sr c="i mages/ spl ash_i mage. swf " qual i t y="hi gh" pl ugi nspage="ht t p: / / www. macr omedi a. com/ shockwave/ downl oad/ i ndex. cgi ?P1_Pr od_V
141: </ t d>
142: <t d vAl i gn="t op" wi dt h="1"> </ t d>
143: <t d vAl i gn="t op" wi dt h="313" bgCol or ="#9e9e9e"> </ t d>
144: <t d vAl i gn="t op" wi dt h="1"> </ t d>
145: <t d wi dt h="100%" hei ght ="100%" vAl i gn="t op" bgcol or ="#ECE6C4">
146: <t abl e cel l spaci ng="0" cel l paddi ng="0" wi dt h="100%" bor der ="0">
147: <t r >
148: <t d val i gn="t op" wi dt h="100%" bgcol or ="#CC0000" col span="2">
150: </ t abl e>
151: <t abl e wi dt h="100%" bor der ="0" cel l Paddi ng="0" cel l Spaci ng="0" cl ass="copybol d">
4 of 5 2/23/2012 7:48 AM
152: <t r >
153: <t d hei ght ="50" al i gn="cent er " val i gn="mi ddl e" bgCol or ="#E5e5e5">
154: <i nput name="empl oyee" t ype="but t on" cl ass="but t onf or m" i d="empl oyee" val ue="User Logi n" onCl i ck="document . l ocat i on=' / wps/ e
155: <! - - <i nput name="admi ni st r at or 2" t ype=" but t on" cl ass="but t onf or m" i d="admi ni st r at or 2" val ue="Manager Logi n" o
156:
157: 
159: </ t r >
160: <t r >
161: <t d hei ght ="6" bgcol or ="#999999"> </ t d>
162: </ t r >
163: </ t abl e>
164: <t abl e hei ght ="160" bor der ="0" al i gn="cent er " cel l paddi ng="0" cel l spaci ng="0">
165: <t r >
166: <t d wi dt h="100%" hei ght ="176">
167: <a hr ef ="ht t ps: / / net secur e. adp. com/ pages/ pub/ ccmai n. j sp">Fi r st Ti me User s Reg
168: <a hr ef ="/ publ i c/ Get t i ng_St ar t ed/ get t i ng_st ar t ed/ 1_f r ameset . ht m">Need Hel p Ge
169: <a hr ef ="ht t ps: / / net secur e. adp. com/ pages/ basi c/ EU/ edi t User . j sp">Updat e My Sec
170: <a hr ef ="ht t ps: / / net secur e. adp. com/ pages/ basi c/ changepw. j sp">Change your Passwor d</ a> 
171:
172: <a hr ef ="#" onCl i ck="t hi s. st yl e. behavi or =' ur l ( homepage) ' ; t hi s. set HomePage( ' h
173: <a hr ef ="ht t ps: / / net secur e. adp. com/ pages/ pub/ f or got UI D. j sp?r t nURL=ht t ps: / / por
174: <a hr ef ="ht t ps: / / net secur e. adp. com/ pages/ pub/ f or got PW. j sp?r t nURL=ht t ps: / / por t al . adp. com/ ">For got your
175: </ t d>
176: </ t r >
177: </ t abl e>
178: </ t d>
179: </ t r >
180: </ t abl e>
181: <t abl e bor der ="0" cel l spaci ng="0" cel l paddi ng="0" wi dt h="100%" bgcol or ="#333333">
182: <t r >
183: <t d></ t d>
184: <t d wi dt h="50%"><f ont col or ="#CCCCCC" si ze="1" f ace="Ar i al , Hel vet i ca, sans- ser i f ">© 2010 Aut omat i c Dat a Pr ocessi ng, I nc. </ f ont
185: <t d wi dt h="50%" al i gn="r i ght ">
186: <f ont i d="m3">
187: <! - - ADP PRI VACY LI NK - POP UP WI NDOW- - >
188: <a hr ef ="j avascr i pt : openWi n( ' ht t p: / / www. adp. com/ pr i vacy. ht ml ' ) " onMouseOver ="wi ndow. st at us = ' ADP Web Si t e Pr i vacy St at ement '
189: <a hr ef ="j avascr i pt : openWi n( ' ht t p: / / www. adp. com/ pr i vacy. ht ml ' ) " cl ass="f oot l i nk" onMouseOver ="wi ndow. st at us = ' ADP Web Si t e P
190: <f ont i d="m4">
191: <! - - ADP LEGAL LI NK - POP UP WI NDOW- - >
192: <a hr ef ="j avascr i pt : openWi n( ' ht t p: / / www. adp. com/ l egal . ht ml ' ) " onMouseOver ="wi ndow. st at us = ' ADP Web Si t e Legal Ter ms and Co
193: <a hr ef ="j avascr i pt : openWi n( ' ht t p: / / www. adp. com/ l egal . ht ml ' ) " cl ass="f oot l i nk" onMouseOver =""wi ndow. st at us = ' ADP Web Si t e
194: <f ont i d="m5"></ f ont >
195: </ f ont >
196: </ f ont >
197: </ f ont >
198: </ t d>
199: <t d wi dt h="20" val i gn="t op"> </ t d>
200: </ t r >
201: </ t abl e>
202: </ t d>
203: </ t r >
204: </ t abl e>
205:
206: 
207: Thi s si t e r equi r es <a hr ef ="ht t p: / / www. mi cr osof t . com/ wi ndows/ i e/ def aul t . asp" t ar get ="_bl ank"><st r ong>Mi cr osof t I nt er net
208: Expl or er Ver si on 6. 0 </ st r ong></ a> or hi gher . 
209: </ t d>
210: </ t r >
211: </ t abl e>
212: <scr i pt l anguage="J avaScr i pt ">
213: </ scr i pt >
214:
215: </ body>
216: </ ht ml >
5 of 5 2/23/2012 7:48 AM
2940

Help on the options is available.
Revalidate
Markup Validation Service
Check t he markup (HTML, XHTML, ) of Web document s
Jump To: Notes and Potential Issues Validation Output
Errors found while checking this document as HTML 4.01
Transitional!
Result: 33 Errors, 11 warning(s)
Address : ht t ps: / / por t al . adp. com/ publ i c/ i ndex. ht m
Encoding : windows-1252 (detect automatically)
Doctype :
HTML 4.01
Transitional
(detect automatically)
Root Element: html
The W3C validators are developed with assistance
from the Mozilla Foundation, and supported by
community donations.
Donate and help us build better tools for a better web.
Options
M|.Show Source M|.Show Outline
*'^List Messages Sequentially *'Group
Error Messages by Type
M|.
Validate error pages
M|.
Verbose Output
M|.Clean up Markup with HTML-Tidy
Notes and Potential Issues
The following notes and warnings highlight missing or conflicting information which
caused the validator to perform some guesswork prior to validation, or other things
affecting the output below. If the guess or fallback is incorrect, it could make
Page 1of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup V...
2/23/2012 http://validator.w3.org/check?uri=https%3A%2F%2Fportal.adp.com%2Fpublic%2Findex....
validation results entirely incoherent. It is highly recommended to check these
potential issues, and, if necessary, fix them and re-validate the document.
Unable to Determine Parse Mode!
The validator can process documents either as XML (for document types such
as XHTML, SVG, etc.) or SGML (for HTML 4.01 and prior versions). For this
document, the information available was not sufficient to determine the parsing
mode unambiguously, because:
the MIME Media Type (t ext / ht ml ) can be used for XML or SGML
document types
No known Document Type could be detected
No XML declaration (e.g <?xml ver si on=" 1. 0" ?>) could be
found at the beginning of the document.
No XML namespace (e.g <ht ml
xml ns=" ht t p: / / www. w3. or g/ 1999/ xht ml " xml : l ang=" en" >)
could be found at the root of the document.
As a default, the validator is falling back to SGML mode.
No DOCTYPE found! Checking with default HTML 4.01 Transitional
Document Type.
No DOCTYPE Declaration could be found or recognized in this document. This
generally means that the document is not declaring its Document Type at the
top. It can also mean that the DOCTYPE declaration contains a spelling error, or
that it is not using the correct syntax.
The document was checked using a default "fallback" Document Type Definition
that closely resembles HTML 4.01 Transitional.
Learn how to add a doctype to your document from our FAQ.
TOP
Validation Output: 33 Errors
Line 1, Column 1: no document type declaration; implying " <!DOCTYPE
HTML SYSTEM>"
< ht ml l ang=" en" >
The checked page did not contain a document type ("DOCTYPE") declaration. The
Validator has tried to validate with a fallback DTD, but this is quite likely to be
incorrect and will generate a large number of incorrect error messages. It is highly
recommended that you insert the proper DOCTYPE declaration in your document --
instructions for doing this are given above -- and it is necessary to have this
declaration before the page can be declared to be valid.
Line 13, Column 32: required attribute " TYPE" not specified
<scr i pt l anguage=" j avascr i pt " >
The attribute given above is required for an element that you've used, but you have
omitted it. For instance, in most HTML and XHTML document types the "type"
attribute is required on the "script" element and the "alt" attribute is required for the
"img" element.
Typical values for t ype are t ype=" t ext / css" for <st yl e> and
t ype=" t ext / j avascr i pt " for <scr i pt >.
<SCRI PT LANGUAGE=" J avaScr i pt " >
"img" element.
Line 121, Column 15: an attribute value must be a literal unless it
contains only name characters
<body bgCol or =# f f f f f f l ef t Mar gi n=0 t opMar gi n=0 mar gi nwi dt h=" 0" mar gi nhei ght ="0

You have used a character that is not considered a "name character" in an attribute
value. Which characters are considered "name characters" varies between the
different document types, but a good rule of thumb is that unless the value contains
only lower or upper case letters in the range a-z you must put quotation marks
around the value. In fact, unless you have extreme file size requirements it is a very
very good idea to always put quote marks around your attribute values. It is never
wrong to do so, and very often it is absolutely necessary.
Line 121, Column 34: there is no attribute " LEFTMARGIN"
<body bgCol or =#f f f f f f l ef t Mar gi n=0 t opMar gi n=0 mar gi nwi dt h=" 0" mar gi nhei ght ="0
You have used the attribute named above in your document, but the document type
you are using does not support that attribute for this element. This error is often
caused by incorrect use of the "Strict" document type with a document that uses
frames (e.g. you must use the "Transitional" document type to get the "target"
attribute), or by using vendor proprietary extensions such as "marginheight" (this is
usually fixed by using CSS to achieve the desired effect instead).
This error may also result if the element itself is not supported in the document type
you are using, as an undefined element will have no supported attributes; in this
case, see the element-undefined error message for further information.
How to fix: check the spelling and case of the element and attribute, (Remember
XHTML is all lower-case) and/or check that they are both allowed in the chosen
document type, and/or use CSS instead of this attribute. If you received this error
when using the <embed> element to incorporate flash media in a Web page, see
the FAQ item on valid flash.
Line 121, Column 46: there is no attribute " TOPMARGIN"
<body bgCol or =#f f f f f f l ef t Mar gi n=0 t opMar gi n=0 mar gi nwi dt h=" 0" mar gi nhei ght ="0

Line 121, Column 60: there is no attribute " MARGINWIDTH"
<body bgCol or =#f f f f f f l ef t Mar gi n=0 t opMar gi n=0 mar gi nwi dt h=" 0" mar gi nhei ght =" 0
Line 121, Column 77: there is no attribute " MARGINHEIGHT"
<body bgCol or =#f f f f f f l ef t Mar gi n=0 t opMar gi n=0 mar gi nwi dt h=" 0" mar gi nhei ght =" 0

Line 122, Column 30: there is no attribute " HEIGHT"
<t abl e wi dt h=" 100%" hei ght =" 100%" bor der =" 0" cel l paddi ng=" 0" cel l spaci ng=" 0"
Line 125, Column 83: there is no attribute " BORDERCOLOR"
e wi dt h=" 740" bor der =" 1" cel l paddi ng=" 0" cel l spaci ng=" 0" bor der col or =" #000000
Line 128, Column 118: there is no attribute " DWCOPYTYPE"
" 0" cel l paddi ng=" 0" cel l spaci ng=" 0" cl ass=" pagehead" dwcopyt ype=" CopyTabl eRow

Line 130, Column 139: NET-enabling start-tag requires SHORTTAG YES
dex. ht ml " ></ a></ t
The sequence <FOO /> can be interpreted in at least two different ways, depending
on the DOCTYPE of the document. For HTML 4.01 Strict, the '/' terminates the tag
<FOO (with an implied '>'). However, since many browsers don't interpret it this
way, even in the presence of an HTML 4.01 Strict DOCTYPE, it is best to avoid it
completely in pure HTML documents and reserve its use solely for those written in
XHTML.
" i mages/ bl ank_pi xel . gi f " wi dt h=" 20" hei ght =" 10" bor der =" 0" al t =" " / > </ t
XHTML.
" i mages/ bl ank_pi xel . gi f " wi dt h=" 20" hei ght =" 10" bor der =" 0" al t =" " / > </ t

XHTML.
Line 138, Column 127: required attribute " ALT" not specified
gCol or =" #CC0000" > </ t
"img" element.
Line 140, Column 32: there is no attribute " SRC"
<embed sr c=" i mages/ spl ash_i mage. swf " qual i t y=" hi gh" pl ugi n
Line 140, Column 66: there is no attribute " QUALITY"
sr c=" i mages/ spl ash_i mage. swf " qual i t y=" hi gh" pl ugi nspage=" ht t p: / / www. macr omed

Line 140, Column 85: there is no attribute " PLUGINSPAGE"
i mage. swf " qual i t y="hi gh" pl ugi nspage=" ht t p: / / www. macr omedi a. com/ shockwave/ do
Line 140, Column 178: there is no attribute " TYPE"
?P1_Pr od_Ver si on=ShockwaveFl ash" t ype=" appl i cat i on/ x- shockwave- f l ash" wi dt h="

Line 140, Column 216: there is no attribute " WIDTH"
veFl ash" t ype=" appl i cat i on/ x- shockwave- f l ash" wi dt h=" 250" hei ght =" 250" ></ embe
veFl ash" t ype=" appl i cat i on/ x- shockwave- f l ash" wi dt h=" 250" hei ght =" 250" ></ embe
Page 10of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup...

Line 140, Column 234: element " EMBED" undefined
veFl ash" t ype=" appl i cat i on/ x- shockwave- f l ash" wi dt h=" 250" hei ght =" 250" > </ embe
You have used the element named above in your document, but the document type
you are using does not define an element of that name. This error is often caused
by:
incorrect use of the "Strict" document type with a document that uses frames (e.g.
you must use the "Frameset" document type to get the "<frameset>" element),
by using vendor proprietary extensions such as "<spacer>" or "<marquee>" (this is
by using upper-case tags in XHTML (in XHTML attributes and elements must be all
lower-case).
i mg sr c=" i mages/ bul l et . gi f " bor der =" 0" > <a hr ef =" ht t ps: / / net secur e. adp. c
"img" element.
i mg sr c=" i mages/ bul l et . gi f " bor der =" 0" > <a hr ef =" / publ i c/ Get t i ng_St ar t ed

"img" element.
"img" element.
"img" element.
i mg sr c=" i mages/ bul l et . gi f " bor der =" 0" > <a hr ef =" #" onCl i ck=" t hi s. st yl e.
"img" element.

"img" element.
"img" element.
g sr c=" i mages/ bl ank_pi xel . gi f " wi dt h=" 10" hei ght =" 10" bor der =" 0" al t =" " / ></ t
XHTML.
vet i ca, sans- ser i f " >© 2010 Aut omat i c Dat a Pr ocessi ng, I nc. </ f ont > </ t

XHTML.
i mg4" bor der =" 0" al t =" Cl i ck her e f or t he ADP Web Si t e Pr i vacy St at ement " / ></
XHTML.
Line 190, Column 35: ID " M4" already defined
<f ont i d=" m 4" >
An "id" is a unique identifier. Each time this attribute is used in a document it must
have a different value. If you are using this attribute as a hook for style sheets it
may be more appropriate to use classes (which group elements) than id (which are
used to identify exactly one element).
Line 189, Column 245: ID "M4" first defined here
eOut =" " t i t l e=" ADP Web Si t e Pr i vacy St at ement " ><f ont i d=" m 4" >PRI VACY</ f ont ><
al t =" Cl i ck her e f or t he ADP Web Si t e Legal Ter ms and Condi t i ons of Use" / ></
XHTML.

Line 193, Column 136: there is no attribute " WINDOW.STATUS"
ot l i nk" onMouseOver =" " wi ndow. st at us = ' ADP Web Si t e Legal Ter ms and Condi t i on
Line 193, Column 184: character " ;" not allowed in attribute specification
list
i t e Legal Ter ms and Condi t i ons of Use' ; r et ur n t r ue; " onMouseOut =" " t i t l e=" ADP
Line 194, Column 37: ID " M5" already defined
<f ont i d=" m 5" ><f ont i d=" m 5" >LEGAL</ f ont ><

<f ont i d=" m5" > </ f on
"img" element.
Line 197, Column 27: end tag for element " FONT" which is not open
</ f ont >
The Validator found an end tag for the above element, but that element is not
currently open. This is often caused by a leftover end tag from an element that was
removed during editing, or by an implicitly closed element (if you have an error
related to an element being used where it is not allowed, this is almost certainly the
case). In the latter case this error will disappear as soon as you fix the original
problem.
If this error occurred in a script section of your document, you should probably read
this FAQ entry.
r c=" i mages/ bl ank_pi xel . gi f " wi dt h=" 1" hei ght =" 1" bor der =" 0" al t =" " / > </ t
XHTML.
r c=" i mages/ bl ank_pi xel . gi f " wi dt h=" 1" hei ght =" 1" bor der =" 0" al t =" " / > </ t


XHTML.
<scr i pt l anguage=" J avaScr i pt " >
"img" element.
TOP
Home About... News Docs Help & FAQ Feedback Contribute
This service runs the W3C Markup Validator, v1.2+hg.
COPYRIGHT 1994-2011 W3C (MIT, ERCIM, KEIO), ALL
RIGHTS RESERVED. W3C LIABILITY, TRADEMARK,
DOCUMENT USE AND SOFTWARE LICENSING RULES APPLY.
YOUR INTERACTIONS WITH THIS SITE ARE IN ACCORDANCE WITH OUR PUBLIC AND
MEMBER PRIVACY STATEMENTS.

Web Accessibility Checker
Web Accessibility Checker
Web Page URL HTML File Upload Paste HTML Markup

Check Accessibility By:
Address:
CheckIt
Options
Login Register
Free Accessibility Test
Test for Section 508, WGAG & more
using SSB BART Group's AMP
Express
Section 508 Support
Expert evaluation and assistance Easy
and strategic
inclusive.com
Roku @ just $49
Our Lowest Price Ever. Free Shipping
Available. In Stock.
Roku.com/Free_Shipping
IDI Web Accessibility Checker : Web Accessibility Checker http://achecker.ca/checker/index.php
1 of 3 2/23/2012 7:37 AM
Known Problems(27) Likely Problems (0) Potential Problems (100) HTML Validation CSS Validation
Accessibility Review
Accessibility Review (Guidelines: WCAG 2.0 (Level AA))
1.1 Text Alternatives: Provide text alternatives for any non-text content
Success Criteria 1.1.1 Non-text Content (A)
Check 163: embed element missing noembed element.
Repair: Add a noembed element within or beside the embed element. Add text to the noembed element that is equivalent to the embed
element.
Line 140, Column 21:
<embed sr c=" i mages/ spl ash_i mage. swf " qual i t y="hi gh" pl ugi nspage="ht t p: / / www. macr omedi a. com/ shockwave . . .
Check 1: img element missing alt attribute.
Repair: Add an al t attribute to your i mg element.


Line 170, Column 1:
Line 174, Column 3:
Export Format:
PDF
Report to Export:
All Get Fi le

IDI Web Accessibility Checker : Web Accessibility Checker http://achecker.ca/checker/index.php
2 of 3 2/23/2012 7:37 AM
2940

Revalidate
Markup Validation Service
Check t he markup (HTML, XHTML, ) of Web document s
Jump To: Notes and Potential Issues Validation Output
Errors found while checking this document as HTML 4.01
Transitional!
Result: 92 Errors, 23 warning(s)
Source :

Encoding : utf-8 (detect automatically)
Doctype :
HTML 4.01
Transitional
(detect automatically)
Root Element: html
The W3C validators are hosted on server technology
donated by HP, and supported by community
donations.
Donate and help us build better tools for a better web.
Options
M|.Show Source M|.Show Outline
*'^List Messages Sequentially *'Group
Error Messages by Type
M|.
Validate error pages
M|.
Verbose Output
M|.Clean up Markup with HTML-Tidy
Page 1of 38 [Invalid] Markup Validation of upload://Form Submission - W3C Markup Validator
2/23/2012 http://validator.w3.org/check
The source code for the ADP Portal Welcome Screen was uploaded to the Validator.
Help on the options is available.
Notes and Potential Issues
The following notes and warnings highlight missing or conflicting information which
caused the validator to perform some guesswork prior to validation, or other things
affecting the output below. If the guess or fallback is incorrect, it could make
validation results entirely incoherent. It is highly recommended to check these
potential issues, and, if necessary, fix them and re-validate the document.
Unable to Determine Parse Mode!
The validator can process documents either as XML (for document types such
as XHTML, SVG, etc.) or SGML (for HTML 4.01 and prior versions). For this
document, the information available was not sufficient to determine the parsing
mode unambiguously, because:
in Direct Input mode, no MIME Media Type is served to the validator
No known Document Type could be detected
No XML declaration (e.g <?xml ver si on=" 1. 0" ?>) could be
found at the beginning of the document.
No XML namespace (e.g <ht ml
xml ns=" ht t p: / / www. w3. or g/ 1999/ xht ml " xml : l ang=" en" >)
could be found at the root of the document.
As a default, the validator is falling back to SGML mode.
No DOCTYPE found! Checking with default HTML 4.01 Transitional
Document Type.
No DOCTYPE Declaration could be found or recognized in this document. This
generally means that the document is not declaring its Document Type at the
top. It can also mean that the DOCTYPE declaration contains a spelling error, or
that it is not using the correct syntax.
The document was checked using a default "fallback" Document Type Definition
that closely resembles HTML 4.01 Transitional.
Learn how to add a doctype to your document from our FAQ.
No Character encoding declared at document level
No character encoding information was found within the document, either in an
HTML met a element or an XML declaration. It is often recommended to declare
the character encoding in the document itself, especially if there is a chance that
the document will be read from or saved to disk, CD, etc.
See this tutorial on character encoding for techniques and explanations.
Using Direct Input mode: UTF-8 character encoding assumed
Unlike the by URI and by File Upload modes, the Direct Input mode of the
validator provides validated content in the form of characters pasted or typed in
the validator's form field. This will automatically make the data UTF-8, and
therefore the validator does not need to determine the character encoding of
your document, and will ignore any charset information specified.
If you notice a discrepancy in detected character encoding between the Direct
Input mode and other validator modes, this is likely to be the reason. It is neither
a bug in the validator, nor in your document.
TOP
Validation Output: 92 Errors
Line 29, Column 1: no document type declaration; implying " <!DOCTYPE
HTML SYSTEM>"
< ht ml >
The checked page did not contain a document type ("DOCTYPE") declaration. The
Validator has tried to validate with a fallback DTD, but this is quite likely to be
incorrect and will generate a large number of incorrect error messages. It is highly
recommended that you insert the proper DOCTYPE declaration in your document --
instructions for doing this are given above -- and it is necessary to have this
declaration before the page can be declared to be valid.

scr i pt l anguage=" J avaScr i pt " sr c=" / st at i c/ common/ j s/ scr ol l Cont r ol . j s" > </ scr i p
"img" element.
l anguage=" J avaScr i pt " sr c=" / st at i c/ por t l et s/ por t al / t heme/ j s/ ut i l s. j s" > </ scr i p
"img" element.
" J avaScr i pt " sr c=" / st at i c/ por t l et s/ por t al / t heme/ j s/ cont ent Suppor t . j s" > </ scr i p
"img" element.
vaScr i pt " sr c=" / st at i c/ por t l et s/ por t al / t heme/ j s/ navi gat i onSuppor t . j s" > </ scr i p
"img" element.
anguage=" J avaScr i pt " sr c=" / st at i c/ por t l et s/ por t al / t heme/ j s/ domai n. j s" > </ scr i p
"img" element.
l anguage=" J avaScr i pt " sr c=" / st at i c/ por t l et s/ por t al / t heme/ j s/ adpj s. j s" > </ scr i p
"img" element.
Line 42, Column 44: there is no attribute " DJCONFIG"
scr i pt l anguage=" J avaScr i pt " dj Conf i g=" par seOnLoad: t r ue, i sDebug: f al se" sr c
=" par seOnLoad: t r ue, i sDebug: f al se" sr c=" / st at i c/ t ool s/ doj o/ doj o. j s" > </ scr i p
"img" element.
<scr i pt sr c=" / st at i c/ t ool s/ di j i t / di j i t . j s" > </ scr i pt >
"img" element.
yl esheet " t ype=" t ext / css" hr ef =" / st at i c/ t ool s/ di j i t / t hemes/ t undr a/ t undr a. css"
XHTML.
<l i nk r el =" st yl esheet " t ype=" t ext / css" hr ef =" / st at i c/ adpui / css/ ADPUI . css?XYZ"
XHTML.
Line 46, Column 82: document type does not allow element " LINK" here
<l i nk r el =" st yl esheet " t ype=" t ext / css" hr ef =" / st at i c/ adpui / css/ ADPUI . css?XYZ"
The element named above was found in a context where it is not allowed. This
could mean that you have incorrectly nested elements -- such as a "style" element
in the "body" section instead of inside "head" -- or two elements that overlap (which
is not allowed).
One common cause for this error is the use of XHTML syntax in HTML documents.
Due to HTML's rules of implicitly closed elements, this error can create cascading
effects. For instance, using XHTML's "self-closing" tags for "meta" and "link" in the
"head" section of a HTML document may cause the parser to infer the end of the
"head" section and the beginning of the "body" section (where "link" and "meta" are
not allowed; hence the reported error).
yl esheet " t ype=" t ext / css" hr ef =" / st at i c/ por t l et s/ por t al / t heme/ css/ por t al . css"
XHTML.
yl esheet " t ype=" t ext / css" hr ef =" / st at i c/ por t l et s/ por t al / t heme/ css/ por t al . css"
is not allowed).
=" t ext / css" hr ef =" / st at i c/ por t l et s/ por t al / t heme/ css/ l ayer edWi ndowSuppor t . css"
XHTML.
=" t ext / css" hr ef =" / st at i c/ por t l et s/ por t al / t heme/ css/ l ayer edWi ndowSuppor t . css"
is not allowed).
l esheet " t ype=" t ext / css" hr ef =" / st at i c/ por t l et s/ por t al / t heme/ css/ l ef t nav. css"
XHTML.
l esheet " t ype=" t ext / css" hr ef =" / st at i c/ por t l et s/ por t al / t heme/ css/ l ef t nav. css"
is not allowed).
xt / css" hr ef =" / st at i c/ cl i ent s/ t hemes/ geor gi a_sout her n/ t heme/ oneadpcust om. css"
XHTML.
xt / css" hr ef =" / st at i c/ cl i ent s/ t hemes/ geor gi a_sout her n/ t heme/ oneadpcust om. css"
is not allowed).
<scr i pt >
"img" element.
Line 58, Column 7: end tag for element " HEAD" which is not open
</ head >
The Validator found an end tag for the above element, but that element is not
currently open. This is often caused by a leftover end tag from an element that was
removed during editing, or by an implicitly closed element (if you have an error
related to an element being used where it is not allowed, this is almost certainly the
case). In the latter case this error will disappear as soon as you fix the original
problem.
If this error occurred in a script section of your document, you should probably read
this FAQ entry.
Line 61, Column 38: there is no attribute " MARGINWIDTH"
<body st yl e=" mar gi n: 0; " mar gi nwi dt h=" 0" mar gi nhei ght =" 0" onunl oad=" unl oad( ) ; "
Line 61, Column 55: there is no attribute " MARGINHEIGHT"
r gi n: 0; " mar gi nwi dt h=" 0" mar gi nhei ght =" 0" onunl oad=" unl oad( ) ; " cl ass=" t undr a
Line 61, Column 94: document type does not allow element " BODY" here
r gi n: 0; " mar gi nwi dt h=" 0" mar gi nhei ght =" 0" onunl oad=" unl oad( ) ; " cl ass=" t undr a
is not allowed).
Line 71, Column 34: ID " MAINLAYOUTTABLE" already defined
<t abl e cl ass=" PORTAL- LAYOUT" i d=" m ai nLayout Tabl e" cel l spaci ng=" 0" cel l paddi n
Line 63, Column 34: ID "MAINLAYOUTTABLE" first defined here
<t abl e cl ass=" PORTAL- LAYOUT" i d=" m ai nLayout Tabl e" wi dt h=" 100%" cel l spaci ng="
Line 87, Column 70: there is no attribute " VALIGN"
<t abl e cel l paddi ng=" 0" cel l spaci ng=" 0" bor der =" 0" val i gn=" t op" >
Line 98, Column 52: ID " HEADERCOLOR" already defined

Line 90, Column 55: ID "HEADERCOLOR" first defined here
Wel come, Phi l i s Wai nf
Line 104, Column 66: " NOWRAP" is not a member of a group specified
for any attribute
 
 Suppor t Cent er Company Di r ect or y <
<a i d=" h eader Col or " cl ass=" PORTAL- anchor " hr ef =' /
Line 145, Column 89: required attribute " ACTION" not specified
<f or mname=" sear chFor m" met hod=" POST" st yl e=" mar gi n: 0px; bo
"img" element.
r y' ) " onbl ur =" onbl ur _companyDi r ect or ySear ch( t hi s, ' Sear ch Company Di r ect or y' ) "
XHTML.
Line 147, Column 177: value of attribute " ALIGN" cannot be
" ABSMIDDLE" ; must be one of " TOP" , " MIDDLE" , " BOTTOM" , " LEFT" ,
" RIGHT"
al / sear ch_ar r ow. gi f " al t =" sear ch" hspace=" 2" al i gn=" absMi ddl e " bor der =" 0" / ></
The value of the attribute is defined to be one of a list of possible values but in the
document it contained something that is not allowed for that type of attribute. For
instance, the sel ect ed attribute must be either minimized as sel ect ed or
spelled out in full as sel ect ed=" sel ect ed" ; a value like sel ect ed=" t r ue"
is not allowed.
al / sear ch_ar r ow. gi f " al t =" sear ch" hspace=" 2" al i gn=" absMi ddl e" bor der =" 0" / ></
XHTML.
Line 168, Column 143: duplicate specification of attribute " STYLE"
ami l y: Ar i al ; f ont - si ze: 12px; " st yl e=" paddi ng- r i ght : 8px; f ont - f ami l y: Ar i al
You have specified an attribute more than once. Example: Using the "hei ght "
attribute twice on the same "i mg" tag.
Line 169, Column 6: element " NOBR" undefined
 <a hr ef =" #" st yl e=" col or : #000000; t ext - decor at i on: none; " oncl i ck=" chec
by:
lower-case).
by:
lower-case).
Line 179, Column 347: there is no attribute " NOWRAP"
Cont r ol " i d=" menuHi t . cur r Rol e" nowr ap=" nowr ap" onmouseout =" hi deMenu( ) ; " onmou
Line 185, Column 39: there is no attribute " BORDER"
<di v i d="menuBar " bor der =" 0" cel l paddi ng=" 0" cel l spaci ng=" 0" cl ass
Line 185, Column 55: there is no attribute " CELLPADDING"
v i d=" menuBar " bor der =" 0" cel l paddi ng=" 0" cel l spaci ng=" 0" cl ass=" PORTAL- menuB
Line 185, Column 71: there is no attribute " CELLSPACING"
or der =" 0" cel l paddi ng=" 0" cel l spaci ng=" 0" cl ass=" PORTAL- menuBar " st yl e=" backg
by:
lower-case).
by:
lower-case).
by:
lower-case).
by:
lower-case).
by:
lower-case).
by:
lower-case).
by:
lower-case).
Line 241, Column 5: end tag for " DIV" omitted, but its declaration does
not permit this
</ t d >
You forgot to close a tag, or
you used something inside this tag that was not allowed, and the validator is
complaining that the tag should be closed before such content can be allowed.
The next message, "st ar t t ag was her e" points to the particular instance of
the tag in question); the positional indicator points to where the validator expected
you to close the tag.
Line 76, Column 5: start tag was here
< di v i d=" mai nHeader Di v" cl ass=" PORTAL- header " >
<scr i pt > var cl i ent Typ = " 1" ; </ scr i pt ><scr i pt > document . get El ement ByI d( " suppo
"img" element.
var cl i ent Typ = " 1"; </ scr i pt ><scr i pt > document . get El ement ByI d( " suppor t _l i nk
"img" element.
<scr i pt >
"img" element.
<scr i pt > changeMenuBar St at e( ' menuHi t . node2' ) ; </ scr i pt >
"img" element.
Line 320, Column 14: there is no attribute " IS"
<wps: bi di i s=" r t l " >
Line 320, Column 19: element " WPS:BIDI" undefined
by:
lower-case).
 
XHTML.
st yl esheet " t ype=" t ext / css" hr ef =" / st at i c/ por t l et s/ cms/ css/ cmsLi nks. css?00001
is not allowed).
<scr i pt > document . domai n=' adp. com' ;
"img" element.
<scr i pt >
"img" element.
<scr i pt >
"img" element.

by:
lower-case).
 
XHTML.
is not allowed).
"img" element.
<scr i pt >
"img" element.
<scr i pt >
"img" element.

by:
lower-case).
 
XHTML.
is not allowed).
"img" element.
<scr i pt >
"img" element.
<scr i pt >
"img" element.
<scr i pt >
"img" element.

<scr i pt >
"img" element.
cl oseDi al og' st yl e=' di spl ay: none' sr c=' / st at i c/ common/ i mages/ cl ose_but t on. gi f
"img" element.
Line 634, Column 27: there is no attribute " WIDHT"
<t abl e bor der =' 0' wi dht =' 100%' hei ght =' 100%' >
<t abl e bor der =' 0' wi dht =' 100%' hei ght =' 100%' >
Line 645, Column 69: document type does not allow element " TABLE"
here; missing one of " APPLET" , " OBJECT" , " MAP" , " IFRAME" , " BUTTON"
start-tag
<t abl e cel l spaci ng=" 0" cel l paddi ng=" 0" s
The mentioned element is not allowed to appear in the context in which you've
placed it; the other mentioned elements are the only ones that are both allowed
there and can contain the element mentioned. This might mean that you need a
containing element, or possibly that you've forgotten to close a previous element.
One possible cause for this message is that you have attempted to put a block-level
element (such as "" or "<table>") inside an inline element (such as "<a>",
"", or "").
<t body><t r ><t d cl ass=" ADPUI - act i veBut t on
XHTML.
<t d cl ass=" ADPUI - act i veBut t onsRi ght " / >
XHTML.
Line 653, Column 69: document type does not allow element " TABLE"
here; missing one of " APPLET" , " OBJECT" , " MAP" , " IFRAME" , " BUTTON"
start-tag
<t abl e cel l spaci ng=" 0" cel l paddi ng=" 0" s
The mentioned element is not allowed to appear in the context in which you've
placed it; the other mentioned elements are the only ones that are both allowed
there and can contain the element mentioned. This might mean that you need a
containing element, or possibly that you've forgotten to close a previous element.
One possible cause for this message is that you have attempted to put a block-level
element (such as "" or "<table>") inside an inline element (such as "<a>",
"", or "").
<t body><t r ><t d cl ass=" ADPUI - act i veBut t on
XHTML.
<t d cl ass=" ADPUI - act i veBut t onsRi ght " / >
XHTML.
by:
lower-case).
 
XHTML.

is not allowed).
"img" element.
<scr i pt >
"img" element.
by:
lower-case).
 
XHTML.
is not allowed).
"img" element.
<scr i pt >
"img" element.
Line 857, Column 110: there is no attribute " ALT"
t t p: / / www. adp. com/ pr i vacy. ht ml ' ) " al t =" Cl i ck her e f or t he ADP Web Si t e Pr i vac
<scr i pt >
"img" element.
" / st at i c/ adpui / i mages/ bul l et _gr ey. gi f " / > Cl i ck </ scr i p
"img" element.
TOP
Home About... News Docs Help & FAQ Feedback Contribute
This service runs the W3C Markup Validator, v1.2+hg.
COPYRIGHT 1994-2011 W3C (MIT, ERCIM, KEIO), ALL
RIGHTS RESERVED. W3C LIABILITY, TRADEMARK,
DOCUMENT USE AND SOFTWARE LICENSING RULES APPLY.
YOUR INTERACTIONS WITH THIS SITE ARE IN ACCORDANCE WITH OUR PUBLIC AND
MEMBER PRIVACY STATEMENTS.
April 23, 2012
Page 47 of 47

Appendix G- Path for logging time starting from the Welcome Screen
Documentation of steps for locating employee timecard in the ADP Portal

1. User clicks to login from https://portal.adp.com

2. Once logged in, the following page is presented to the user:

3. Notice the menu tabs available and the instructions for navigating to eTime (highlighted in green),
which do not seem to apply to any of the menu options:

4. To find the employee role the logged in user must know to click the Manager tab. Employees do
see an Employee Tab but must click Employee again as shown on the following page.

5. After clicking Employee from the Manager Tab or the Employee Tab, the use is presented with the
following screen and can now click Welcome after clicking the Time and Attendance menu:

6. After clicking Welcome the user is presented with the following screen where s/he must click one
of the available text links to access the eTime interface:

7. The following eTime interface opens in a new window when the All other employees option is
chosen. The user then must click My Timecard:

8. Finally, the users timecard is displayed:

9. The system allows the user to edit the timecard by choosing various options for vacation or sick
time; however, users must request time off through a different feature of the system that
automatically calculates the time off once the manager approves it. Because users can also input the
time off, this often results in the hours being charged twice against the employees accumulated
leave time. The system will allow 16 hours of leave time to be charged against an 8 hour work day,
without prompting the user that the leave time request is waiting on approval by the manager:

10. It takes 6 clicks from the welcome screen to get to the timecard if you know where you are going
from the start. Navigation is not self-evident and instructions are not accurate or clear, which could
easily result in the users clicking various menu items to find the path that leads to the timecard. In
addition, the system allows monthly employees to input data in the timecard where data entry
should be prohibited. Monthly employees need only access the timecard to approve it at the end of
the pay period.

Evaluation Plan FRIT 8435

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Evaluation Plan FRIT 8435

Uploaded by

Copyright:

Available Formats

Program Evaluation- Final Report 1

Program Evaluation- Final Report

Page 3of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup V...

Page 4of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup V...

Page 5of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup V...

Page 6of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup V...

Page 7of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup V...

Page 8of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup V...

Page 9of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup V...

Page 10of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup...

Page 11of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup...

Page 12of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup...

Page 13of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup...

Page 14of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup...

Page 15of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup...

Page 16of 17 [Invalid] Markup Validation of https://portal.adp.com/public/index.htm - W3C Markup...

You might also like