Copyright 2003 Information Systems Audit and Control Association. All rights reserved. www.isaca.org.

Impact of SAS No. 94 on Computer Audit Techniques

By M. Virginia Cerullo, CPA, CIA, CFE, and Michael J. Cerullo, CPA, CITP, CFE

n recent years, information technology (IT) used by firms, large and small, has become increasingly sophisticated and complex. The explosive growth in IT includes computer hardware, databases, networks, telecommunications, the Internet, extranets, electronic commerce, client/server architecture, data warehouses, integrated accounting systems software (such as enterprise resource planning software), automated reasoning systems and neural networks software. The advances in IT have significantly changed the methods firms employ to gather and report information. Thus, auditors encounter many IT environments that maintain data on electronic media rather than paper-based media. Auditors must determine how the firm uses IT systems to initiate, record, process and report transactions or other financial data.1 This understanding is necessary to plan the audit and to determine the nature, timing and extent of tests to be performed to gain a sufficient understanding of internal controls. SAS No. 94 was recently issued to provide guidance to auditors concerning the proper assessment of internal control2 activities in IT systems. The auditing standard states that computer-assisted auditing techniques (CAATs) are needed to test automated controls in certain types of IT environments. This paper revisits auditing-through-the-computer techniques, which should become more widely used with the issuance of SAS No. 94, and focuses on the test data technique, which can be applied in almost any audit to test automated programmed controls. This technique is relatively easy to apply and does not require the auditor to have a high degree of computer expertise. An extended illustration of the steps involved in applying this technique is presented.

substantive tests.3 When assessing the effectiveness of the design and operation of controls in complex IT environments, it is necessary for the auditor to test these controls. The decision to test controls is not related to the size of the firm but to the complexity of the IT environment.

Examples of Situations Requiring Testing of Controls

The following are examples of complex IT situations that require the auditor to conduct tests of controls and substantive tests to obtain sufficient evidence about financial statement assertions. They include: IT systems that significantly automate the process of initiating, recording, processing or reporting financial information, such as integrated enterprise resource planning systems Electronic data interchange and payment transfer systems that electronically transmit (paperless) orders and payments from one computer system to another Systems that provide electronic services to customers. In these situations, the IT system automatically initiates bills for the services rendered and processes the billing transactions. Automated reasoning systems (ARS) (e.g., artificial intelligence systems) that employ complex heuristical if/then rules to make decisions (for instance, an ARS system that automatically prepares journal entries for complex transactions or a neural network application that uses financial ratios as independent variables to predict bankruptcy) Computer programs containing algorithms or formulas that make complex calculations, such as automatically computing commissions, allowance for doubtful accounts, reorder points, loan reserves and pension funding calculations

SAS No. 94 and Tests of Controls

Under the auditing standards (SAS Nos. 48, 55 and 78) relevant to computer-based systems issued prior to SAS No. 94, a large percentage of auditors assessed control risk at the maximum and performed only substantive tests of account balances and classes of transactions to gather evidence about financial statement assertions. SAS No. 94 recognizes that this approach may not be viable in complex IT environments. When evidence of a firms initiation, recording and processing of transactions exists only in electronic form, the auditors ability to obtain the desired assurance only from substantive tests is significantly diminished. SAS No. 94 does not change the requirement to perform substantive tests on significant amounts, but states that it is not practical or possible to restrict detection risk to an acceptable level by performing only

Testing of Controls
In the above situations, the auditor should identify control activitiespolicies and proceduresin place to prevent or detect material misstatements in specific financial statement assertions. Two major categories of control activities related to information processing are general controls and application controls. General controls concern all computer activities and include controls over systems development, access security, program change, data center and networks, and maintenance. Application controls relate to specific tasks performed by individual applications. They include checks performed by IT, such as editorial checks of input data and checks performed by individuals, including the manual follow-up of reconciliations and exception reports.

INFORMATION SYSTEMS CONTROL JOURNAL, VOLUME 1, 2003

Tests of controls consist of gathering evidential matter concerning how effectively and consistently the current control procedures function. These tests include inquiries, inspection of documents or electronic files, observation of the application of the control and reprocessing transactions. In designing tests of automated controls, the auditor should consider the need to obtain evidence supporting the effective operation of controls directly and indirectly related to the assertions. The techniques used to test automated controls may differ from the techniques used to test manual controls.4 Audit techniques to test automated controls are discussed below.

Auditing With the Computer

The auditing with the computer approach embraces a variety of techniques and often is referred to as computer-assisted audit techniques (CAATs). CAATs involve using computers, often a microcomputer, to aid auditors. Although the utilization of CAATs has radically improved the capabilities and effectiveness of auditors, they are primarily used to perform substantive tests. One widely used CAAT, known as general audit software (GAS), is frequently employed to perform substantive tests and may be used for limited testing of controls. For example, GAS can be used to test the functioning of complex algorithms in computer programs, but it requires extensive experience in using the software. In contrast, the auditing through the computer techniques are designed specifically to test automated controls, and some techniques do not require extensive IT experience.

Computer-assisted Audit Techniques

The auditor may use three broad categories of computerassisted techniques to test controls: Auditing around the computer Auditing with the computer Auditing through the computer

Auditing Through the Computer

These techniques focus on testing automated processing steps, programming logic, edit routines and programmed controls. The approach assumes that, if the processing programs are soundly developed and incorporate adequate edit routines and programmed checks, then errors and irregularities are not likely to slip by undetected. If these programs are functioning as designed, the outputs can reasonably be accepted as reliable. The auditing through the computer approach is particularly appropriate for testing controls in the complex IT systems emphasized in SAS No. 94. This approach embraces a family of techniques (see table 1), including test data, parallel simulation, integrated test facility and embedded audit module. In a survey conducted by the authors, only 26 of 91 responding Fortune 500 firms, or 28.6 percent, indicated that auditing through the computer techniques were used in an audit of the purchase function, usually a highly automated and complex IT application. This survey, conducted before SAS No. 94, confirms that a majority of auditors continue to set control risk at the maximum level and rely solely on substantive testing to obtain evidence about the accuracy and completeness of the relevant information. When SAS No. 94 becomes widely adopted, the number of all firms, regardless of size, using auditing through the computer techniques should increase.

Auditing Around the Computer

With this technique, auditors test the reliability of computergenerated information by first calculating expected results from the transactions entered into the system. Then, the auditors compare these calculations to the processing or output results. If they prove to be accurate and valid, it is assumed that the system of controls is effective and that the system is operating properly. The auditing around the computer approach is adequate when automated systems applications are relatively simple and straightforward. SAS No. 94 does not eliminate the use of this technique. This approach may be suitable for firms using a variety of accounting software that process applications periodically and, when the audit trail generated is extensive, allow outputs to be traced back to inputs. The major weakness of the auditing around the computer approach is that it does not determine whether the program logic is correct. In addition, this approach does not reveal how the automated controls respond to a wide variety of transactions containing errors. Therefore, in complex IT environments, this approach may overlook potentially significant errors and may be ineffective in restricting detection risk to an acceptable level.

Table 1Auditing Through the Computer Approach: A Family of Techniques

Test data technique Uses a set of hypothetical transactions to audit the programmed checks and program logic in both transaction and nontransaction processing programs. The test data approach requires only a modest investment in time to apply in practice and does not require an extensive background in information technology. Parallel simulation Attempts to simulate or duplicate the firms actual processing results. To employ this technique, the auditor writes a computer program, using an audit software package, or using packaged accounting software, such as BusinessWorks, Oracle Financials, PeopleSoft Financials, M.A.S. 90 Evolution/2 and Sap R/3. The auditors objective is to use the software to input the firms actual data for a past period and generate the same output as live production programs. The auditors simulated results and the actual processing results are compared, and differences noted, investigated and corrected. Integrated test Enables test data to be continually evaluated when transactions are processed by online systems. The auditor creates facility (ITF) fictitious situations, such as a bogus department completing purchasing requisitions or purchase orders being sent to bogus vendors, and performs a wider variety of tests compared to the test data approach. The implementation of ITF is time-consuming and costly, requiring a high-level of computer expertise. Embedded audit Is a programmed module or segment that is inserted into an application program. Its purpose is to monitor and to collect module data based on transactions, particularly those processed by online computer-based systems. The data are then used by the auditor in the tests of controls and the evaluation of control risk. The application of this method requires the auditor to have a good working knowledge of computer technology, including computer programming. INFORMATION SYSTEMS CONTROL JOURNAL, VOLUME 1, 2003

The first two techniques described in table 1 are noncontinuous audit approaches, and the last two are continuous audit approaches. Continuous audit approaches are relevant for firms employing real-time financial reporting of transaction processing applications. Noncontinuous audit techniques are relevant for firms using periodic financial reporting of transaction processing applications. Currently, most firms employ periodic financial reporting. In the future most firms will employ a mix of the two approaches. Thus, both sets of approaches are important in assessing the reliability of the internal controls and the financial reporting information. Of all the auditing through the computer techniques, the test data technique is recommended as a first choice for auditors attempting to meet the requirements of SAS No. 94. The test data technique uses a set of hypothetical transactions to audit the edit checks, programmed checks and program logic in computer programs. It is a relatively inexpensive technique to implement and requires little IT experience on the part of the auditor. This technique is powerful and easy to use in periodic financial reporting applications. Another advantage of the test data technique is that it can be employed in almost any audit to test those segments that constitute the significant risks in computer programs. The remainder of this paper presents a simple illustration of the steps involved in designing test data for a portion of a payroll application that involves calculations of sales commissions.

Figure 1 illustrates the steps in applying the test data technique for a payroll application. In the planning phase, the auditor: Obtained and studied the most recent copy of the BusinessWorks payroll documentation Determined the relevant or significant risks that could impede the achievement of the payroll cycle objectives Determined the significant or critical edit routines and programmed checks required to address the relevant risks Tests were performed for control areas considered to be vital to the overall accounting function. These are the areas that have the greatest potential for the control of material financial statement errors. These control areas were identified based upon the potential size or frequency of erroneous transactions. Figure 1The Test Data Technique for a Payroll Application
1
A Obtain payroll documentation

11

2
Evaluate and analyze exceptions

Evaluate payroll programs to be tested

Determine conditions to be tested Simulated payroll transactions Key Payroll computer processing

12

4
Prepare payroll simulated transactions

Test Data Illustration

The firm in this illustration uses an integrated BusinessWorks ERP accounting software package to automate a variety of accounting applications. All software modules are installed on a server computer. This software package is more sophisticated and complex than the previous software package used by the firm. BusinessWorks can transfer transaction totals automatically to the general ledger and initiate, record and process journal entries and recurring adjustments to the financial statements in the general ledger. The following BusinessWorks modules, or cycles, have been implemented: General ledger and financial reporting, including relevant special journals Accounts receivable Accounts payable Order entry Billing and invoicing Inventory control Payroll Job cost Considering SAS No. 94 requirements, the audit manager decided that it was critical to determine if significant internal controlsedit routines and programmed checkshad been incorporated into the BusinessWorks software package sufficiently to address the relevant risks associated with initiating, recording and processing journal entries. This illustration is limited to applying the test data technique to selected controls in the payroll application. Before beginning, the auditor must first understand the major objectives of the payroll cycle subsystems.

Written recommendations

7 8

Payroll program to be tested

13

Auditors manual pre-computed results from payroll test data

Auditors summary results from payroll test data

To client

10

Exception report

The illustration of test data design is limited to testing the program logic in calculating sales commissions. After studying the record layouts and the conditions to be tested, the fourth step shown in figure 1 is to prepare a collection of test transactions. Therefore, the auditor developed simulated test transactions for a past payroll period. The auditor used decision tables to aid the design of the test data. These tables show, in a matrix format, all the rules pertaining to a processing transaction or decision situation.

INFORMATION SYSTEMS CONTROL JOURNAL, VOLUME 1, 2003

A decision table for sales commissions is shown in table 2. Sales commissions expense is considered a material item, therefore the auditor developed test data to test controls over the calculation of sales commissions. A decision table is constructed to aid in developing the test data using the following steps: List all the conditions that apply to the calculation of sales commission. Place those conditions in the condition stub of the decision table. Construct the rules for the decision table by constructing all combinations of condition values. If there are n conditions in the decision table, each of which can take on a yes or no value, the number of conditions will be 2n. Show which actions will be taken under which combination of condition values. The above rules were used to construct the decision table in table 2. In this example, there are three possible conditions stated in the condition stub and, therefore, there are eight rules. The correct actions are shown in the action stub. For instance, rule 1 is If sales are less than US $500, then the salary is equal to the base plus 5 percent of sales. Test data are designed by choosing at least one test transaction for each rule of the decision table. Table 3 shows test data results for the accurate calculation of commissions under rules 1, 5 and 8. Table 2Decision Logic Table for Commissions Rules
1 Condition Stub Sales < 500 Sales > 500 < 1000 Sales > 1000 Action Stub Salary = base + 5% sales > 0 base + 25 + 10% sales > 500 base + 75 + 15% sales > 10 Error Y N N 2 Y Y N 3 Y Y Y 4 N Y Y 5 N N Y 6 N N N 7 Y N Y 8 N Y N

Table 3Example of Results for Accuracy of Commission Computations

Test Purpose Test for accurate calculation of sales commissions with sales less than US $500 Low boundary test for accurate calculation of sales commissions with sales between US $500 and US $1,000 High boundary test for accurate calculation of sales commissions with sales between US $500 and US $1,000 Test for accurate calculation of sales commissions greater than US $1,000 Test Description Enter $499.99 in sales field Expected Results Commission of $25 Actual Output Results Pass Reference Commis- Y (Omitted sion of for this $25 illustration)

Enter $500.01 in sales field

Commis- Commis- Y sion of sion of $25 $25

Enter $999.99 in sales field

Commis- Commis- Y sion of sion of $75 $75

Enter $1,500 in sales field

Commis- Commis- Y sion of sion of $150 $150

X X X X X X X X

The fifth step in figure 1 is for the auditor to manually precompute the expected results. Table 3 shows the test purpose, test description and expected results for four test data. The sixth step, which creates the simulated payroll transactions, is to enter the test transactions using a PC. The auditor must ascertain that the program used during testing is the actual production program used during normal processing. A convenient way of obtaining this assurance is to arrive unannounced at the processing site during the scheduled time for processing. When the processing is completed, the auditor then requests the operator either to process the test transactions before removing the program or to download them to a laptop.

After processing the test transactions, the auditor evaluates the critical control strengths and weaknesses that existed in the pay programs. For the test data illustrated in table 3 (i.e., rules 1, 5 and 8), the precomputed results and the actual results are equal, indicating no error in program logic. In a similar manner, all the test data are designed to test the operation of the internal controls implemented into the payroll module and to determine how the pay programs processed data (i.e., program logic). The simulated payroll transactions and payroll programs are processed to generate the auditors summary results, which are printed on a summary report (e.g., a weekly payroll register). The eighth step depicted in figure 1 is to compare the payroll register with the auditors manually computed results. The ninth step is to prepare an exception report listing detected errors. The final steps are to analyze and evaluate the exceptions, and write a letter of reportable conditions to the board of directors covering deficiencies in internal controls.

INFORMATION SYSTEMS CONTROL JOURNAL, VOLUME 1, 2003

Conclusion
IT, which is becoming ever more complex and sophisticated, is revolutionizing businesses. A larger percentage of firms, large and small, rely on IT to initiate, record, process and report financial data. Audit techniques must take into account the impact of this reliance in a financial statement audit, or in an audit of the internal control structure. Prior to the issuance of SAS No. 94, many financial audits of IT systems bypassed testing of controls. In these situations, the auditor often assessed control risk at a maximum level and performed only substantive tests to gather evidence about managements financial statement assertions. SAS No. 94 provides specific guidance when a significant amount of financial information supporting one or more financial statement assertions is automated by complex electronic IT. In these situations, the auditor must assess control risk by performing tests of controls, regardless of firm size. Auditing through the computer techniques, such as test data, parallel simulation or embedded audit module, should be used to test controls when a firm has sophisticated IT systems. The test data technique is recommended for auditors with little IT experience.

M. Virginia Cerullo, CPA, CIA, CFE is a professor of accounting at Southwest Missouri State University, Springfield, Missouri, USA. She is the coordinator of the Institute of Internal Auditors Endorsed Internal Audit Program at Southwest Missouri State. She received her doctorate from Louisiana State University. She has published about 40 articles in professional and academic journals. Michael J. Cerullo, CPA, CITP, CFE is a professor of accounting at Southwest Missouri State University. He specializes in teaching accounting information systems and information systems auditing. He has published about 150 articles in professional and academic journals. He received his doctorate from Louisiana State University.

Endnotes
Statement on Auditing Standards No. 94, The Effect of Information Technology on the Auditors Consideration of Internal Control in a Financial Statement Audit, AICPA, New York, USA, May 2001. (Amends Statement on Auditing Standards No. 55, Consideration of Internal Control in A Financial Statement Audit, AICPA, New York, USA, April 1988.) SAS No. 94 is effective for audits of financial statements beginning on or after 1 June 2001, although earlier implementation is allowed. 2 In this paper, internal control and the internal control structure will be used interchangeably 3 SAS No. 94, paragraph No. 66 4 Ibid, paragraphs No. 77 and 79
1

Information Systems Control Journal, formerly the IS Audit & Control Journal, is published by the Information Systems Audit and Control Association, Inc.. Membership in the association, a voluntary organization of persons interested in information systems (IS) auditing, control and security, entitles one to receive an annual subscription to the Information Systems Control Journal. Opinions expressed in the Information Systems Control Journal represent the views of the authors and advertisers. They may differ from policies and official statements of the Information Systems Audit and Control Association and/or the IT Governance Institute and their committees, and from opinions endorsed by authors' employers, or the editors of this Journal. Information Systems Control Journal does not attest to the originality of authors' content. Copyright 2003 by Information Systems Audit and Control Association Inc., formerly the EDP Auditors Association. All rights reserved. ISCATM Information Systems Control AssociationTM Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, Mass. 01970, to photocopy articles owned by the Information Systems Audit and Control Association Inc., for a flat fee of US $2.50 per article plus 25 per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited. www.isaca.org

INFORMATION SYSTEMS CONTROL JOURNAL, VOLUME 1, 2003