Internal Auditor 8th July 2011

Print Close

Attribute Sampling Plans

A simple statistical application may dramatically improve the reliability of internal control testing.
DENNIS APPLEGATE

A reliability assessment of the organization's internal control system involves deciding how much evidence to gather. Because an examination of all underlying control data is not always feasible, auditors must often draw samples, audit the items selected, and extrapolate the results to the larger population. Either a statistical or nonstatistical approach to sampling is acceptable under The IIA's International Standards for the Professional Practice of Internal Auditing and The American Institute of Certified Public Accountants' (AICPA's) Professional Auditing Standards. The use of statistics, however, will help auditors develop sample plans more efficiently and assess sample results more objectively than nonstatistical methods alone. Even a well-designed nonstatistical sample cannot measure the risk that the sample is not representative of the population - a distinct advantage of statistically based sampling plans. Moreover, increased regulatory requirements to provide greater assurance over internal accounting controls and company demands for greater productivity from their audit shops make statistical sampling a necessary part of the internal auditor's tool kit. Fortunately, auditors can use statistical sampling techniques without any detailed knowledge of classical statistical theory and still accomplish their audit objectives.

ATTRIBUTE SAMPLING
Attribute sampling plans represent the most common statistical application used by internal auditors to test the effectiveness of controls and determine the rate of compliance with established criteria. The results of these plans provide a statistical basis for the auditor to conclude whether the controls are functioning as intended, reflecting either control compliance or noncompliance - a binary (yes/no) proposition.

In developing an attribute sampling plan, the auditor must first define the audit test objective, population involved, sampling unit, and control items to be tested. For example, if the auditor's objective is to determine the percentage of sales orders lacking credit approval, the population will consist of all sales orders within a given period. Each sales order becomes the sampling unit, and sales order credit approval represents the control attribute to be tested.

STATISTICAL CRITERIA
The auditor must consider four statistical parameters to determine an appropriate sample size to select for the planned control test: confidence level, expected deviation rate, tolerable rate, and population. Although guided by assessed risk, inquiries of the audit client, and prior audit experience, each parameter is ultimately based on professional auditor judgment. Confidence Level The sample's confidence level refers to the reliability the auditor places on the sample results. Confidence levels of 90 percent to 99 percent are common. A 95 percent confidence level means the auditor assumes the risk that five out of 100 samples will not reflect the true values in the population. The auditor's assessment of the control environment contributes to the level of risk the auditor is willing to assume. At a 95 percent confidence level, 5 percent the complement of the confidence level reflects the auditor's risk of "assessing control risk too low." Expected Deviation Rate The expected deviation rate represents the auditor's best estimate of the actual failure rate of a control in a population. The rate usually is based on client inquiries, changes in personnel, process observations, prior year test results, or even the results of a preliminary sample. Tolerable Rate The tolerable rate defines the maximum rate of noncompliance the internal auditor will "tolerate" and still rely on the prescribed control. Many auditors will coordinate with their audit client before establishing a tolerable level. Client control objectives help determine

the nature and frequency of deviations that can occur and still allow reliance on the control. Population The population contains all items to be considered for testing. Each must have an unbiased chance of selection to ensure the final sample is representative of the population. For large populations containing thousands of items, population size will cause little impact on total sample size and is often irrelevant for audit sample planning.

APPLICATION OF THE METHODOLOGY

In a test of sales orders for appropriate credit approval, suppose the auditor estimates a 1.5 percent expected deviation rate of missing credit approvals relative to total sales orders, establishes a tolerable rate of 6 percent, and accepts a 95 percent confidence level that the sample results will reflect missing credit approvals fairly in the population. To calculate sample size, the auditor could use a variety of tools and techniques, including manual computations, statistical tables, and commercial software packages. For the statistical parameters provided, a sample size of 103 sales orders would be needed based on the "Statistical Sample Sizes for Test of Controls" chart below. Each of the sales orders selected for audit must be randomly drawn to prevent bias in the sample results. Simple random sampling, such as choosing sales orders based on a random-number table, is the most common selection technique. Systematic selection picking every nth sales order - is also acceptable if the first item sampled is randomly selected, though the results may be skewed if missing credit approvals occur in a systematic pattern. Because the random nature of the selection process will protect the validity of the statistical inferences, simple random sampling is normally the preferred method. After selecting a sample of sales orders, the auditor would compare the documented credit approvals against the operating procedures in place, noting exceptions and performing other audit steps as necessary in light of sales order protocols unique to the business. Special consideration should be given to data anomalies resulting from the selection process. For example, missing sales order documentation should be treated as an audit exception because the condition implies that control over credit approvals has not been applied as prescribed. Alternatively, voided sales orders should be replaced by orders that have not been voided. Mere voiding of a sales order does not alone suggest a weakness in control over credit approval.

Based on these procedures, suppose four sales orders lacked appropriate credit approval in the sample test. The auditor would project these results to the sales order population by calculating the upper deviation rate, a statistical estimate of the maximum deviation rate in the population. This rate can be determined using a simple statistical table or a manual or computer-generated computation. Based on the sample size and number of deviations found, the upper deviation rate in the sales example would be approximately 9 percent based on the "Statistical Sampling Results Evaluation Table for Tests of Controls" chart below.

AUDIT CONCLUSION
To form a statistical conclusion about the control tested, the auditor must compare the upper deviation rate to the tolerable rate in the sampling plan. If the upper deviation rate is less than the auditor's tolerable rate, the auditor would consider the control effective. Alternatively, if the upper deviation rate exceeds the auditor's tolerable rate, the auditor would consider the control ineffective. In the sales order example, the upper deviation rate(9 percent) exceeds the auditor's tolerable rate (6 percent). Therefore, the auditor would advise management not to rely on the control, concluding with 95 percent certainty that the rate of missed credit approvals exceeds the tolerable rate. All audit sampling plans use the upper deviation rate as the basis for an audit conclusion because it includes an allowance for sampling risk, which provides protection against undetected deviations. For nonstatistical sampling plans, only the sample deviation rate can form the basis for an audit conclusion - a limitation of the nonstatistical approach.

WORKPAPER OBJECTIVES
As with all audit procedures, the auditor must appropriately document the work performed. For a statistical sampling plan, the auditor's workpapers should include the essential elements, including the nature of the control tested (in the earlier example, sales order credit compliance with organizational procedure); details of the population and sampling unit (prior-year sales orders and related credit approvals); the control deviation (missing credit approvals); the statistical parameters used (including the deviation and tolerable rates); the sample size; and the evaluation of results. The auditor's documentation should also describe how the audit test steps were performed, and should provide a list of the actual deviations found (namely, in our example, the missing credit approvals).

AUDITOR JUDGMENT

Regardless of the sampling approach used, professional auditor judgment must always govern the quality of the audit evidence. Even with statistical sampling, auditors must exercise judgment in determining the appropriate statistical parameters to use for a valid audit conclusion. Nonetheless, a statistical approach to evidence gathering, such as attribute-based sampling, will normally provide a more objective basis for evaluating sample results than nonstatistical techniques and enhance the quality of auditors' reporting to management.

To comment on this article, e-mail the author at dennis.applegate@theiia.org.