It Strategic Audit Plan387

The Auditor General of Qubec
Strategic Plan
For IT Performance Audit
A three-year plan of proposed audits with a framework to guide the audit approach.
Related strategies for audits to be performed in the fields of information technology and electronic
communications in the ministries and agencies of the Qubec government.
February 2001
Prepared by:
Yves Denis
Clarence Kimpton
Denys Martin
Guy Perron
This document has been translated from French using a translating

software.
For information, clarification and queries on this report,

contact:
Mr. Denys Martin, Phone: 418-691-5935 ext. 4085
Email: dmartin7777@yahoo.com.au
THE AUDITOR GENERAL OF QUBEC
CONTENTS
Foreword ........................................................................................
Introduction .............................................................................
II
IT Environment Definition...........................................................
3
A.
3
..
III
IV.
Governmental Directions and Recent Developments............
Legal and Regulatory Framework.......................................
Roles and Responsibilities.................................................
Resources Invested.........................................................
Strategy of intervention ............................................................

8
A.
Steps Carried Out............................................................
Benchmarks And Audit Limitations .........................
10
Principles at the Basis of the Strategy................................
10
D.
Audit Universe.............................................................
11
E.
Problems Associated with IT& C .................................
12
Reference Works.............................................................
12
Audit Selected. ...........................................................
12
H.
Follow Up on Previous Audits.........................................
13
Audits Report............................................................
14
Audit Resources Requirements...........................................
14
K
15
Annual Audit Cycle and Timeline..................................
Training ........................................................
M.
16
Audit Process and Project Management ........................
Conclusion .......................................................................
16
19
Appendices
1
Qubec Policy for the Internet

To act differently: Initiatives associated with the priority " Bringing
together the State the citizen and the companies "
Plan of action " the governmental information highway For better

serving the citizens and the companies " ( Qubec)
Description of key issues covered by legislative auditors in Canada and

overseas
Comparison of the principal models of analysis for IT& C
List of Audit Subjects
Project Organisation Manual Table of Contents
Foreword
The growing strategic importance of the fields of Electronic Communications and
Information Technology (IT& C) led the Auditor General of Qubec (AGQ) to dedicate
to it of the specific work of audit in February 1997. Within the AGQ office, a
directorate, Information Systems Audit Management (DVSI) - section optimisation,
was given this task and was provided April 1998 of a first strategy with intervention
identifying the particular problems which are attached to the fields of IT& C.
Thus, the DVSI constituted and carried out various projects of optimisation of the
resources specific to the fields of IT& C and collaborated in other audit projects on IT
aspects. Work to date led to the principal following reports:
Overall Funds (aspect information system): Report 1997-1998, volume 1;
Adaptation to year 2000 of the IT systems to the government of Qubec: Report
1997-1998, volume 1;
Observations of the Auditor General (aspect adaptation to year 2000 of the IT
systems): Report 1997-1998, volume 2;
Management of the social services for young people (aspect information systems):
Report 1997-1998, volume 2;
Adaptation to year 2000 of the IT systems to the government of Qubec: Report
1998-1999, volume 1;
Acquisition of services (partnerships public/private phase): Report 1999-2000
divided into volumes;
Process of development and system maintenance of information for the Ministry of
revenue: Report 1999-2000, volume 1:
Management of electronic communication and information technology for the
Ministry for Transport: Report 1999-2000, volume 2;
ERP Project ("GIRES") Report 1999-2000, volumes 1 and 2.
Regarding audit priorities for the three next years, those were established by
selecting projects comprising of high level of importance and risk starting from a
summary list of principal projects from specialised documents, other legislative
auditors, recognised stakeholders or team-members.
This strategy is a tool for periodic analysis on the evolution of IT& C and related audit
interventions carried out, approaches used and the results obtained, in order to
ensure the relevance of our work and continuously to improve our practice. It seeks
to be also a tool to identify the types of audit interventions to be most useful and in
consideration of the priorities of the AGQ and the government of Qubec vision for
IT& C.
Strategic IT Audit Plan
Prepared by Denys Martin Email: dmartin7777@yahoo.com.au
295386150.doc
23/03/2001 04:07:00 PM
Page 2
Introduction
Fruit of convergence between IT one and telecommunications, IT& C represents a

strategic sector for the public administration, because they appear among the
principal factors of modernisation and improvement of the productivity. In Qubec,
information technology constitutes one of the priority axes of the governmental
strategy of renewal of the public service.
The Treasury Board of Qubec defines information technology as being any software,
electronic material or combination of these elements used to collect, store, process,
communicate, reproduce, protect or delete from information. The electronic
communications constitute a specialised subset of information technology supporting
the processes of transmission of remote information using electronics, radioelectrical,
and optical or electromagnetic means. Also, by extension, taking into account their
intimate links with information technology, the processes of management of
information belong to the sphere of activity of IT& C. The whole of these elements is
gathered under the name of IT Resources.
II.
IT Environment Definition
A.
Governmental Orientations and Recent Developments
In its advising in the management of IT resources, the Treasury Board, contributes to

set up a renewed framework of governmental management and ensures the coherent
national deployment in order to improve quality and the effectiveness of the public
service and to support the work of modernisation of the public administration and the
socio-economic and cultural development of Qubec.
For the financial year 2000-2001, the Treasury Board pursues, via its secretariat, the
following goals:
To ensure the co-ordination of the IT resources so as to preserve governmental
coherence;
To implement an ERP system ("GIRES" (ERP for human, material et financial
Resources));
To define required initiatives to ensure the protection of the personal and
confidential information as well as the security of information and electronic
exchanges;
To support the implementation of the Qubec Policy for internet information;
To support the interdepartmental projects of partnership aiming at simplifying,
accelerating and at facilitating services to citizens and companies.
The principal stake is to allow the harmonious deployment of the governmental
information highway, by stressing overall co-ordination and by supporting ministerial
initiatives. This information highway constitutes an economic and regional
development tool and should make it possible to profit from productivity in the
governmental administration, to ensure a good exposure for Quebec abroad, to offer
better information to citizens as well as services well adapted to their living and
working conditions, to improve education and the social development and finally, to
promote the French language and the culture qubcoise. In addition, initiatives
295386150.doc
23/03/2001 04:07:00 PM
Page 3

associated with the priority of Bringing Together the State, Citizens and Companies
and " the Qubec Policy for the Internet Acting differently are provided in
appendix 1. Moreover, the contribution of the Public administration to the
implementation of the information highway is framed more particularly by the plan of
action titled The governmental information highway For better serving the citizens
and the companies The strategy of implementation and work engaged appear in
appendix 2.
Several governmental projects have started a few months ago. Among the most
significant, "GIRES" ( Oracle ERP ), an integrated system of human, financial and
material resources, SERTIR, the server for all ministries and agencies to facilitate
electronic transactions with citizens and companies, and the telecommunications
network for social services which, by being secure, provides fast and confidential
exchanges between the stakeholders and the managers of the network, will
contribute to the evolution of initiatives such as access to a single patient file and the
issuing of a smart card for the Government Medical Insurance Board of Qubec
( RAMQ ). In fact, several projects are moving in all the governmental spheres of
activity:
Programs, services and administrative forms (Ministry for Relations with citizens
and Immigration);
Laws and payments (Publications of Qubec);
Electronic Trade (Commission of the health and the occupational safety );
Register of personal and rights (Ministry for Justice);
Municipal Information highway (Ministry for municipal affairs and the greater
Montral);
Hello Qubec.com (Qubec Tourism);
Computerised Program being used for processing of files and information retrieval
(Public records);
Multi-media Catalogue Iris (national Library of Qubec);
Multi-media Qubec (Ministry for Culture and Communications);
special Products (Ministry for natural Resources);
Products and services cadastral (Ministry for natural Resources);
Electronic Exchanges and forms electronic (Ministry for Revenue);
Linguistic Resources (Office of French language);
Certificates of birth, marriage or death by Internet (Director of Civil Status).
Legal And Regulatory Framework
Several laws, regulations, directives and decisions govern the management of

information technology to the government of Qubec. However this framework varies
according to the influence of various laws and Treasury Board instructions.
Thus all the relevant ministries, agencies and entities whose human resources comes
under the Law on Public Administration. Only some elements of the regulatory
framework and directives and decisions touch the agencies distinctly from one
organisation to another. For their part, the companies of the government are not
really affected, because they manage their resources in an autonomous way.
295386150.doc
23/03/2001 04:07:00 PM
Page 4

The principal elements which frame the field of information technology.
Initially, it is Law on Public Administration in its articles 64 to 66, which specifies the
principles and responsibilities for management relating to IT resources. These
resources must be managed in order to:
to use in an optimal way possibilities of information technology and
communications like means of management of human, budgetary and material
resources;
to contribute to the attainment of the objectives of accessibility and simplification
of the services to citizens;
to support dialogue enters the ministries and agencies and the division of their
expertise and their resources.
As regards acquisitions of property and services, Law on Government Purchasing
Services includes provisions for buying or rental services, for ministries and agencies
concerned and the required deliverables for their activities. Moreover, the Treasury
Board sets up a framework on the procurement agreements. Similar activities in
information technology must comply with the regulatory framework. Law on
governmental services in ministries and agencies regulate the specific functions and
capacities of the ministry, defines procedures for services and institutes specific
funds (printing services, air service, IT services, etc.).
The National Assembly currently considers a bill (161) concerning the legal
framework of information technology. It aims at ensuring legal security of
communications carried out by people, associations, companies or the State by
means of documents, the functional equivalence of electronic documents and their
legal authenticity.
On the regulatory plan, Management Framework for IT Resources in the Government
of Qubec (CT # 187036 of April 4, 1995) stipulates that leaders of ministries and
agencies must fully assume the management of their IT resources, in observance of
regulation and governmental orientations, in order to achieve strategic goals
selected. They must, for this purpose, ensure projects and investments relevance,
efficiency and effectiveness.
Ministries and agencies must also comply with the recent Directive on security of
digital information and electronic exchanges in governmental Administration, which
took effect on February 4, 2000. This one states the guiding principles of security of
digital information and electronic exchanges in governmental administration,
identifies the stakeholders concerned with related security management, determines
responsibilities for ministries and agencies and plans the introduction of mechanisms
of suitable co-ordination and collaboration in order to ensure availability, integrity,
confidentiality of digital information, authentication of users and irrevocability of
documents which they compile or of actions that they pose. Another directive
adopted in October 1999 relates more specifically to the processing and destruction
of any information, register, data, software, operating system or another good
protected by a royalty, stored on microprocessing equipment.
Ministries and agencies concerned must also comply with specific decisions and
technological guides of standardisation from the Treasury Board. Various other
295386150.doc
23/03/2001 04:07:00 PM
Page 5

obligations apply to the whole of the information held by ministries and agencies.
These obligations are fixed inter alia by the Law on access to documents of public
agencies and on protection of personal information and by the Law on personal
files.
Finally, since 1992, the government of Qubec has a policy of usage of French in
information technology in order to promote the use of French as a language of
design, use, diffusion and training.
295386150.doc
23/03/2001 04:07:00 PM
Page 6

C
Roles and Responsibilities
The responsibilities for the fields of IT& C in the government administration are
entrusted to ministries and agencies, the Treasury Board and ministries responsible
for particular mandates. Graph 1 presents a global view of these stakeholders.
Graph 1
Electronic Communication And Information Technology
With The Government Of Qubec
Environment Model
Ministry for
Relations with
citizens and
Immigration
Ministry for
Industry and
Trade
Regulator /
co-ordinator /
catalyst
Undersecretariat
information
highways and
IT resources
Suppliers of
services
DGSIG
DGT
Office of the
French
language
Co-ordinating committee of governmental information highway
Ministries and Agencies
The Qubec government management framework for delegates responsibility to

ministries and agencies for management of its IT resources. However, the Treasury
Board must approve investment related to information technology whose original
costs are higher than the threshold established with the regulation. Among the other
obligations, ministries and agencies must submit to the Treasury Board of Qubec,
their forecasts of appropriations for IT resources and a triennial plan detailing their
projects. In addition, four funds of information technology were made up for financing
IT projects.
The Treasury Board develop rules to ensure security of IT resources, including
protection of personal information and other confidential information. It also plans
initiatives to promote common infrastructures and services. It also manages an
incentive fund for interdepartmental partnership.
In term of governmental services, the Treasury Board offers a government wide
server. Its directorate (DGSIG) has the role of providing various IT services to
295386150.doc
23/03/2001 04:07:00 PM
Page 7

ministries and agencies of IT services on various platforms. These include computer
processing, access and connectivity, advice on standards and computerisation, data
warehousing. Its general directorate of telecommunications (DGT), for its part, offer
the following:
Telephony Services ;
Network Services;
Wireless Communications (WAP);
Information Highway Services.
Co-ordinating committee of governmental information highway
This committee is charged to ensure coherence of action and relative co-ordinate
work of key stakeholders implies in the implementation of the plan of action adopted
by the government with the constitution of an information highway for public
administration.
Ministry for Relations with Citizens and Immigration
This Ministry has the mission of supporting the recourse to new technology to give
access to direct transactions between citizens and the administration.
Ministry for Industry and Trade
This ministry proposes, with the government or Standing committee of purchases,
orientations, priorities related to the uses of the governmental purchasing power
likely to support economic and technological development.
Office of French language

The Office ensures follow up on the plan of technology focussing on promoting the
French language for each Ministry and agencies as well as the diffusion of information
on the availability of IT French products.
D.
Resources Invested
The government of Qubec spent CAN $737 million in the field of information
technology in 1995-1996, most recent year for available statistics and represents
more than two thirds of the global expenditure of the government.
295386150.doc
23/03/2001 04:07:00 PM
Page 8
Rseau de
l'ducation
14%
Rseau de la
sant
18%
Organismes
extra
budgtaires
22%
Ministres et
organismes
budgtaires
46%
Figure 1
Source : Bilan et perspectives 1995-1996, Secretariat du Conseil du trsor.
55 p. cent of 432 millions dollars spent in 1998-1999 impacted on these five

ministries:
Social Solidarity
Revenue
Natural Resources
Transport
Education
86,6
78,2
30,5
23,5
20,8
M$
M$
M$
M$
M$
In addition, expenditure in information technology agencies comes mainly from the

Commission Of Work Health And Safety, Qubec Automobile Insurance Board, the
Control of revenues of Qubec and the Control of health insurance of Qubec. In
1997-1998, budgets of expenditure in information technology of these entities were
respectively 57, 50, 32 and 20 million dollars for a total of 159 million dollars.
A more recent source, the consolidated state of the fixed assets of the government of
Qubec at March 31, 1999, mentions that the net amount of the fixed assets relating
to the IT development is 581 million dollars.
III.
Strategy of intervention
Several audit interventions are possible in IT& C. The development of this strategy of
audit intervention required a significant effort to target sectors where the Auditor
General can best support parliamentary control while carrying out the promotion of
sound practices of management. What is required above all, it is the relevance of our
interventions.
The present section describes initially the step for our strategy, benchmarks and
audits limitations, specifies the principles at the basis of the strategy such as our
audit universe. It also surveys audits carried out here and elsewhere in the fields of
IT& C and document their work. Lastly, it provides details on audits follow up,
resources required and training, timelines and project management.
295386150.doc
23/03/2001 04:07:00 PM
Page 9
A.
Steps Carried Out
Activities carried out to define this strategy:
Survey and analyses of benchmarks and audit limitations

Strategic orientations and standards of the Auditor General of Qubec
Limits of interventions in the field of IT& C
Definition of guiding principles for the strategy of intervention

Analysis of strategic orientations and standards of the AGQ
Meeting with AGQ senior staff
Analysis from experiences of previous audits
Survey the Qubec government current initiatives and concerns

Interviews with the Treasury Board of Qubec
Survey and analyses of Treasury Board of Qubec decisions as regards IT& C
Held a one day roundtable with significant stakeholders on the management of
IT& C with the government of Qubec
Analysis of documentation
Analyse audit approaches in the management of IT& C (legislative auditors of

Canada and provinces, internal auditors of various agencies and companies)
Contacts (AG Canada, Hong Kong, Auditor General of Western Australia)
Analysis of audit reports from legislative auditors
Analysis of strategic plans of audit in the field of IT& C
(ex: Hydro-Qubec, AG Saskatchewan, State of Florida)
Analyse leading IT & C Business Models

( COBIT, CMM, etc.)
Selection of criteria to support our proposals for audit projects

Analysis of criteria to support the selection of audit projects
Analysis of Risks Models
Selection and documentation of audit projects
The selection of projects was done in three steps:

1. Survey of audit subjects based on risks. Each team-member had to identify in an
individual way various audit subjects considered significant and/or risky.
Criteria of importance which were to be considered by team-members are

those appearing with the handbook of audit is:
resources used by the entity;
intrinsic importance;
economic, social and environmental incidence of the subject;
295386150.doc
23/03/2001 04:07:00 PM
Page 10
degree of sensitivity and visibility of the subject;

topicality of the subject.
As for the criteria of risks, several models of analysis of risks were analysed
and we used that knowledge to develop our own, focussing on management
processes. Four types of risks were considered:
Inherent risks of the potential audit subject of (concerns expressed by IT

audit experts, documentation and audits carried out here and elsewhere);
Risks related to strategic management
Risks related to operational management (planning and organisation,
development and implementation, operations and support, monitoring);
Risks related to the management of customers service
As a result, we then agreed to sixteen (16) potential audit subjects

2. Evaluation and classification of the most relevant audit projects. Each team
member initially rank the relevance and the risks associated with each subject. By
consensus, we then arrived at a list.
3. Proposal for a sequence of projects to be completed in the three next years by
taking account initially of the scope and the risks of the projects, and then of
various considerations and impact on the strategy.
B.
Benchmarks And Audit Limitations
The strategy of intervention is supported by several key elements. Initially the

strategic orientations of the Auditor General require that we maximise the impact of
our audit work in optimisation of resources and our recommendations. Our work
focuses on resources optimisation for activities with significant expected deficiencies.
These orientations recognise the increasing importance of IT& C and related audit
work in optimisation of resources.
These audits should be performed on an annual basis and the focus is on the field of
education, health and social services. The orientations further stipulate that our work
must encourage the government to improve its management practices, the
measurement of its performance and its Accountability. In addition, a handbook of
audit dedicated to the audit of the optimisation of the resources specifies the
methodology that must be applied for this type of audit.
C.
Principles At The Basis Of The Strategy
Our audit vision of IT& C is based on an analysis of the context and acquired
experience:
Focus on management and use of technology in governmental administration;

We focus on management and use of technology in governmental administration.
We therefore do not consider audit of administrative programs such as subsidies
to business and citizens for connecting to the Internet.
295386150.doc
23/03/2001 04:07:00 PM
Page 11
Identification of audits where an IT management expertise is required;

Various work of audit is completed in the fields of IT&C and this, as well by the
regular teams of audit as the DVSI. The present strategy makes state only
interventions where a particular expertise in audit management of IT&C is
specifically required.
Priority with government wide audits;

This type of audit supports observations reflecting an overall situation, more
useful for the members of Parliament. Moreover, these audits emphasise the
deficiencies on a comparative basis.
Audit focus on risks;

A structured approach of evaluation of the risks was tested successfully at the
time of our audits relating to the development of the information systems. As
recommended in the new strategic orientations of the Auditor General, an
evaluation of the risks is carried out for the selection of the projects.
295386150.doc
23/03/2001 04:07:00 PM
Page 12
Implementation of audits for ongoing projects;

The government is setting up various IT structures (information highway, ERP,
etc) whose design and implementation should take a few years, an approach of
audit a priori
(during the development stages) is required to ensure the
publishing of timely recommendations. This does not exclude that the approach a
after the fact.
Closer links between the constitution of audit projects and their completion;
Authorisations from senior AG executives earlier in the annual cycle.
Priority with the audits with restricted SCOPE and of short duration;
The fields of IT& C evolve quickly. Consequently the formulated observations are
likely more quickly to become obsolete. These fields cover also a broad range of
activities and the whole of the functions of management. It is thus desirable that
our audits are sufficiently targeted and that the fruit of these audits is available
quickly.
D.
Audit Universe
Entities likely to be audited
The Law on the Auditor General stipulates that the audit of the books and accounts of
the Funds consolidated of the revenue, of a public organisation and an agency of the
government include financial audits, compliance audits, and performance audits.
This law also mentions that the Auditor General can proceed to audit registers, files,
documents and accounts of agencies, associations or companies who use any
subsidy granted by a government agency.
In the light of this information, the Auditor General of Qubec is entitled to audit:
Government Agencies
Treasury Board;
Ministries;
National Assembly;
Lieutenant-governor;
Citizen Ombudsman;
Electoral Commission;
Government enterprises (with agreement with the Administration Board)
Network of services
Health and social services
Education
Demarcation with standard audits of optimisation of the resources
295386150.doc
23/03/2001 04:07:00 PM
Page 13

The regular teams of audit perform work on IT& C activities. They relate mainly to
information required to manage the programs (sufficiency, accuracy, up to date,
comparability) while focussing on some specific aspects of the general management
of IT security and information, system development technology. In terms of
authorised work, a relatively tiny portion of the budget of time is dedicated to this
field of activity.
Although they make it possible to comment on the most obvious gaps, this work does
not present a vision of the whole of the management of IT& C deployed by the
entities. It is thus difficult, based only on this work, to inform the members of
Parliament about the management of IT& C in the governmental administration.
The work of the DVSI makes it possible to obtain this vision, in particular by the
present strategy of intervention, and to cover subjects with the most impact. All the
aspects of the management of IT& C can thus be considered, which includes also the
management of information.
E.
IT& C Related Issues
Many legislative auditors carried out audits in the fields of IT& C and highlighted
several problems such as project management, security and benefits management.
The most active is certainly the US General Accounting Office (GAO). Appendix 3
gives an outline and results of their audits.
F.
Reference works
The literature proposes also various models of analysis likely to be used in the
framework of our audits. The principal ones, described summarily in Appendix 4, are
as follows:
ISACA (Information Systems Auditing Control Association) CobiT Guidelines

Management
CICA The management of IT controls
GAO Information Technology Investment Management (ITIM): With framework for
Assessing and Improving Maturity Process
CICA and AICPA Systrust ms/md Principles and criteria of reliability of the systems
SEI Model evolution of the software capacities (CMM)
ITRB (Information Technology Resources Board) Managing Systems Information:
In Practical Assessment Tool
PIM (Project Institute Management) A Guide to the Management Body of
Knowledge
NSA (National Security Agency) Systems Security Engineering (SSE) Capability
Maturity Model
GAO Information Security Management: Learning from Leading Agencies
Several of these models have a similar analytical structure. Most impressive are CMM
and CobiT. These models help evaluating activities in the fields of IT& C especially in
relation to the levels of performance and quality of the IT resources. However, the
model recommended by " Information Systems Auditing Control Association "
295386150.doc
23/03/2001 04:07:00 PM
Page 14

(CobiT) is the most complete. As a result, interventions of the AGQ in the fields of IT&
C should initially rest on CobiT, yet exploit the best of the other models.
G.
Audit Selected
Sixteen audit subjects were considered in the development of this strategy. Those
were identified using the knowledge obtained of the use of IT& C in the government
of Qubec and the audits which were carried out, of the examination of various
decisions of the Treasury Board of Qubec, the audits carried out by other legislative
auditors, exchanges with stakeholders concerned and models of analysis available.
Starting from a descriptive card of each subject (see Appendix 5), those were initially
evaluated according to their criticality (combination of the scope of an subject and
risks which is associated there) by using the criteria mentioned with the section
relating to the step used. The results are presented at the following graph. According
to the grid used, a project whose risk and scope are estimated at 10 comprises
greatest criticality. It is thus to say that all the evaluated projects obtained a note
equal or higher than 5 for the two aspects considered.
Criticit des objets de vrification
10
1
13
Risque
16
6,7,8
2,3
4,5
11,12
10
15
14
0
0
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
5
Ampleur
10
Central and ministerial management of the electronic service delivery (ESD);

Audit Subject
Audit Subject
Audit Subject
Audit Subject
Audit Subject
Audit Subject
Audit Subject
Audit Subject
Audit Subject
Audit Subject
Audit Subject
Audit Subject
Audit Subject
Audit Subject
Audit Subject
By drawing up the passing note to 6 out of 10 compared to each of two examined

dimensions, four subjects were eliminated for the moment, that is to say projects 13,
14, 15 and 16. Of the twelve remainders, we thereafter structured projects of which
we can audit over a three years period.
295386150.doc
23/03/2001 04:07:00 PM
Page 15
H.
Follow Up On Previous Audits
The new standards for follow up of the recommendations of the audits of the
optimisation of the resources aims at informing the members of Parliament of the
degree of application of recommendations from the Auditor General.
All audits must have a follow up within a maximum of three years and with a level of
high insurance. The follow up must be planned at the end of the audit of the
optimisation of the resources and this planning must be revised, if it is required, after
a parliamentary committee.
The analysis of the audits carried out during the last years by the DVSI and strategies
of follow up:
Year Of
Publicatio
n
1997-1998
(volumes 1 and
2)
Audits
Follow Up Strategy
Adaptation to year 2000
The initial audit on December 31, 1997

was the subject of two follow-ups: one
in
October
1998
regarding
the
activities of the Treasury Board of
Qubec and at December 31, 1998 for
the whole of the audit.
Development
of
the
information systems
Ministry for the Revenue
of Qubec
A follow up in two years, in April 2002.
Management of
information technology
the Ministry for transport
of Qubec
Taking into account the scope of the

reports and recommendations, not very
useful to carry out a follow up before
summer 2003, which will leave time to
the Ministry to bring the corrective
measures required.
1998-1999
(volume 1)
1999-2000
(volume 1)
1999-2000
(volume 2)
I.
Audit Report
As several proposed audits are of governmental scale, it is advisable consequently to

define a strategy. For this type of audit, we plan to survey about fifteen entities in
addition to the central agencies, which could required the drafting of several sectoral
reports in addition to the report to the National Assembly.
Taking into account previous work, we plan to produce only one report whose
observations and recommendations will be customised to obtain relevant responses
from each audited entity.
J.
Audit Resources Requirements
295386150.doc
23/03/2001 04:07:00 PM
Page 16
Various elements must be considered to determine the resources required to the

audits:
Overall workload In this respect, the present strategy presents a first selection of
12 projects at added value, which could be reviewed at the time of subsequent
years. One also should not neglect work of follow up and up to date setting of the
strategy of intervention.
The frequency of publication to the annual Report In corollary with the guiding
principle consisting in carrying out audits of short duration derives frequent
publications. Two reports will be published annually, which means that the human
resource must be adjusted consequently.
Critical Mass The implementation of audits in as specialised and wide fields as
those of IT& C requires the constitution and the maintenance of a team having
various expertise and experiences to ensure required synergy.
The stability of the existing team The effort of work was evaluated according to
the team in place which has an unquestionable experience as well in audit as into
IT. Its basic elements to date carried out successfully various audits. Workload
must be compatible with the resources available.
Taking into account these elements, five resources are required to support the
workload with an adequate framework (planning, support with the drafting of the
reports, quality assurance, and accountability).
K.
Annual Audit Cycle and Timeline
Various activities must be carried out annually to correctly assume our responsibilities
with regard to the audit for the fields for IT& C in the government for Qubec.
Audit of critical subjects identified with the strategy of intervention
Deployment of approximately 3 000 hours over seven months are planned for each
audit, which represents an approximate cost of 210 000 dollars. This estimate is
based on projects where questionnaires will be used each time that it will be possible
to do it and use of previous experience in the domain. Obviously, this estimate will be
revised after the completion of each audit.
Audit Time will be distributed as follows:
Audit Phase
Workload
Distribution
Preliminary analysis
25 %
Preliminary Report of analysis including its validation near the
5%
audited entities
Detailed examination including the validation of the
45 %
observations near the audited entities
Audit report
20 %
Parliamentary committee and preparation of the follow up
5%
Audit Follow Up
295386150.doc
23/03/2001 04:07:00 PM
Page 17

As mentioned previously, a follow up of the audits must be carried out within three
year. During three next years, two follow-ups adding up 2 800 hours will be carried
out (Revenue Ministry and Transport Ministry ).
Updating The Strategy Of Intervention
The present strategy will be updated annually. 450 hours is planned each year for this
exercise. The following table presents the timeline of implementation with five
resources. Five audits could be carried out in the long term from now to December
31 2003 in addition to the follow-ups and annual up to date of the strategy of audit.
L.
Training
This strategic plan has a critical impact on training audit resources. Knowing the next
audit mandates, it will be thus much easier to synchronise the work of training with
the needs for the audits. These specific needs will be listed inside the regular process
of development of the human resources.
M.
Audit Process and Project Management
The principles at the base of this strategy require certain adjustments with the audit
process and the mechanisms of project management. Without compromising the
quality neither of the audits nor to derogate from the code of practice, it is
recommended to reduce certain stages of work such as the process of authorisation
of the projects.
The implementation of mandates of governmental scale within rather short times
requires a very tight management of project. In this context, the following changes
are proposed:
Develop, as well with the stage of the preliminary analysis as of the detailed
examination, a Project Organisation Manual detailing the required work, the
timelines and deliverables such as the mechanisms of management of the
contents, management of the changes and validation of the deliverables (see
standard table of content to appendix 6). This Project Organisation Manual is
to be approved by the principal director.
Systematic Use of a group of review of the contents of the deliverables
composed members of the DVSI or others to discuss and optimise the
strategies, work and observations.
Introduction of a mechanism of quality assurance for formal approvals on
specific deliverables.
Formalisation of follow up using monthly reports of project (work, timeline,
implementations of the period in progress and those to come) and, if need be,
of reports of progress (variations compared to the strategies planned and the
anticipated observations, solutions suggested).
The following table presents the interventions suggested of quality assurance and
peer review and of validation with the audited entity, which aim at ensuring the
quality of work and the results. It is suggested these mechanisms of quality form
an integral part of the framework of project management for the DVSI.
295386150.doc
23/03/2001 04:07:00 PM
Page 18
295386150.doc
23/03/2001 04:07:00 PM
Page 19

TIMELINE OF IMPLEMENTATION OF THE PROJECTS OF AUDIT (5 resources)
2000-2001
2001-2002
2002-2003
2003-2004
108 120 132 126 138 126 72 108 120 138 132 96 108 120 126 132 138 120 78 102 126 138 126 102 108 120 126 132 132 126 78 102 132 138 120 108
01 02 03 04 05 06 07 08 09 10 11 12 01 02 03 04 05 06 07 08 09 10 11 12 01 02 03 04 05 06 07 08 09 10 11 12
H
M
A NUMBER OF RESOURCES
1
2
3
4
5
6
7
8
9
10
3,7 3,5 4,0 4,5 4,0

0,3 0,2
1,0 4,0 4,0 5,0 4,0 2,0 3,0 2,7
0,5 1,0 0,5
1,0 1,0
0,5 1,0 0,5
2,0 2,0
0,3
1
2
3
4
5
6
7
8
9
10
Earl
y
444 462 504 621 504

32 34
126 288 432 600 552 264 288 400
66 126 69
138 132
63 132 69
264 192
36
0,2 0,3
1,0 1,0
2,0 2,0
3,0 5,0 3,0

1,0 4,8 4,5 4,0 4,5 2,0
0,5 1,0 0,5
1,0 1,0
2,0 2,0
0,2 0,3
4,5 4,0 4,5 1,0
2,0 5,0 3,6 1,4 3,0 5,0 5,0
0,2 0,3
4,0 5,0 5,0 5,0 3,8 1,7 3,0
HOURS
28 38
138 126
252 204
360 390 306

108 576 567 528 621 240
63 132 66
138 120
240 216
28 38
567 528 594 126
204 630 497 176 306 540 600
28 36
504 390 510 660 524 204 324
480 528 630 690 630 288 432 600 690 660 480 540 610 630 660 690 600 390 510 630 691 630 510 540 600 630 660 660 630 390 510 660 690 600 540
Activity
#
Project
2000-2001 2001-2002 2002-2003 2003-2004
TOTAL
1
2
7
4
3
5
1 Central and ministerial management of the electronic service delivery (ESD)
TOTAL
2535
846
261
36
3678
66
2104
534
456
1056
2640
204
7060
66
525
456
66
1815
2749
1404
7081
258
456
64
1712
2490
2601
3016
1578
1368
36
1056
2706
1815
3017
3116
20309
NOTE: The black squares indicate the date of publication to the annual Report. 1: See the list of the projects on page 19.
23/03/2001 04:07:00 PM
Page 20
Deliverable
AGQ
Peer
Revie
w
Project Organisation Manual (POM) preliminary
Validation
with the
audited
entity
analysis phase
POM for detailed examination phase
Outline of the audited field
Roles and responsibilities
Regulatory framework
X
X
Audit Project of (objectives and criteria)
Expected deficiencies and significant observations
Strategy of the detailed examination
Timeline of the detailed examination
Resources required for the detailed examination
Strategy of Report
Deliverables for the detailed examination
Final Report
Evaluation of the project of audit
In addition, it is required also to constitute a permanent Consultative Committee in

order to always ensure the relevance of our interventions. This one will have to
validate the present strategy of intervention and its annual up to date settings just as
certain deliverables of the projects of audit the such Report of the preliminary
analysis, the strategy of audit and the various produced reports.
It is suggested the following flow chart, presenting the functional links recommended
at the time of the implementation of a project of audit, is retained. This structure
supports the efficiency of the activities and the accountability of the stakeholders
concerned.
Assistant Auditor
General
Principal director
And director
of audit
Audit
Project Leader
Project Review
Group
Teammember
Consultative
Committee
Teammember
23/03/2001 04:07:00 PM
Page 21
IV.
Conclusion
The present analysis allowed the development of a strategy of intervention, which it

would be advisable to retain with regard to the fields of IT& C for the government of
Qubec. This analysis shows that the resources devoted to this sector are
considerable and that they take an increasingly significant place in the control of the
governmental activities. The Auditor General has thus all the reasons to be interested
in the audit of IT activities in government. On this subject, the strategy of
intervention suggested and the topics suggested will be able to guide our audit work.
The present strategy will be updated annually in the light of the results of next work
of the DVSI and the development of the general expertise on the AGQ in the fields of
IT& C .
23/03/2001 04:07:00 PM
Page 22
23/03/2001 04:07:00 PM
Page 23
Appendix 1
POLICY QUBEC OF THE INTERNET OF INFORMATION - TO ACT DIFFERENTLY

INITIATIVES ASSOCIATED WITH THE PRIORITY " TO BRING THE STATE THE CITIZEN
AND THE COMPANIES "
Initiatives
Persons in charge
To define architecture and the means in order to make sure

that the ministries and agencies make available on the
information highway all the general information considered
as being from public interest that they produce and hold.
Ministry for the Relations with the citizens and

Immigration
The Ministry for Relations with the citizens and Immigration,

to set up the governmental Repertory qubcois in order to
allow the citizens and the companies to have access to the
description of the services offered to the population, the
references concerning the governmental documents like to
the co-ordinates of the employees of the State; the
electronic repertory will be accessible in Internet network.
Treasury Board of Qubec
To make the provisions required so that the citizens and the

companies can communicate directly, by electronic way,
with the employees of the State charged to provide them
information and services.
Ministries and government agencies
To develop, from here June 1998, required applications to

allow the ministries and the government agencies to comply
with governmental engagement to make available in
Internet the forms administrative most frequently used by
the citizens and the companies.
Ministry for the Relations with the citizens and

Immigration
To be appropriate with the ministries and the agencies of

initiatives of adaptation of the human resources of the public
office to the changes rising from the setting in place of the
governmental information highway, in particular by making
this adaptation one of the priorities as regards development
of the human resources for the next years.
To ensure a coherence of action, to co-ordinate the

implementation of the information highway in the public
sector. This function results in particular in the
responsibility:
to assume the presidency and the secretariat of the Coordinating committee of the governmental information
highway, formed of the administrators concerned;
to conceive and make evolve/move a vision of the

governmental information highway and to propose the
means to make it divide by the whole of the
Administration;
to exert a regular and rigorous follow up governmental

plan of action and to ensure the annual up to date
setting of it;
to take care to ensure technological coherence required

to the division of the infrastructures and the services ;
to report to the government annually.
To carry out an evaluation of the mechanisms of the current

process of selection of the partners and to draw up an
assessment of their application; to specify expectations of
the government in the application of the principles of equity
and transparency like in the implementation of the concept
of division of risks and benefit.
To establish strategies aiming at early tracking and the
follow up of the public and parapublic markets strategic.
Moreover, to take care to promote the introduction of
technological innovations into the public and parapublic
markets and their use by way of technological window.
Treasury Board of Qubec and Ministry for Industry,

the Trade, Science and technology
23/03/2001 04:07:00 PM
Page 1
Appendix 1
To constitute funds, for one two years period, dedicated to
the setting in place of services common related to the
deployment of the information highway in the public sector
like to the incentive on the organisational partnerships in the
service of public services.
Ministry for Health and the Social services
To continue the steps having the aim of equipping, within

two years, the sector sociosanitaire of an Intranet allowing
to benefit from the many possibilities of the information
highway.
To take care that are made available on the information

highway from the services of information intended to
increase the autonomy of the citizens in the prevention of
the diseases and the protection of their health and their
wellbeing.
To develop a policy and to define the methods of

deployment of services of telemedecine, telediagnosis and
remote monitoring.
To gradually establish a chart health with microprocessor to

replace the current chart of health insurance.
Medical Health Insurance Board of Qubec
To continue the steps in order to establish applications of

support for the practice in the various mediums of
intervention such as the maintenance for residence, the
urgencies, the lodging of the old people and the protection
of youth.
To define, in collaboration with the Control of the health
insurance of Qubec, the orientations with regard to a
network of information sociosanitaire protected with an aim
of supporting these needs for information.
23/03/2001 04:07:00 PM
Page 2
Appendix 2
THE
PLAN OF ACTION
GOVERNMENTAL INFORMATION HIGHWAY FOR BETTER SERVING THE CITIZENS
AND THE COMPANIES
Strategy of implementation
To ensure an effective co-ordination and leadership;

Focus on partnerships;
To quickly set up the basic infrastructures and common services;
To simplify the processes and to increase coherence in the service of the services to the citizens and the companies;
Focus on human resources for better managing the change;
To exploit all the potential of the existing modes of financing.
Plan of action
Field " human Resources "
Analyses of impact on work, the development of competencies, management of the changes, information, the involvement
and the training.
Field " Process "
It is a question of establishing new shared horizontal applications and reengineering of systems of management to
integrate the management of the material, financial and human resources (project "GIRES" (ERP), to develop a catalogue of
public purchases interns electronic goods and services (CAPE), forms, and systems of transactions and payments electronic.
Field " administrative Framework "
Frameworks of management of the IT resources, the human resources, security and architectures, legal framework and
strategic planning, of the tools and the services of day before allowing to analyse the most significant innovations
developed in the world have regard with the deployment of information technology in the public administration and the
documentary engineering.
Field " Information "
Access to information and services of the government by the means of Web site, of the governmental Intranet, banks and
catalogues of data like the governmental repertory, the diffusion of information, the development of Web sites; it will be a
question of bringing up to date on the one hand, the concept of State network by developing single windows for services
intended for targeted customers by branch of industry and on the other hand, the concept of " government on line " using
forms on line and of direct transactions on line.
Field " common Infrastructures and services "
They are the grid systems of the data, images, sounds, voice, such as the RICIB (integrated Network of IT and office
automation communications), the network sociosanitaire (the RTSS of the network of health and the social services), the
infrastructure with public keys, counters multiservices, message handling facility, CRIMP (search engine, services of
electronic trade, etc), the Intranets and extranets, the connections with Internet and the electronic mail, the access points,
the local infrastructures and the development of information highways, the systems of videoconference.
23/03/2001 04:07:00 PM
Page 1
Appendix 2
23/03/2001 04:07:00 PM
Page 2
Appendix 3
DESCRIPTION
OF PRINCIPAL SUBJECTS COVERED BY LEGISLATIVE AUDITORS
Years reviewed: 1998 1999 - 2000

A search of subjects of interest covered by legislative auditors during last years was
completed in autumn 2000. It aimed at giving a progress report on the principal
tendencies of audit in the fields of IT& C. The exhaustive inventory of these subjects of
interest is available on request.
Except for the GAO (United States General Accounting Office), research was carried
out starting from the two following steps: initially, an examination of the contents of
the reports deposited by the principal legislative auditors referred in the " good
addresses " of the AGQ; then, provided it was possible a search inside these reports
using the search tools provided by the site itself, being " Netscape " or " Acrobat "
For the GAO, a research, initially by the title and then by the contents, was made for
all the reports submitted by the group " Accounting and Information Management
Division " (AIMD). We have the title of the relevant audits and the results obtained as
presented in marginal notes in the reports because of the abundance of the covered
subjects and our desire to remain concise in our descriptions.
Several of the listed subjects belong to work that exceeds the strict framework of the
fields of electronic communication and information technology.
Thus our search identified 135 Audits report related to the fields of IT& C, distributed
according to following breakdown:
57 related to audits highly relevant to our needs

36 related to audits moderately relevant to our needs
42 related to audits related specifically to security, that had been carried out
by the GAO
Overall, twenty-five (25) highly relevant audits relate to implementation activities by

an entity for the development and maintenance of information systems in way similar
to the work undertaken by the AGQ at the Ministry for Revenue and with that of
Transport; several of these audits were carried out by the GAO. In addition, seventeen
(17) highly relevant audits relate to specific information systems, including seven (7)
for ERP. Lastly, six (6) other highly relevant audits focused on strategic aspects of
management of IT& C, four (4) with various aspects of the electronic service delivery,
three (3) with architectural considerations or overall infrastructures and two (2) with
activities associated with the field with telecommunications.
As for moderately relevant audits, those relate to aspects of the fields of the rather
marginal and very specific IT& C issues.
Lastly, the audits connected to security, carried out by the GAO, give a report on
deficiencies as regards to IT security in all American agencies. These deficiencies
generally focus on 1) security management of the programs, 2) access control, 3)
change management for the process of development, 4) segregation of duties, 5)
internal controls in the information systems, 6) controls related to the continuity of the
services. Reports were also produced during the proliferation of IT virus and following
23/03/2001 04:07:00 PM
Page 1
Appendix 3
malicious acts carried against Web sites.
critical infrastructures.
Some reports focus on protection of the
All these audits report indicate the majority of legislative auditors carried out sectoral
rather than government wide audits, due to their traditional approach to auditing.
There is a slight move towards performing audits focussing on the strategic
management in the field of IT& C rather than on their operational management or on
activities associated with only one information system with an entity. We also noted
some highly sophisticated audit models to guide the audit work carried out by the
GAO.
23/03/2001 04:07:00 PM
Page 2
Appendix 4
COMPARISON
OF THE PRINCIPAL MODELS OF ANALYSIS FOR
IT& C
This document presents and comments on nine (9) models of analysis which can be
used with regard to the activities undertaken in the fields of electronic communication
and information technology (IT& C).
It appears timely to stress that all the models presented here comply with the spirit of
subparagraph 32 of section 4220 of the handbook of audit of the Auditor General of
Qubec which stipulates that " the standards enumerate three types of criteria which
are regarded as generally recognised and which, if they are relevant for the mandate,
must be privileged: 1) criteria established in the laws and the payments; 2) criteria of
the CICA; the 3) criteria establish by other agencies of recognised experts who follow a
procedure of approval calling upon consultations and public discussions ".
ISACA- CobiT Guidelines Management
ISACA has a framework based best practices and audit control of the information
systems. It particularly aims at helping the leaders to understand and manage the
risks relating to IT and the links between the management processes, the technical
questions, the needs for control and the risks.
The framework of reference is known under the acronym CobiT (Control Objectives for
Information and related Technology). It is structured around four main fields of
management implying 34 processes of management associated with information
technology:
Planning and organisation

To define a strategic IT plan
To define the architecture of information
To determine the technological orientation
To define the organisation and the working relationships of the IT function
To manage the investment into IT
To communicate the objectives and the orientations of management
To manage the human resources
to ensure conformity with the external requirements
To evaluate the risks
To manage the projects
To manage quality
Acquisition and setting in place

To identify the solutions
To acquire and maintain the application software
To acquire and maintain architecture technical
To develop and maintain the procedures IT
To install and validate the systems
To manage the modifications
Distribution and support

To define the levels of service
23/03/2001 04:07:00 PM
Page
Appendix 4
To
To
To
To
To
To
To
To
To
To
To
To
manage the services ensured by thirds

manage the performance and the capacity
ensure a continuous service
ensure the security of the systems
identify and charge the costs
raise awareness and train the users
assist and advise the customers of the IT services
manage the configuration
manage the problems and the incidents
manage the data
manage the installations
manage the operations
Monitoring
To control the processes
To evaluate the adequacy of the internal control
To acquire an independent insurance
To perform an independent audit
Each process of management has goals and objectives, the critical factors of
successes related to the adequate implementation of this process of management,
resources and IT characteristics, indicators making it possible to measure the
performance as well as a narrative description of the five potential levels of maturity
(derivatives of the model of evolution of the software capacities (CMM).
CICA IT Management Controls
This model is based on the concept of the roles and comes to establish the
responsibilities security control which results from this. In this context, the roles are
broken down according to seven axes with knowing, 1) the general direction, 2) the
head of the service of information, the 3) owners, the 4) agents and 5) users of the
information systems just as the suppliers of services, than it is in the chapter of the 6)
development that of 7) the IT operations of the support of the systems. These roles
are also broken down according to the activities that result from this. The table below
summarises the links between the roles and the activities retained by the business
model
Roles
Directorate-General
Head of the service of
information
Owners
Agents
Users
Activities
Approval of the strategies, the policies and the standards;
distribution of the responsibilities; Develop and approval of the
plans of businesses
Develop of the strategies, the policies and the standards;
service of the services of technical support; directorate of
centralised services
Definition and written requirements; responsibility for the
control and security; confirmation of the controls; evaluation of
the risks; classification; delegation; agreements
Comply with of the policies and the standards; logical and
physical access authorisation and control of the changes
Comply with the requirements of the owners; responsibility for
23/03/2001 04:07:00 PM
Page
Appendix 4
Suppliers of services
development
Suppliers of services
IT operations and
support of the systems
IT resources
Development and acquisition of systems of application; comply
with the policies and the standards; management of the
changes; documentation
Agreements on the levels of service; planning; operations;
management of the problems; safeguards; disaster recovery
plans; management of the changes; support of the systems;
physical access
The model makes also a distinction between the concepts of authority, responsibility
and accountability.
On these bases, the model then comes to specify the responsibilities as regards
management for the risks and control before giving a report on the control, broken
down in objectives, standards and techniques:
planning IT
acquisition, the development and the maintenance of the IT systems
IT operations and the support of the systems
IT security
plans of continuity and the resumption of IT services
controls on the applications
GAO Information Technology Investment Management: In Framework for

Assessing and Improving Process Maturity 1
This model results from the work undertaken by United States General Accounting
Office. It identifies the critical processes ensuring success of the investments in the
fields of IT& C and organises them around five levels of maturity (in a way similar to
the CMM). It is also based on guides developed by the GAO (Assessing Risks and
Returns: With Guide for Federal Evaluating Agencies' IT Investment Decision-Making
(GAO/AIMD-10.1.13, February 1997)) and the OMB (Evaluating Information Technology
Investments, A Practical Guides, Executive Office of the President, Office of
Management and Budget, November 1995).
This model focus on investments in the fields of IT& C; according to the following
phases:
Selection of the projects: determination of the projects which best support the
needs related to the mission for the organisation by taking account of the risks and
the returns on the investment.
Control of the projects: assurance that the projects continue to meet the needs and
the required levels.
Evaluation of the projects: comparison of the results anticipated and reached.
To satisfy the preceding goals, the model is broken down in five levels of maturity and
fifteen critical processes. The model also presents, for each process criticises, the goal
of this process, the required prerequisites, essential engagements of the top
management, the activities which must be ensured to satisfy the critical process in
1
United States General Accounting Office Information Technology Investment Management:

With Framework for Assessing and Improving Maturity Process, GAO/AIMD-10.1.23, may 2000
23/03/2001 04:07:00 PM
Page
Appendix 4
question just as the objective elements which prove as this critical process is
formalised in a suitable way in the evaluated entity. The unit also describes the key
practices (tasks) which it is essential to carry out to satisfy the critical process in
question.
The table below shows the five levels of maturity and the fifteen critical processes that
are associated for them.
Level of maturity
Training course 1
Creating Investment
Awareness
Training course 2
Building the Investment
Foundation
Training course 3
Developing a Complete
Investment Portfolio
Training course 4
Improving the Investment
Process
Training course 5
Leveraging IT for Strategic
Outcomes
Critical processes
IT expenditure without a structured investment processes
IT Investment Board Operations

IT Project Oversight
IT Asset Alignment
Business Needs Identification for IT Projects
Proposal Selection
IT Investment Board
Portfolio Selection Criteria Definition
Investment Analysis
Development Portfolio
Portfolio Oversight Performance
Post-Implementation Reviews and Feedback
Portfolio Performance Evaluation and Improvement
Systems and Technology Succession Management
Investment Benchmarking Process
IT-Driven Strategic Business Changes
Lastly, an appendix describing the process of evaluation that the teams of audit
should adopt when they undertake work resting the recommended model
accompanies the document deposited by the GAO.
CICA and AICPA SysTrust
systems 2
MS/MD
Principles and criteria of reliability of the
American Institute of Public Certified Accountants (AICPA) and the Canadian Institute
of the Chartered Accountants (CICA) offer a professional service of certification on the
reliability of the information systems called " SysTrust ".
In the framework of this service, the auditor evaluates and audits up to what point an
information system is reliable compared to four essential principles as regards
reliability of the systems: 1) the availability of the system according to agreements'
taken; 2) security; 3) integrity, 4) scalability
This model seeks to determine if an information system is reliable, i.e. if a system is
able to function without significant error, breakdown and failure during a given period.
Talk-survey AICPA/CICA SysTrust Principles and criteria of reliability of the systems (version
1.0), July 15, 1999
23/03/2001 04:07:00 PM
Page
Appendix 4
Criteria established for each of the four principles evoked previously:
the definition and documentation relating to the objectives of performance, the

policies and the standards compared to the expected performances and the
engagements of the entity like their communication with the human resources
concerned;
procedures implementation with an aim of achieving the goals of performance, in
accordance with the policies and the standards;
activities of monitoring of the system and the environment allowing to identify any
potential degradation and to take suitable initiatives.
SEI Models evolution of the capacities software (CMM
This model results from work of the Software Institute Engineering of the university
Carnegie Mellon de Pittsburgh. It is known under the term " Capability Maturity Model
" (CMM) in English.
This model makes it possible to evaluate the capacities (power to make) of an
organisation development maintenance of information systems. It comprises eighteen
key sectors gathered around five levels of maturity. The table below presents the
correspondence between the levels of maturity and the key sectors.
Level of maturity
1 Initial
2 Can be replicated
3 Defined
4 Controlled
5 Optimised
Key sectors
The Nile
Management of the requirements
software Project planning
Follow up and supervision of software project
Management of subcontracting software
Quality assurance software
Management of configuration software
organisational focusing on the processes
Definition of the process of the organisation

Training scheme
Management integrated software
software Engineering of products
Co-ordination joint committee
Peers Review
quantitative Management of process
Management of software quality
Prevention of the deficiencies
Management of the technological changes
Management of the changes of the process
Thus an entity can have a given level of maturity if all the key sectors of this level of
maturity and preceding levels of maturity (if required) are satisfied. A key sector
known as is satisfied if the very large majority (more than 80%) of the key practices of
this sector are adequately controlled by the entity.
3
Software Institute Engineering Model of evolution of the capacities software, version 1.1,
CMU/SEI-93-TR-24, ESC-TR-93-177, February 1993
23/03/2001 04:07:00 PM
Page
Appendix 4
It is possible to observe that business model is at the base of the principles of

evaluation recommended by CobiT as well as some of the processes of management
of this last. It was however observed that CobiT disregarded sometimes certain
elements of business model
This model is also at the base of two tools for analysis of the risks in the fields of IT& C
used with the government of Qubec, S:PRIME and S:P 2 RAM
ITRB (Information technology resources
Information: In Practical Assessment Tool 4
Board)
Managing
Systems
This reference work is the fruit of experiences accumulated by Information Technology

Resources Board (ITRB), group of leaders in the fields of IT& C of many American
federal agencies. It is in fact a tool for evaluation which aims at making it possible the
American entities governmental federal to better understand how the strategic
implementation of the fields of IT& C can support their mission and improve their
products and services.
The grid of analysis comprises nearly three hundred questions (yes/no) gathered
around three prospects:
the strategy, which makes it possible to know where the organisation moves:
determination of a mission and a vision
the need comprehension of the customers
the presence of a plan of business
the leadership, which makes it possible to mobilise the people
actions deployed by the general directorate
the decision-making process and of strategic planning
the process of management of project
the process of management of the performance
the technology, which makes it possible to set up information systems
the process of acquisition of the goods and the services
the presence of architectures (work, data, systems, technological, flow of
information)
SME (Project Institute management) A Guide to the Project Management
Body of Knowledge5
This guide focuses on best practices for project management of project. The aspects
approached revolve around the various elements of management required to a
powerful management of the projects (of any nature).
More precisely, the document identifies and describes the practices of management of
project generally accepted which should implementation in the agencies. It gathers
them around nine sectors of expertise (presented below).
The guide also positions the various phases of a project (initiation; planning;
implementation; control) as well as the practices of management of project related to
4
Information Technology Board Resources
Managing Systems Information:
In Practical
Assessment Tool, February 1999

5
Project Management Institute Standards Committee

Body of Knowledge, 1996
23/03/2001 04:07:00 PM
A Guides to the Project Management

Page
Appendix 4
each one of these phases. The practices of management of project generally accepted
can thus be considered according to two axes: according to the processes to be
satisfied (e.g. management of the cost of the projects) or according to phases' of the
projects (e.g. planning).
The list below presents the nine sectors of expertise to be satisfied as well as the
aims:
Management of the integration of the projects:

to co-ordinate the various
components of a project such as planning and execution of the project, and control
of the change, cost and quality of the project
Management of the scope of the projects
Time Management of the projects:
Management of the cost of the projects:
Management of the quality of the projects:
Management of the human resources assigned to the projects
Management of communications in projects:
Management of the risks of the projects: to detect and control the risks associated
with the project
Management of acquisitions : to take care to obtain services or products of quality
NSA (National Security Agency) Systems Security Engineering Capability

Maturity Model (CMM) 6
This model initiated by the National Security Agency describes the essential
characteristics of an organisational architecture of security in the fields of IT& C
according to the practices generally observed in the agencies. It covers the following
aspects:
the whole of the cycle of life of an information system:

the whole of the organisation, including the activities of management, organisation
and software engineering
interactions with the various fields of IT& C
interactions with the other sectors such those of acquisitions, the management of
the systems or of certification
The model breaks down initially security in the fields of IT& C according to three axes:
1) the evaluation of the risk: determination and scope of the threats; 2) the setting in
place of measurement: design and implementation of the required solutions; 3)
assurance: corroboration and needs for security.
As for the CMM, the model proposes to determine a level of maturity (from 1 to 5) of
the security of the evaluated organisation. It examines the status of the twenty-two
practical issues of security and management and to compare them with a grid of
maturity defined in term of results. Level 1 implies that all the activities related to
security are carried out at least in a basic way whereas the higher levels require than
the activities are planned and followed (level 2), well defined (level 3), controlled
quantitatively (level 4) and continuously optimised (level 5).
6
Carnegie Mellon University Systems Security Engineering - Capability Maturity Model: Model
Description Document, Version 2.0, April 1, 1999
23/03/2001 04:07:00 PM
Page
Appendix 4
GAO Information Security Management: Learning From Leading Agencies
This model results from work undertaken by the GAO in eight private agencies
recognised as leaders for IT security. It identifies critical issues required to ensure an
adequate management of IT security.
Co-ordination of the activities

To set up a group dedicated to IT security
person in charge for the group can reports to top management

To provide the group with human and financial resources required
To ensure a continuous training and required professional certifications
Evaluation of the risks and determination of the needs
Acknowledge IT resources are critical to the organisation
To evaluate the risks and the security
Ensure users are accountable
To manage the risks on a continuous basis
To set up suitable policies and controls
To lay down policies and controls in relation to risks
Ensure the central group can support the policies
To implement the required programs of involvement
Ensure training of users with regard to the risks and adopted policies
To measure and evaluate the effectiveness of the policies and the controls
To evaluate the factors which affect the risks and which undermine security
To take account of the results of evaluation to determine the later needs and to
report to the authorities.
United States General Accounting Office Security Information: Serious Deficiencies Places
Critical Federal Operations and Assets At Risk, GAO/AIMD-98-92, September 1998
23/03/2001 04:07:00 PM
Page
Appendix 5
SUBJECT OF AUDIT IN THE FIELDS OF IT& C

Name of the subject
Central and ministerial management of the electronic service
delivery (ESD)
Summary description
The government of Qubec use the fields of IT& C to increase its performance and to improve to a
significant degree services to the citizens and the companies. With this intention, it counts, like
other public administration, to carry out significant investments in structuring projects aiming at
the setting in place of new fashions of organisation of work in a context of electronic service
delivery (ESD).
In a very simplified way, the ESD of the government to the citizens and the companies implies
four large functional components: 1) services gateway; 2) batch services or specific; 3)
integration with the information systems of the ministries and agencies (M/O); 4) the specific and
shared infrastructure.
However, the design and the standardisation of several of the strategic elements composing this
ESD remain to carry out whereas significant work are already made by the M/O in the framework
of the modernisation of the public administration. Moreover, the interdepartmental committees of
required work were not set up yet. In this context, significant challenges of management can be
identified, related to the diversity of the existing resources which must be connected in the
context of an integrated solution. Moreover, of new common or divided components must be
made available, operated and managed from the point of view of the overall needs.
The following challenges are currently listed: 1) the overall management of the ESD to the
citizens and the companies; 2) the management of the change; 3) the management of the
security and the confidentiality of information; 4) the standardisation of information and the
mechanisms of exchanges; 5) the management of the development of the information systems
associated with the ESD; 6) the management and operations of the common infrastructures.
Brief description of the importance of the subject and its risks
Importance of the subject
8/10
This subject implies significant investments (Difficult to quantify but several hundreds of
million dollars) in many governmental entities. Many initiatives are already in hand. The ESD
must make it possible to radically modernise the way in which the public administration
relates with the citizens and the companies and offer electronic services.
Risks of the subject
9/10
This subject involves very significant risks based on the scope of work required in the long
term and the gaps corroborated by another AGQ audit team. Four risks can be raised: 1) of
erroneous alignments of the ESD offered by the entities taking into account the governmental
and ministerial strategic needs; 2) uncoordinated work; 3) management (planning,
organisation, co-ordination, evaluation) overall incoherent in the absence of a framework of
management, an overall architecture and well defined scenarios of implementation; 4) a
management " all alone " rather than in " network " of the changes (organisation of work,
structures organisational, culture of the agencies).
23/03/2001 04:07:00 PM
Page 1
Appendix 5
I - Objectives of audit and Expected Deficiencies

Name of the project
delivery (ESD)
Question of importance:
Do mechanisms put in place by the government for the ESD

ensure it is with economy and efficiency and that it contribute in a substantial way to a greater
efficiency of the State and to better quality of the overall services offered.
Preliminary audit objectives
Expected preliminary deficiencies
To ensure that the development and the deployment of the

projects associated with the implementation with the ESD
fall under a governmental plan and ministerial plans
including:
a business model (governmental or ministerial),

objectives of businesses and a strategy
a plan of migration including the priorities, the costs,

the benefit, the financing, the risks and the timeline
To ensure government and the ministries and agencies
evaluate the results compared to the governmental and
ministerial objectives strategic relating to the modernisation
of the public administration
To ensure that standards are established to frame the
interactions with the customers (citizens and companies) in
a context of ESD
To ensure the setting in place and the optimal use of the

common or shared infrastructures
No effective management by results processes in place.
standardisation is incomplete and many standardised

elements are not respected
poor corporate image
To ensure mechanisms were installed to constitute and share

at lower cost the expertise specific to the ESD
ESD is carried out in an anarchistic way, without

examination of respective priority, of problems of
financing, contribution to the modernisation of the
public administration in a multiannual vision.
No overall management for the setting in place of ESD
the management of the changes, when it exists, is only
sectoral
Sectoral infrastructures is set up whereas it would be

possible to be based on common infrastructures
Common infrastructures (e.g. TO CRIMP) were created
but they are little used
Mechanisms implemented but they are insufficient
(participating entities; subjects considered)
Strategy of audit:
Budget of time adding up 2 600 hours over a 6 months period by a team made up of four people.
The budget of time includes the indirect work associated with the quality assurance, with the
review by the peers, the drafting of the Audit report and a possible parliamentary committee on
the subject. The majority of work will be held between February 2001 and June 2001, being
understood that the final Report would be published in volume II of the Report to the national
Assembly for the year 2000-2001.
To the level of the central entities, our work will bring us to the Treasury Board of Qubec and the
Ministry for the Relations with the citizens and Immigration. On the level of the sectoral entities,
our work will imply the publishing of a questionnaire to some fifteen entities (ministries and the
most significant agencies of the government of Qubec) and work on the spot near five of them.
It is expected that the preliminary analysis report of this project of audit will be submitted to the
principal director concerned in April 2001.
23/03/2001 04:07:00 PM
Page 2
Appendix 5
I Evaluation of the project of audit proposed

Audit Project Name
delivery (ESD)
Timely
YES X
period of
implementat
Justification:
ion
the activities considered are upstream many activities of implementation to come and they
condition at least partly success of them
Of the stakeholders pleads that the central management of the ESD is at present deficient
the issue of the modernisation of the public office is a contemporary subject of interest for the
member of Parliament
Criteria of
evaluation
Importance of
the deficiencies
Innovative
aspects
Importance
Public exposure
NOT
Summary of the principal characteristics
Gaps on overall management
Risks identified: alignments of the ESD; resumption of work; incoherent overall

management; management " all alone " of the changes
The consequences of the gaps can be large: corporate image of the

government; effective implementation of modernisation; efficient
implementation and deployment of work
The governmental services are delivered traditionally by mail, telephone or in

person; it is only since very recently in a way integrated by electronic means
The central and ministerial management of the ESD has never been the subject
of a performance audit.
The changes brought by the ESD are of a great visibility for the citizens and the
companies
The modernisation of the public administration includes the implementation of

the ESD
Hundreds of million dollars will be allocated to the ESD during the next years
The level of the debate is very high and covers the two facets of overall
management (central and sectoral aspects)
Evaluatio
n
COPLAN
/ 25 PTAs
/ 20 PTAs
/ 20 PTAs
/ 15 PTAs
Efficiency of the
project
For total work of some 2 600 hours, the performance audit would make it
possible to the national Assembly to have independent information on the
overall management of this aspect related to modernisation of the public
administration and better services to the citizens and the companies
/ 100 PTAs
TOTAL
/ 20 PTAs
23/03/2001 04:07:00 PM
Page 3
Appendix 5
23/03/2001 04:07:00 PM
Page 4
Appendix 6
PROJECT ORGANISATION MANUAL

TABLE OF CONTENTS
1. INTRODUCTION
- Description of the audit project
- Scope
- Purpose
2. ORGANISATION
- Flow chart of functional links

- Description of the roles and the responsibilities
3. PROJECT DEFINITION
Summary of the project: - Definition
- Objectives
- Question of importance
- Principal observations
Detailed examination:
- Audit Project audit objectives and evaluation criteria
- Expected deficiencies and significant issues
4. WORK PLAN
- Preliminary analysis strategy
- Timeline of the preliminary analysis
- Resources required for the preliminary analysis
- Deliverables from the preliminary analysis:
- Outline of the audited field and scope of the project
- Roles and responsibilities
- Regulatory Framework
- Audit Project audit objectives and evaluation criteria
- Expected deficiencies and significant issues
- Strategy of the detailed analysis
- Timeline of the detailed analysis
- Resources required for the detailed analysis
- Report Strategy
For the detailed examination:
- Detailed analysis strategy of the (identification of the deliverables)
- Timeline of the analysis detailed by deliverables
- Resources required for the detailed analysis
- Report strategy
- Audit Programs
- detailed Plans are in appendix
- deliverables at the end of the detailed analysis
- Report (final)
5. RULES OF MANAGEMENT
23/03/2001 04:07:00 PM
Page 1
Appendix 6
- General Principles
- Follow up mechanisms
- timelines
- monthly project report to the director of audit
- Contents Management
- Quality assurance (Q&A), Peers Review (PR) and validation of deliverables
For the detailed examination, to add:
- Follow up on deficiencies observed / observations, at review progress meetings
6. RISKS ASSOCIATED WITH THE AUDIT PROJECT
23/03/2001 04:07:00 PM
Page 2

It Strategic Audit Plan387

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

It Strategic Audit Plan387

Uploaded by

Copyright:

Available Formats

The Auditor General of Qubec

This document has been translated from French using a translating

For information, clarification and queries on this report,

THE AUDITOR GENERAL OF QUBEC

Governmental Directions and Recent Developments............

Legal and Regulatory Framework.......................................

Roles and Responsibilities.................................................

Strategy of intervention ............................................................

Steps Carried Out............................................................

Benchmarks And Audit Limitations .........................

Principles at the Basis of the Strategy................................

Problems Associated with IT& C .................................

Audit Selected. ...........................................................

Follow Up on Previous Audits.........................................

Audit Resources Requirements...........................................

Annual Audit Cycle and Timeline..................................

Audit Process and Project Management ........................

THE AUDITOR GENERAL OF QUBEC

Qubec Policy for the Internet

Plan of action " the governmental information highway For better

Description of key issues covered by legislative auditors in Canada and

Comparison of the principal models of analysis for IT& C

List of Audit Subjects

Project Organisation Manual Table of Contents

THE AUDITOR GENERAL OF QUBEC

THE AUDITOR GENERAL OF QUBEC

THE AUDITOR GENERAL OF QUBEC

Strategic IT Audit Plan

Prepared by Denys Martin Email: dmartin7777@yahoo.com.au

THE AUDITOR GENERAL OF QUBEC

Strategic IT Audit Plan

Prepared by Denys Martin Email: dmartin7777@yahoo.com.au

Fruit of convergence between IT one and telecommunications, IT& C represents a

Governmental Orientations and Recent Developments

In its advising in the management of IT resources, the Treasury Board, contributes to

THE AUDITOR GENERAL OF QUBEC

Strategic IT Audit Plan

Prepared by Denys Martin Email: dmartin7777@yahoo.com.au

Several laws, regulations, directives and decisions govern the management of

THE AUDITOR GENERAL OF QUBEC

Strategic IT Audit Plan

Prepared by Denys Martin Email: dmartin7777@yahoo.com.au

THE AUDITOR GENERAL OF QUBEC

Strategic IT Audit Plan

Prepared by Denys Martin Email: dmartin7777@yahoo.com.au

THE AUDITOR GENERAL OF QUBEC

Strategic IT Audit Plan

Prepared by Denys Martin Email: dmartin7777@yahoo.com.au

Roles and Responsibilities

Co-ordinating committee of governmental information highway

Ministries and Agencies

The Qubec government management framework for delegates responsibility to

THE AUDITOR GENERAL OF QUBEC

Strategic IT Audit Plan

Prepared by Denys Martin Email: dmartin7777@yahoo.com.au

Office of French language

THE AUDITOR GENERAL OF QUBEC

Strategic IT Audit Plan

Prepared by Denys Martin Email: dmartin7777@yahoo.com.au

Source : Bilan et perspectives 1995-1996, Secretariat du Conseil du trsor.

55 p. cent of 432 millions dollars spent in 1998-1999 impacted on these five

In addition, expenditure in information technology agencies comes mainly from the

THE AUDITOR GENERAL OF QUBEC

Strategic IT Audit Plan