Graduate Skills Asessment GSA BERA 8 September 2000

Developing an Instrument to Assess the Generic Skills of

Students in Australian Universities
Sam Hambur, Deirdre Jackson.

The Australian Council for Educational Research, ACER

Paper presented at the British Educational Research Association Conference,

Cardiff University, 7-10 September 2000

In March 1999 The Australian Council for Educational Research (ACER) was
commissioned by the Department of Education, Training and Youth Affairs (DETYA)
to develop instruments to measure graduate performance. Funds were granted from
the Higher Education Innovation Programme (HEIP) to consult with universities and
other interested parties to identify a set of valued generic skills, which could be
assessed effectively at university entry and exit level.
In order to ensure the relevance of the test to universities, invitations were issued to
all Australian universities, to attend meetings at which representatives were asked to
provide a list of generic skills they valued and were aiming to address in their course
teaching. The most frequently mentioned skills are indicated in Appendix 1.

Written Communication, Critical Thinking, Problem Solving and Interpersonal

Understandings were chosen for the initial test because they were popular, seemed
to be essential elements of other skills (such as capacity for Lifelong Learning), and
were likely to be transferable and readily measurable.

The resulting test is called the Graduate Skills Assessment (GSA). The initial format of
the GSA is two hours of multiple-choice items and one-hour of writing tasks.
The test was trialed on a group of 1,000 students from ten volunteering universities
The first exit test has now been developed and will be offered by universities to
students in their final year of their undergraduate degree this year. Approximately
half of the 38 Australian universities have expressed interest in participating. A
second, parallel test will be offered to students entering university in April 2001.
A major focus of this paper is to explain the process and rationale underpinning the
test construct, explain the development of each Component Rationale and to describe
the preliminary outcomes from the trial testing that has taken place as part of the test
development process. It is also worthwhile considering some of the major questions
raised by university staff regarding the usefulness and reliability of the test as an
assessment instrument for generic skills.

A Comment on Transferability
Concern is frequently expressed regarding the question of transferability of skills
from one context to another. In research relating to generic skills and transferability,

1
Graduate Skills Asessment GSA BERA 8 September 2000

Clanchy1 and Ballard point out that generic skills are learned in a context and that
their form varies from discipline to discipline and state that:

while such skills cannot be learned in vacuo, indeed they must be learned in the context of
a specific discipline and body of knowledge, they do not once learned have to be learned
totally anew in each context of learning. Some degree of transfer does occur, and the most
effective learners are those who in fact most quickly recognise the relevance of previously
learned skills to the new contexts and are most readily able to adapt them to those new
contexts.

The Graduate Skills Assessment (GSA) is based on premises that parallel these views.

In her book, Assiter2 (p164) points out that We cannot take it for granted that skills
developed in one social or cognitive context will automatically transfer to another
context. She suggests it may be useful to talk about the meta-competence of
transferring and quotes Fleming, for whom,

transferring skillsare the metaskills, the second-order skills which enable one to select,
adapt, adjust and apply ones other skills to different situations, across different social
contexts and .across different cognitive domains.

According to Assiter, in Marginsons (1994 see Assiter p.165) model of the expert
practitioner the skills required are generic, and include an awareness of context, the
capacity to move between view-points, self-reflection and learning how to learn.

In work with another model, originating from the Alverno College in Milwaukee,
several generic skills have been shown to be general across all disciplines.

Such evaluations of generic/transferable skills are consistent with the test construct
developed for this test. In particular, the emphasis of the test is on skills being
applied across a range of accessible contexts. The assumption is that, the more that
graduates can apply generic skills across the range of contexts provided, the more
likely they are to have mastery of these skills and can apply them to other contexts.

The GSA and International Models

The test that is most similar to the GSA with an equivalent entry- and exit-level test is
the Academic Profiles the US. This test consists of the following components:
Reading/Critical Thinking, Mathematics and Writing (multiple-choice), and an
optional Writing test. The exit-level GRE, in contrast, has the following components:
Verbal (verbal comprehension, vocabulary, analogies), Mathematics (based on a well-
defined syllabus) and Analytical/ Logical Reasoning, and an optional Writing test.

1
Clanchy, J. and Ballard, B. (1995) Generic Skills in the Context of Higher Education, Higher
Education Research and Development, Vol. 14, No. 2
2
Assiter, A. (Ed.) (1995). Transferable Skills in Higher Education, Kogan Page, London

2
Graduate Skills Asessment GSA BERA 8 September 2000

The GSA appears to be unique because of those components it includes. It appears

that no other such test has attempted to deal with the difficult area of Interpersonal
Understandings.

The Consultation Process

Our brief was to consult with universities and other interested parties to identify a
set of generic skills that can be effectively assessed at university entry and exit level
by pencil and paper testing, preferably two hours of multiple-choice testing and one
hour of writing.
All Australian universities were invited to attend meetings held in each mainland
state capital city plus Canberra. Nearly all the institutions sent representatives to one
of the meetings. Participants had the opportunity to express their views on which
components might be useful and practicable in a test of the sort described. In
addition, they were asked to provide feedback in writing from their institution.
Eight institutions responded formally with lists of components. In some cases,
different campuses of an institution provided separate lists.
In addition to the formal responses, a check was made of the published generic
skills/graduate attributes objectives of six other universities. These seemed generally
consistent with the formal responses. Because of the consistency, the formal
responses and published generic skills/ attributes were tallied together (Appendix 1).
In addition, views were sought from other stakeholders such as employers and the
graduate careers councils, either directly or indirectly from published material.

Designing the Construct

Many attributes/skills suggested by universities were poorly defined or too broadly

defined for effective testing in a battery or were simply not amenable to reliable
pencil and paper testing. However, there seemed to be a core of popular skills that
might be amenable to testing.

The most popular skills/ attributes mentioned were:

Communication/Structured Writing
Problem Solving/Applied Reasoning/Strategic Reasoning
Interpersonal Skills/Teamwork/Leadership
Critical Thinking
Ethics/Citizenship/Social Responsibility/Empathy
Commitment to and capacity for Lifelong Learning
IT familiarity/Use of Technology/Information Literacy and Management

Some other skills/attributes, though not as common, were frequently mentioned.

These included a sceptical but open-minded approach, a variety of personal skills
such as self-management, confidence, adaptability, initiative and self-reliance,
analytical skills, logical reasoning, national and international perspective, and
creativity.

3
Graduate Skills Asessment GSA BERA 8 September 2000

Aside from these being less frequently mentioned, they were excluded from the
initial core test because they did not appear readily amenable to effective testing in
the required format or could be subsumed in or indicated by more popular skills.

Other considerations that led to the exclusion of some components at this stage
included:

Reliability, which, amongst other things, limits the number of components that
can be tested (only three multiple-choice components at most in two hours given
the need for the sufficient reliability of comparison between entry and exit level
testing of a student).
Whether the skill/attribute is readily definable as a sufficiently discrete (uni-
dimensional), valid and enduring dimension.
Diagnostic usefulness, predictive value and transferability, for example, the value
of the skills as predictors of, for example, capacity for academic success, interest
in lifelong learning and ability to adapt to and succeed in post-graduate work
and study situations.
The preference for test components that focus on cognitive skills rather than
personality and attitudes.

Some specific reasons for excluding certain components are:

Ethics/Citizenship/Social Responsibility/Empathy - the cognitive elements may

be subsumed by Critical Thinking and the affective elements may not be readily
amenable to value free testing, this also tends to be a specific topic in certain
Philosophy courses.
Commitment to and capacity for Lifelong Learning this is broad, and the
cognitive elements may be partly indicated by components such as Critical
Thinking and Problem Solving, while the affective elements may not be readily
amenable to testing in the given format.
IT familiarity/Use of Technology/Information Literacy and Management some
underlying cognitive abilities may be picked up by Problem Solving, the area is
taught specifically, it is difficult to define its generic elements, and the definition
may change between entry and exit.
Communication, as a general concept - people tend to define this in
psychometrically divergent ways, including written, oral, quantitative and visual,
the area is taught in specific courses, and some of the generic elements can be
addressed in Interpersonal Understandings and Written Communication.

Nevertheless, there is the intention to give further consideration to the testing of

these and other components, such as International Perspective.

Problem Solving, Critical Thinking and the Q/V Problem

4
Graduate Skills Asessment GSA BERA 8 September 2000

It is well known that high order test items are psychometrically distinguishable on
the basis of whether they focus on verbal or non-verbal/ quantitative material and
skills.

Critical Thinking and Problem Solving approaches overlap and both can be applied
to verbal or non-verbal material. Psychometrically, the Q/V effect would be much
stronger than any distinction between general Critical Thinking and general Problem
Solving approaches. (For example, Problem Solving requires some critical thinking.)
Thus it was decided that the Problem Solving component would focus on low verbal
material and the Critical Thinking component would focus on verbal material.

A common thread linking Critical Thinking and Problem Solving is analytical and
logical ability. An alternative approach would be to separately test verbal, general
logical and analytical, and quantitative abilities as is done in the GRE, or just logical
and analytical ability. If this latter approach were to be used, another component
could be added.

Component Rationales
Because it is necessary to narrow the focus of the four broadly-defined components
in order to produce psychometrically coherent test dimensions, many types of item
were trial tested and much consideration has been given to criteria for item selection
so that the focus of each component will be suitable.

The four components are defined in such a way that it is expected that the skills
assessed will have a significant degree of transference to new contexts once sufficient
familiarity has been gained in that context (Assiter, Clanchy and Ballard 3, Gibbs et al4
and Mumford et al5).

In the following, a description is given of each component dimension, together with

a rationale for the way it is defined in the GSA. Also provided for each component
dimension is an example of possible brief descriptions of level of performance.

Aside from Written Communication, the components of the GSA, do not attempt to
assess personality traits that may be related to the ability to put understandings into
action. It is expected however that the validity studies that are under way will show
significant correlation between test performance and real-world performance.

Critical Thinking

3
Clanchy, J. and Ballard, B., (1995 )Generic Skills in the Context of Higher Education, Higher
Education Research and Development, Vol. 14, No. 2
4
Gibbs et al (1994). Developing Students Transferable Skills, The Oxford Centre for Staff
Development.
5
Mumford et al, (1998) Creative Thinking Skills Chapter 7 in Beyond Multiple Choice:
Evaluating Alternatives to Traditional Testing for Selection, Ed. Milton D Hakel, Lawrence
Erlbaum and Associates

5
Graduate Skills Asessment GSA BERA 8 September 2000

There are many approaches to Critical Thinking. At one extreme, there can be a focus
on abstract logic, such as may be exemplified by formal truth tables. Others treat
Critical Thinking as a type of problem solving grab bag. However, mainstream
Critical Thinking focuses on informal logical reasoning in everyday contexts as is
described by authors such as Browne and Keeley6, Diestler7, Kahane and Cavender8,
and Norris and Ennis9. This is the approach used in GSA.

In the GSA, students are asked to comprehend, analyse and evaluate statements and
passages that present viewpoints as they would present in the real world. There is
an avoidance of de-contextualised abstract logic, although a mastery of abstract logic
would help if it could be applied to the real-world material.
Since the ability to think critically depends on familiarity with the context, items
used in the GSA tend to be generally accessible. Further, specialised language is
avoided.

To maintain psychometric coherence, the focus is on text in which a viewpoint is

given and there is an avoidance of mathematical/statistical material (some data
interpretation is included in the Problem Solving component) and other non-verbal
or specialised material.

The items in GSA Critical Thinking can be categorised as follows (though a single
item may have facets of more than one category):

Comprehension in order to identify explicit and implicit meaning.

Analysis in order to identify elements such as definitions being applied, claims
being made, points of view, key issues, lines of reasoning, evidence, conclusions,
arguments, assumptions, logical flaws, logical implications , missing information,
rhetorical devices, ambiguity, inappropriate analogies etc; and
Evaluation in order to judge aspects such as the strength and credibility of
evidence and the validity of lines of reasoning, conclusions and arguments etc.

Candidates who can analyse and evaluate a range of such stimuli at a high level of
proficiency (although the contexts have not been specifically taught) would be
considered to a have a high degree of generic (transferable) critical thinking ability.

Problem Solving

6
Browne, M. N. and Keeley, S. M., (1998) Asking the Right Question: A Guide to Critical
Thinking, Prentice Hall, 5th Ed
7
Diestler, S., (1998) Becoming a Critical Thinker, Prentics Hall, 2nd Ed.
8
Kahane, H and Cavender, (1998) N, Logic and Contemporary Rhetoric, Wadsworth Publishing
Company, 8th Ed.
9
Norris, S. P. and Ennis, R. H., (1989) Evaluating Critical Thinking, Critical Thinking Press and
Software, CA

6
Graduate Skills Asessment GSA BERA 8 September 2000

Problem Solving can be defined in many ways from traditional mathematical

approaches to Polyas10 general mathematical approach to the more generalised
IDEAL approach described by Bransford and Stein11 and business, administration
and interpersonal oriented approaches such as those described by Higgins12 and Hoy
and Tarter13. Clearly there is a massive range of problem types and approaches

Problem solving ability has often been related to intelligence. Gottfredson14 argues,
for example, that, although personal traits such as conscientiousness and motivation
are important in job success, as job complexity increases, it is the higher order
cognitive skills that generally become most important. These skills include capacities
to learn independently, to gather, synthesise, interpret and apply information, and to
deal with complex information and issues. Such higher order skills are said to
correlate significantly with g, which, it is argued, makes it the best general predictor
of job success for complex jobs.

Mumford et al15 note evidence that creative problem solving is related to the ability
to construct the problem, encode information, select categories (concepts) for use,
and combine and reorganise extant knowledge structures.

Clearly the range of problems that can be encountered and approaches that can be
used is enormous. It is not possible in this test to cover a wide range of specific types
of problems and approaches. Further, psychometrically, it is likely that the type of
problem is more important than the general problem-solving approach and it is also
likely that only a few students will have been exposed to business and
administration techniques (which may be learned on the job). Hence, the GSA
approach has been to focus on generally applicable and accessible everyday
problems that vary in complexity, and on the ability of students to identify, analyse,
translate, reorganise and appropriately apply problem-related information. Skills
required range from logico-verbal reasoning to applications of general numeracy.

Specialised mathematical, interpersonal and business/administration problems are

not addressed. Further, the Problem Solving component does not assess personality
traits that may be related to the ability to put problem solving ideas into action.

10
Polya, G., (1957) How To Solve It, Doubleday Anchor, 2nd Ed,
11
Bransford, J. D. and Stein, B. S., (1993) The Ideal Problem Solver, 2nd Ed
12
Higgins, J. M., (1994) 101 Creative Problem Solving Techniques: The Handbook of New Ideas for
Business, The New Management Publishing Company
13
Hoy, W. K. and Tarter, C. J., (1994) Administrators Solving the Problems of Practice, Allyn and
Bacon, Boston
14
Gottfredson, L. S., (1997) Why g Matters: The Complexity of everyday Life, Intelligence,
24(1)
15
Mumford et al, (1998) Creative Thinking Skills Chapter 7 in Beyond Multiple Choice:
Evaluating Alternatives to Traditional Testing for Selection, Ed. Milton D Hakel, Lawrence
Erlbaum and Associates

7
Graduate Skills Asessment GSA BERA 8 September 2000

The types of items used can be categorised by the following (though a single item
may have facets of each type):

Data Interpretation
General Problems, which require little numeracy
Applications of basic Quantitative Reasoning and Numeracy

where the following approaches may be applied:

Identify, Comprehend, Restate the problem

Identify and Analyse information relevant to the problem
Translate and Represent features of the problem
Reorganise, Synthesise and Apply information relevant to a problem
Conceptualise/Develop Strategy and Generate/ Identify problem solution
Evaluate solution strategies and their outcomes

Candidates who can deal with a range of general, practical problems at high level
(even though the contexts have not been specifically taught) would be considered to
have a high level of generic (transferable) problem-solving ability.

Interpersonal Understandings

There is long history of a search for a factor that explains differences in how
effectively people deal with others. Such a facility is generally defined as how well
one person understands others and can apply that understanding in social
situations16.
In this context, concepts such as Social Intelligence17, Interpersonal Intelligence18 and
Emotional Intelligence19 have been theorised and have varying degrees of
experimental support20. Such dimensions have been related to g in a manner
analogous to the verbal factor of intelligence.
This is a complex and evolving area, often focusing on personal attributes of the
people being tested. The GSA Interpersonal Understandings component addresses a
limited aspect of this field.

Interpersonal Understandings items in the GSA focus on the ability of students to:

16
Wechsler, D. (1958). The measurement and appraisal of adult intelligence (4th Ed). Baltimore:
Williams and Wilkins
17
Legree, P. J., (1995) Evidence for an Oblique Social Intelligence Factor Established with a
Likert-Based Testing Procedure, Intelligence, 21
18
Gardner, H, (1993) Frames of mind: The theory of multiple intelligences, Basic Books, NY
19
Mayer, J. D. et al, Emotional Intelligence Meets Traditional Standards for an Intelligence,
Intelligence, Vol. 27, No. 4,
20
Sternberg, R. J. and Smith, C., (1985) Social Intelligence and Decoding Skills in Non-Verbal
Communication, Social Cognition, Vol. 3, No. 2

8
Graduate Skills Asessment GSA BERA 8 September 2000

Comprehend roles and relationships

Show insight into the feelings, motivation and behaviour of other people, and
into issues related to helping or working with others
Recognise how such insight may be applied in order to effectively help or work
with others, including effective feedback, listening, communication, teamwork
and leadership
Recognise how such insight may be applied in order to help solve interpersonal
problems or optimise team performance

The issue of what is the correct interpersonal insight or response to a situation is a

thorny one. In GSA, items are selected with the help of experts and on the basis of
their capacity to represent well defined principles of effective interpersonal
behaviours.
Interpersonal items are generally presented as text (generally low in verbal demand),
though pictorial material may be used.
The Interpersonal Understandings component does not assess personality traits that
may be related to the ability to put ideas into action.

Written Communication

Assessment follows a fairly standard model where there is multi-level assessment of

each task on the basis of each of the following two criteria.

Language and Expression (e.g. control of language conventions, clarity and

effectiveness of expression)
Organisation and Thought (e.g. effectiveness and purposefulness of organisation,
sophistication of analysis of issues or information)

The use of a six-level marking guide will ensure that growth in skill levels can be
identified in those students who sit both the entry and exit level tests.
The Written Communication component of GSA involves two tasks, a Reporting task
and an Argument task. The Reporting task requires students to comprehend, select,
organise, summarise and clearly present factual information. The Argument task
requires students to develop a point of view about an issue and structure and clearly
present an argument in support of that view.

The Trial Test

In response to our request for representative samples of students from various fields
of study, ten universities initially offered about 4000 students for the trial testing of
about 700 multiple-choice items and 8 original essay topics. In fact, only about 1,000
students volunteered and actually participated despite offers of prizes and some
universities offering incentives and putting in a substantial effort. Fortunately, as a
precaution many items were also trial tested with year 11 and 12 students, whose
performance correlated quite well with the university students. Although the school
students did not perform as well as the university students in general, there was

9
Graduate Skills Asessment GSA BERA 8 September 2000

much overlap in range, making analysis of the school population very valuable in
assessing item performance.

Analysis of Results and Item Selection

In general, a high percentage of the multiple-choice items in each component
worked according to routine IRT measures, and there was a remarkably good
match between item difficulty and student ability for a first attempt.

For the university students, preliminary work (using all items in each component
rather than the refined subsets that will be used in the test) indicates that the
correlation between the broad components is as follows:

correlation between Critical Thinking and Problem Solving: 0.51

correlation between Critical Thinking and Interpersonal: 0.66
correlation between Problem Solving and Interpersonal: 0.45

Such correlations, and good internal reliabilities for each component, suggest that
sufficiently discrete components have been identified from which suitable sets of
items can be chosen. IRT, factor analysis and other analytical techniques have been
applied to the data to select the most appropriate items.

It is not surprising to have a significant correlation between components that assess

higher order developed cognitive abilities, especially when the development of the
ability is likely to be related to common experiences, such as those at school and
university, and general executive reasoning ability. The correlation of 0.66 (about
45% of common variance) between Critical Thinking and Interpersonal is probably
related to the substantial verbal component of some Interpersonal units. This has
been reduced in the real test by selecting low verbal units in general, but may not be
totally inappropriate in that social understanding is often verbally mediated.

Sampling and Data Interpretation

Because of the light, self-selected nature of the university sample and the diverse
nature of the trial items, the results of analysis for university students need to be
interpreted with added caution and certain kinds of analysis, including comparison
between groups, cannot be done with any confidence at this stage. However, the
overall sample is good enough to assess general item performance in order to
produce the first tests and to prepare a preliminary reference range for university
student performance. Interesting differences between student performances based on
field of study have already emerged.
The following charts indicate the potential of the test to collect valuable information
on student performance within and across different course areas.

10
Graduate Skills Asessment GSA BERA 8 September 2000

Problem Solving

800

700
P ro b le m So lving

600

500

400

300

200

100
N= 141 130 187 204 43 62 58 28 69 2 19

Field of study

Critical Thinking

11
Graduate Skills Asessment GSA BERA 8 September 2000

800

C ritical T hink ing 700

600

500

400

300

200

100
N= 141 130 187 204 43 62 58 28 69 2 19

Field of study

12
Graduate Skills Asessment GSA BERA 8 September 2000

Interpersonal understandings

800
Inte rp erso nal U nd e rstand ing

700

600

500

400

300

200

100
N= 141 130 187 204 43 62 58 28 69 2 19

Field of study

Written Communication

700
W ritt e n C o m m u n ica tio n

600

500

400

300

200

100
N= 141 130 187 203 43 62 58 28 69 2 29

Field of study

13
Graduate Skills Asessment GSA BERA 8 September 2000

Validity
As with other tests of this nature, test validity will take years to assess
comprehensively. Aside from assessments of content validity, work is under way
looking at correlation between GSA performance, Year 12 results, Quantitative and
Verbal test results and student self-assessments for students in the trial. There may
also be an opportunity to compare academic and non-academic performance in
university with results on the test. In addition, teacher assessments of school student
interpersonal skills will be correlated with performance on the Interpersonal
component. Other studies will be done as more data is collected.

The development of a test of generic skills allows universities to have access to an

instrument to monitor the impact that they are having in design courses to prepare
graduates to meet the challenges of a rapidly changing world. Highly developed
generic skills such as critical thinking, problem solving and interpersonal
understandings should assist graduates to adapt quickly to the complex demands of
the workplace in a global environment. The availability of the test at the beginning
and at the completion of an undergraduate degree will also allow students,
educators, employers and the wider community, for the first time, to have detailed
information on how much value a university is adding to their students education in
these areas.

14
Graduate Skills Asessment GSA BERA 8 September 2000

Bibliography

Academic Profiles Test, Educational Testing Services, Princeton, NJ.

ACNielsen Research Services (1998). Research on Employer Satisfaction with Graduate Skills Interim
Report, Evaluations and Investigations Programme Higher Education Division, DEETYA.

Assiter, A. (Ed.) (1995). Transferable Skills in Higher Education, Kogan Page, London.

Brannick, M. T. et al (1997). Team performance assessment and measurement: theory, methods and
applications, Lawrence Erlbaum Associates, New Jersey.

Bransford, J. D. and Stein, B. S. (199 ). The Ideal Problem Solver, W. H. Freeman and Company, New
York.

Browne, M. N. and Keeley, S. M., (1998) Asking the Right Question: A Guide to Critical Thinking,
Prentice Hall, 5th Ed

Carrol, J. B. (1993). Human Cognitive Abilities, Cambridge University Pres, Cambridge.

Clanchy, J. and Ballard, B. (1995). Generic Skills in the Context of Higher Education, Higher
Education Research and Development, Vol. 14, No. 2, p155-166.

de Bono, E., (1977 ), Lateral Thinking, Pelican, London.

Diestler, S., (1998) Becoming a Critical Thinker, Prentics Hall, 2nd Ed.

Gardner, H, (1993) Frames of mind: The theory of multiple intelligences, Basic Books, NY

Gibbs et al (1994). Developing Students Transferable Skills, The Oxford Centre for Staff
Development.

Goleman, D. (1998). Working with Emotional Intelligence, Bloomsbury, London.

Gottfredson, L. S., (1997) Why g Matters: The Complexity of everyday Life, Intelligence, 24(1)
Graduate Records Examinations, Educational Testing Services, Princeton, NJ.

Hakel, M. D. (1998). Beyond Multiple-Choice: Evaluating Alternatives to Traditional Testing for

Selection, Lawrence Erlbaum Associates, New Jersey.

Hambur, S. (1997). Generic Factors and Curriculum, An Investigation of Test Balance in Cross-
curricular Tests: 1. A Comparison of AST-C and AST-E, (unpublished internal monograph), ACER,
Melbourne.

Hambur, S. (1998). Generic Factors and Curriculum, A Review of Construct Validity and Factorial
Balance in Cross-curricular Tests: 2. GAT and GAMSAT, (unpublished internal monograph), ACER,
Melbourne.

Higgins, J. M., (1994) 101 Creative Problem Solving Techniques: The Handbook of New Ideas for
Business, The New Management Publishing Company

Hoy, W. K. and Tarter, C. J., (1994) Administrators Solving the Problems of Practice, Allyn and Bacon,
Boston

Jones, E and Ratcliffe, G, Critical Thinking Skills for College Students, ERIC-No.: ED358772

Kahane, H and Cavender, (1998) N, Logic and Contemporary Rhetoric, Wadsworth Publishing
Company, 8th Ed.

15
Graduate Skills Asessment GSA BERA 8 September 2000

Legree, P. J., (1995) Evidence for an Oblique Social Intelligence Factor Established with a Likert-
Based Testing Procedure, Intelligence, 21

Marginson, S. (1995). Competency based Education: A compilation of views, Paper presented at the
Australian Education Union, January 1995.

Mayer, J. D. et al, Emotional Intelligence Meets Traditional Standards for an Intelligence,

Intelligence, Vol. 27, No. 4

Mumford, B et al (1998). Creative Thinking Skills, Chapter 7 in Beyond Multiple-Choice: Evaluating

Alternatives to Traditional Testing for Selection, Hakel, M. D., Lawrence Erlbaum Associates, New
Jersey.

Murphy, R et al (1997). The Key Skills of Students Entering Higher Education, University of

Nottingham for the Department for Education and Employment.

Norris, S. P. and Ennis, R. H. (1989). Evaluating Critical Thinking, Critical Thinking Press and
Software, Pacific Grove, California.

Polya, G., (1957) How To Solve It, Doubleday Anchor, 2nd Ed.

Stanton, J. (1995). Business, Industry and Key Competencies, National Industry Education Forum.

Sternberg, R. J. and Smith, C., (1985) Social Intelligence and Decoding Skills in Non-Verbal
Communication, Social Cognition, Vol. 3, No. 2

The Association of Graduate Recruiters, (1995). Skills for Graduates in the 21st Century, The
Association of Graduate Recruiters, Cambridge.

The Australian Psychological Society (1999). Comparison of Employers and Academics Views about
the importance of Generic Competencies for Psychology Graduates, Interface, 27 April, 1999.

The Secretarys Commission on Achieving Necessary Skills, Skills and Tasks for Jobs: A SCANS Report
for America 2000, US Department of Labor.

Watson, G. and Glaser, E. M. (1980). Watson-Glaser critical thinking appraisal manual, Psychological
Corporation, New York.

Wechsler, D. (1958). The measurement and appraisal of adult intelligence (4th Ed). Baltimore:
Williams and Wilkins

Whimbey, A. and Lochhead, J. (1991). Problem Solving and Comprehension, Lawrence Erlbaum and
Associates, New Jersey.

16
Graduate Skills Asessment GSA BERA 8 September 2000

Appendix 1
Responses to Consultations
Component Institutions official or Individuals and other
general stakeholders (such as
Employers and Careers
Councils)
Communication/ ///// ///// ///// ///// / ///// /
Structured Written
Response
Problem Solving/Applied ///// ///// ///// / ///// /
Reasoning/ Strategic
Analytical Skills ///// //// /////
Critical Thinking ///// ///// ///// //
Logical Reasoning ///// //// //
Ethics/ Citizenship/Social ///// ///// ///// ///
Responsibility/Empathy
Creativity ///// /// //
Interpersonal skills/ ///// ///// ///// //// ///// //
Teamwork/ Leadership
Sceptical but Open- ////////
minded
Flexibility/ tolerate ///// / //
uncertainty
Capacity for or ///// ///// // ///
commitment to Lifelong/
Independent Learning
Numeracy/ ability to ///// / //
quantify
Literacy /// /
Information literacy/ IT ///// ///// /// ///
familiarity/ IT Use
Personal Skills/ self- ///// / /////
management/ reflective/
confidence/self-
reliance/initiative
Global/national / ///// // //
historical/cross-cultural
perspective
Information Literacy/ ///// ///
Management/Research
Skills

This summarises the number of times a component has been chosen by stakeholders. Only components
that have been clearly mentioned several times have been tallied.

17
Graduate Skills Asessment GSA BERA 8 September 2000

18