Professional Documents
Culture Documents
George Tecuci
tecuci@cs.gmu.edu
http://lalab.gmu.edu/
Knowledge
Engineer
Inference Engine
Dialog
Programming
Knowledge Base
Results
Adapted from:
B.G. Buchanan, D. Barstow, R. Bechtal, J. Bennett, W.
Clancey, C. Kulikowski, T. Mitchell, D.A. Waterman,
Constructing an Expert System,
Identification of a problem
Investigated solution
Develop a computer system that incorporates the expertise
of people familiar with spill detection and containment
(i.e. a knowledge-based system, expert system or agent).
A knowledge engineer is assigned the job of building
the system.
The knowledge engineer becomes familiar with the problem
and the domain.
SME: Sometimes you can tell what the substance is by its smell. Sometimes
you can tell by its color, but that's not always reliable since dyes are used a
lot nowadays. Oil, however, floats on the surface and forms a silvery film,
while acids dissolves completely in the water. Once you discover the type
of material spilled, you can eliminate any building that either don't store
the material at all or don't store enough of it to account for the spill.
G.Tecuci, Learning Agents Laboratory
SME: Sometimes you can tell what the substance is by its smell. Sometimes
you can tell by its color, but that's not always reliable since dyes are used a
lot nowadays. Oil, however, floats on the surface and forms a silvery film,
while acids dissolves completely in the water. Once you discover the type
of material spilled, you can eliminate any building that either don't store
the material at all or don't store enough of it to account for the spill.
G.Tecuci, Learning Agents Laboratory
Attributes of material
Color: Silvery, clear, etc.
Odor: Pungent/choking, etc.
Does it dissolve?
Possible locations: <A set of buildings>
Amount stored: <A number of liters>
G.Tecuci, Learning Agents Laboratory
To determine-spill-material:
[1]
IF
[2]
IF
[4]
IF
THEN
End.
G.Tecuci, Learning Agents Laboratory
IF
THEN
IF
THEN
IF
THEN
KE: Here are some rules I think capture your explanation about
determining the type of material spilled and eliminating possible spill
sources. What do you think?
SME: Uh-huh (long pause). Yes, that begins to capture it. Of course if
the material is silver nitrate it will dissolve only partially in the water.
KE: I see. Well, let's add that information to the knowledge base and
see what it looks like.
Add:
Modify:[1]
IF
Add: [1.5]
IF
the solubility of the spill is moderate,
THEN let the material of the spill be silver-nitrate
with certainty .6
Feedback
loops
among all
phases
Elicitation methodology
(based primarily on Gammack, 1987)
1. Concept elicitation: methods
(elicit the concepts of the domain i.e. an
agreed vocabulary)
2. Structure elicitation: the card-sort method
(elicit some structure for the concepts)
3. Structure representation
(formally represent that structure in a
semantic network)
It is goal oriented.
G.Tecuci, Learning Agents Laboratory
Illustration
Elicitation experiment in the domain of domestic
gas-fired hot water and central heating system
(Gammack, 1987).
Initial interview resulted in about 90 nouns or compound
nouns, both concrete and abstract in nature.
Illustration
The expert initially considered the dictionary definition of these
concepts to be adequate, but since there is no guarantee that the
expert's own definition necessarily matches the dictionary one, a
personal definition of the concepts was given. This produced a few
new concepts, such as "fluid", "safety", and "room".
The definitions indicated that sometimes a concept went beyond the
level of detail given in a general purpose dictionary and sometimes
it meant one very specific idea in the context of the domain.
Illustration
Weaknesses
incomplete and arbitrary coverage
Thermostat
Gas Control
Control Electricity
Electrical Supply
Electrical Components
Mechanical Components
Strengths
gives clusters of concepts and hierarchical organization
splits large domains into manageable sub-areas
easy to do and widely applicable
Weaknesses
incomplete and unguided
strict hierarchy is usually too restrictive
Structure representation
Represents the acquired concepts into a semantic network
and acquires additional structural knowledge:
Ask the expert to sort the concepts by considering each
concept C as a reference, and identifying those related to it.
Ask the expert to order the concepts related to C along a
scale from 0 to 100, marked at the side of a table. The
values are read off the scale and entered in a data matrix.
Generate a network from the matrix, where the nodes are
the concepts and the weighted links represent proximities.
Water
Supply
Flow
Header
Tank
Water
Expansion
Gas
Control
Valve
Main
Gas
Supply
Thermostat
Radiator
Control
Valve
Heat
Boiler
Electrical
Supply
Electrical
Contact
Feedback
Loop
Thermal
Circuit
Pilot
Light
Time
Switch
Gravity
Radiator
Primary
Circuit
Motorized
Valve
Air
Hot
Water
Cylinder
Immersion
Heater
Weaknesses
results depend on various parameter settings
requires more time from the expert
combinatorial explosion limits its applicability
Grid analysis
mild
abrasive
mild
G.Tecuci, Learning Agents Laboratory
abrasive
intelligent
willing
newboy
littlesupervisionreqd
motivated
reliable
mild
ideasmen
L1|114533523|R1dim
L2|124511431|R2unwilling
L3|123544414|R3oldsweats
L4|314521522|R4needsupervision
L5|114522533|R5lessmotivated
L6|322511512|R6notsoreliable
L7|345223154|R7abrasive
L8|115423134|R8staid
D L B P A D M R R
i i o a n o a u o
c z b u n n r t b
y h
k
l
The implicit pole may be elicited by the difference method (in what
way does the singleton differ from the pair) or by the opposite
method (what would be the opposite of the description of the pair).
G.Tecuci, Learning Agents Laboratory
intelligent
dim
intelligent L1|114533523|R1dim
D L B P A D M R R
i i o a n o a u o
c z b u n n r t b
k
y h
l
G.Tecuci, Learning Agents Laboratory
intelligent
willing
newboy
littlesupervisionreqd
motivated
reliable
mild
ideasmen
L1|114533523|R1dim
L2|124511431|R2unwilling
L3|123544414|R3oldsweats
L4|314521522|R4needsupervision
L5|114522533|R5lessmotivated
L6|322511512|R6notsoreliable
L7|345223154|R7abrasive
L8|115423134|R8staid
D L B P A D M R R
i i o a n o a u o
c z b u n n r t b
y h
k
l
intelligent
willing
newboy
littlesupervisionreqd
motivated
reliable
mild
ideasmen
L1|114533523|R1dim
L2|124511431|R2unwilling
L3|123544414|R3oldsweats
L4|314521522|R4needsupervision
L5|114522533|R5lessmotivated
L6|322511512|R6notsoreliable
L7|345223154|R7abrasive
L8|115423134|R8staid
D L B P A D M R R
i i o a n o a u o
c z b u n n r t b
y h
k
l
intelligent
willing
newboy
littlesupervisionreqd
motivated
reliable
mild
ideasmen
L1|114533523|R1dim
L2|124511431|R2unwilling
L3|123544414|R3oldsweats
L4|314521522|R4needsupervision
L5|114522533|R5lessmotivated
L6|322511512|R6notsoreliable
L7|345223154|R7abrasive
L8|115423134|R8staid
D L B P A D M R R
i i o a n o a u o
c z b u n n r t b
y h
k
l
intelligent
willing
newboy
littlesupervisionreqd
motivated
reliable
mild
ideasmen
selfstarters
creative
helpful
professional
overallratinghigh
messers
L1|1145335235|R1dim
L2|1245114312|R2unwilling
L3|1235444143|R3oldsweats
L4|3145215223|R4needsupervision
L5|1145225332|R5lessmotivated
L6|3225115123|R6notsoreliable
L7|3452231545|R7abrasive
L8|1154231344|R8staid
L9|2155135345|R9needapush
L10|1155234345|R10noncreative
L11|4342351455|R11unhelpful
L12|1233215244|R12lessprofessional
L13|2134125234|R13overallratinglow
L14|2254351531|R14tidy
D L B P A D M R R J
i i o a n o a u o o
c z b u n n r t b h
k
y h
n
l
intelligent
willing
newboy
littlesupervisionreqd
motivated
reliable
mild
ideasmen
selfstarters
creative
helpful
professional
overallratinghigh
messers
L1|1145335235|R1dim
L2|1245114312|R2unwilling
L3|1235444143|R3oldsweats
L4|3145215223|R4needsupervision
L5|1145225332|R5lessmotivated
L6|3225115123|R6notsoreliable
L7|3452231545|R7abrasive
L8|1154231344|R8staid
L9|2155135345|R9needapush
L10|1155234345|R10noncreative
L11|4342351455|R11unhelpful
L12|1233215244|R12lessprofessional
L13|2134125234|R13overallratinglow
L14|2254351531|R14tidy
D L B P A D M R R J
i i o a n o a u o o
c z b u n n r t b h
k
y h
n
l
intelligent
willing
newboy
littlesupervisionreqd
motivated
reliable
mild
research
ideasmen
oriented
selfstarters
creative
helpful
professional
overallratinghigh
messers
L1|1145335235|R1dim
L2|1245114312|R2unwilling
L3|1235444143|R3oldsweats
L4|3145215223|R4needsupervision
L5|1145225332|R5lessmotivated
L6|3225115123|R6notsoreliable
L7|3452231545|R7abrasive
L8|1154231344|R8staid
L9|2155135345|R9needapush
L10|1155234345|R10noncreative
L11|4342351455|R11unhelpful
L12|1233215244|R12lessprofessional
L13|2134125234|R13overallratinglow
L14|2254351531|R14tidy
D L B P A D M R R J
i i o a n o a u o o
c z b u n n r t b h
k
y h
n
l
D L B P A D M R R J
i i o a n o a u o o
Positive examples of
c z b u n n r t b h Negative examples of
overall rating high
k
y h
n overall rating high
l
intelligent
willing
newboy
littlesupervisionreqd
motivated
reliable
mild
ideasmen
selfstarters
creative
helpful
professional
overallratinghigh
messers
intelligent(Dick), willing(Dick),
new-boy(Dick), little-sprv-req(Dick),
motivated(Dick), ideas-man(Dick),
self-starters(Dick), ...
overall-rating-high(Paul)
dim(Paul), unwilling(Paul),
experienced(Paul),
need-supervision(Paul),
less-motivated(Paul),
not-so-reliable(Paul), mild(Paul), ...
overall-rating-high(x)
intelligent(x), little-sprv-reqd(x),
reliable(x), self-starters(x),
creative(x), professional(x)
1. Extend the design and identify constraints on the extension just formed.
2. Identify constraint violations; if none, go to step 1.
3. Suggest potential fixes for a constraint violation.
4. Select the least costly fix not yet attempted.
5. Tentatively modify the design and identify constraints on
the modification just formed.
6. Identify constraint violations due to the revision; if any, go to 4.
7. Remove relationships incompatible with the revision.
8. If the design is incomplete, go to 1.
G.Tecuci, Learning Agents Laboratory
Knowledge roles
There are three roles that knowledge plays in this method:
PROPOSE-A-DESIGN-EXTENSION
IDENTIFY-A-CONSTRAINT on a part of the design
PROPOSE-A-FIX for a constraint violation
There are three types of knowledge pieces (each for one role):
PROCEDURE to determine the value of a design parameter
CONSTRAINT to identify limits on the value of a design parameter
car-jamb-return
door-opening = center
calculation
[platform-width - opening-width] / 2
center-opening doors look best when
centered on the platform.
1 Name:
machine-model
2 Precondition:
none
3 Procedure:
database-lookup
4 Table name:
machine
5 Column with needed value: model
6 Parameter test:
max-load suspended-load
7 Parameter test:
done
8 Ordering column:
height
9 Optimal:
smallest
10 Justification:
this procedure is taken from
standards manual iiia, p. 139.
1 Constrained value:
2 Constraint type:
3 Constraint name:
4 Precondition:
5 Procedure:
6 Formula:
7 Justification:
car-jamb-return
maximum
maximum-car-jamb-return
door-opening = side
calculation
panel-width * stringer-quantity
this procedure is taken from installation
manual i, p.12b.
1 Violated constraint:
2 Value to change:
3 Change type:
4 Step type:
5 Step size:
6 Preference rating:
7 Reason for preference:
maximum-car-jamb-return
stringer-quantity
increase
by-step
1
4
Changes minor equipment sizing.
platformwidth
openingwidth
carjambreturn
maximumcarjambreturn
stringerquantity
panelwidth
contributesto
constrains
suggestsrevisionof
Detection of cycles
hoist-cable-quantity = suspended-load / hoist-cable-strength
hoist-cable-weight = hoist-cable-unit-weight * hoist cable-quantity * hoist-cable-length
cable-weight = hoist-cable-weight + comp-cable-weight
suspended-load = cable-weight + car-weight
hoistcablestrength
hoistcablequantity
hoistcableunitweight
hoistcableweight
carweight
hoistcablelength
compcableweight
cableweight
suspendedload
machinemodel
Cycle elimination
Ask for a PROCEDURE which provides a first estimate for one of the parameters in the loop:
1 Name:
hoist-cable-quantity
2 Precondition:
none
3 Procedure:
database-lookup
4 Table name:
hoist-cable
5 Column with needed value: quantity
6 Parameter test:
max-load > car-weight
7 Parameter test:
done
8 Ordering column:
quantity
9 Optimal:
smallest
10 Justification:
this estimate is the smallest hoist cable quantity that
can be used on any job.
Change the role of the original procedure for hoist-cable-quantity from PROCEDURE to
CONSTRAINT:
hoist-cable-quantity minimum-hoist-cable-quantity
minimum-hoist-cable-quantity = suspended-load / hoist-cable-strength
Ask for a FIX knowledge piece corresponding to the violation of this constraint:
1 Violated constraint:
2 Value to change:
3 Change type:
4 Step type
5 Preference rating:
6 Reason for preference:
minimum-hoist-cable-quantity
hoist-cable-quantity
increase
same
4
changes minor equipment sizing
hoistcableunitweight
hoistcableweight
carweight
hoistcablestrength
compcableweight
cableweight
suspendedload
minimumhoistcablequantity
hoistcablelength
machinemodel
car-jamb-return
door-opening = center
calculation
[platform-width - opening-width] / 2
center-opening doors look best when centered on the platform.
IF
car-jamb-return
maximum
maximum-car-jamb-return
door-opening = side
calculation
panel-width * stringer-quantity
this procedure is from installation manual i, p.12b.
IF
maximum-car-jamb-return
stringer-quantity
increase
by-step
1
4
Changes minor equipment sizing.
IF
Three rule types are used to explore the success of a proposed fix or fix combination in a lookahead context before extending the proposed design on the basis of the proposed revision.
IF
THEN
IF
THEN
IF
THEN
Weaknesses
based on a specific problem solving method that
has a limited domain of applicability
defining the knowledge pieces is not easy
Solution:
Ontology reuse (import, merge, export, OKBC protocol, CYC)
Example:
Solution:
Automation of knowledge acquisition through machine learning
Example:
Solution:
Use of machine learning methods by the agent, to continuously
update its knowledge base in response to changes in
the application domain or in the requirements of the application.
Example:
A subject matter expert providing feedback to the agent
and guiding it to update its knowledge base.
Remark:
Currently, software maintenance is four times more expensive that
software development.
With learning agents that are directly taught by humans, there is no
longer a distinction between building the agent and maintaining it.
G.Tecuci, Learning Agents Laboratory
Solution:
Customizable learning agent shell.
It is applicable to a wide variety of application domains.
Requires limited customization.
Example:
Interface
Problem
Solving
Learning
Ontology
+ Rules
Interface
Mixed-initiative
reasoning
between the
expert that has
the knowledge
to be formalized
and the agent
that knows how
to formalize it.
Problem
Solving
Learning
Ontology
+ Rules
Ontology import
Ontology refinement
Agent teaching
Feedback
loops
among all
phases
Recommended reading
G. Tecuci, Lecture Notes on Systematic Elicitation of Expert Knowledge (required
reading)
B.G. Buchanan, D. Barstow, R. Bechtal, J. Bennett, W. Clancey, C. Kulikowski, T.
Mitchell, D.A. Waterman, Constructing an Expert System, in F. Hayes-Roth, D.
Waterman and D. Lenat (eds), Building Expert Systems, Addison-Wesley, 1983,
pp.127-168.
John G. Gammack, Different Techniques and Different Aspects on Declarative
Knowledge, in Alison L. Kidd (ed), Knowledge Acquisition for Expert Systems: A
Practical Handbook, Plenum Press, 1987.
Shaw M.L.G. and Gaines B.R., An interactive knowledge elicitation technique using
personal construct technology, in Alison L. Kidd (ed), Knowledge Acquisition for
Expert Systems: A Practical Handbook, Plenum Press, 1987.
Marcus S. and McDermott J., SALT: A Knowledge Acquisition Language for Proposeand-Revise Systems, Artificial Intelligence, 39 (1989), pp.1-37. Also in Buchanan B.,
Wilkins D. (Eds.), Readings in Knowledge Acquisition and Learning: Automating the
Construction and the Improvement of Programs, Morgan Kaufmann, 1993.