Learning Technology Systems

Architecture -- Overview
LTSA 4.00

Frank Farance
Farance/Edutool
1998-05-21
frank@farance.com, +1 212 486 4700
Slides and Document: http://www.edutool.com/ltsa

1998-05-21 LTSA 4.00 Overview, Frank Farance 1

 Overview

• Review of goals
• Abstraction-Implementation Layers
• Review of each layer
• Demonstration of perspectives
• What does conformance mean?
• Open issues

1998-05-21 LTSA 4.00 Overview, Frank Farance 2

 Review of goals

• Identify common components

• Identify critical interfaces
• Widespread applicability:
– platform-neutral
– content-neutral
– pedagogically-neutral

1998-05-21 LTSA 4.00 Overview, Frank Farance 3

 Architecture Definition
And Purpose
• LTSA 4.0 specification:
http://www.edutool.com/ltsa
• [Excerpt:] “In general, the purpose of developing systems
architectures is to discover high-level frameworks for
understanding certain kinds of systems, their subsystems,
and their interactions with related systems. ...”

1998-05-21 LTSA 4.00 Overview, Frank Farance 4

 Architecture Definition
And Purpose (cont’d)
• [Excerpt cont’d:] “… An architecture isn't a blue print for
designing a single system, but a framework for designing a
range of systems over time, and for the analysis and
comparison of these systems. …”

1998-05-21 LTSA 4.00 Overview, Frank Farance 5

 Architecture Definition
And Purpose (cont’d)
• [Excerpt cont’d:] “… By revealing the shared components
of different systems at the right level of generality, an
architecture promotes the design and implementation of
components and subsystems that are reusable, cost-
effective, and adaptable.”

1998-05-21 LTSA 4.00 Overview, Frank Farance 6

 Abstraction-Implementation Layers

• The Groupings -- abstractions vs. implementations

• The Priorities -- ordering
• The Notations -- text, system, bus
• The LTSA Layers -- five layers

1998-05-21 LTSA 4.00 Overview, Frank Farance 7

 Abstraction-Implementation Layers:
The Groupings
• Organize engineering details into several groupings from
high-level (coarse granularity) to low-level (fine
granularity)
• Groupings are called layers
• Layers above are called abstractions
• Layers below are called implementations

1998-05-21 LTSA 4.00 Overview, Frank Farance 8

 Information Inclusion
Organize raw details as: good, borderline, and bad examples

Raw Details Bad

Good

Borderline

1998-05-21 LTSA 4.00 Overview, Frank Farance 9

 Information Exclusion
Essential vs. Non-Essential Details: Depends on Context

#1: Minimal Details, Only Data Flow #2: More Details, Control Flow

Web Content Web Content

Browser Server Browser Server
User Request
(“Pull”)

Web Content Web Content

Browser Server Browser Server
Server Initiated
(“Push”)
CGI HTML

#3: (Again) More Details, Control Flow #4: More Details, Related Subsystems

1998-05-21 LTSA 4.00 Overview, Frank Farance 10

 Architecture Abstraction Hierarchy
Reference Model (Belz, Suthers, Wheeler)

Application Model LTSA Layer 1; IEEE 1484.1

No category:
IEEE 1484.3,
Abstract System Model 1484.9
- Interaction Model LTSA Layer 2; IEEE 1484.1; AICC
- Conceptual Model LTSA Layer 3; IEEE 1484.1; IMS

Abstract Implementation LTSA Layer 4; IEEE 1484.1; IMS; AICC

Model
LTSA Layer 5; IEEE 1484.1, 1484.2,
Resource Model 1484.4, 1484.6, 1484.7, 1484.10, 1484.11,
1484.12, 1484.13; IMS; AICC, ARIADNE
1998-05-21 LTSA 4.00 Overview, Frank Farance 11
 Abstraction-Implementation Layers:
The Priorities
• High-level design issues have most pervasive effect on
design: first priority
• Low-level design issues are localized: consider these issues
later
• Example: building a house
– High-level (higher design priority): layout and number of rooms
– Low-level (leave till later): choosing wallpaper

1998-05-21 LTSA 4.00 Overview, Frank Farance 12

 Iterative Abstraction
Design Priorities: Colors (high), Shapes (medium), Raw Details

Abstraction of Shapes:
Colored Polygons (Red,
Green, Blue)

Abstraction of Raw Details:

Shapes (Arrow, Box, Star)

Raw Details

1998-05-21 LTSA 4.00 Overview, Frank Farance 13

 Iterative Implementation
Design Priorities: Colors (high), Shapes (medium), Raw Details

Implementation of Colored
Polygons with Constraints:
Eight or fewer sides

Implementation of Shapes with

Constraints: Red, Green, Blue

Implementation of
Colored Shapes:
Constraints: Arrows
Connected to
Polygons

1998-05-21 LTSA 4.00 Overview, Frank Farance 14

 Abstraction-Implementation Layers:
The Notations
• High-level: systems (Yourdon) diagrams
– Process (oval), store (box), flow (arrow)
– Static connections (flows) identify functional decomposition
• Low-level: bus diagrams
– Bus (bar), connector (line), system (box)
– Dynamic, point-to-point connections for on-demand, arbitrary
component communication

1998-05-21 LTSA 4.00 Overview, Frank Farance 15

 Systems Notation:
Processes, Stores, Flows

Two-Way Data Flow

Data One-Way Data

Control Flow
Process Process

One-Way Control Two-Way

Data Flow Control Flow
Process

Data Control
Store Store

1998-05-21 LTSA 4.00 Overview, Frank Farance 16

 Bus Notation

• Common naming
• Common control flow (initiating transactions)
• Non-homogeneous data flow (different transactions)
• Dynamic, on-demand connections

1998-05-21 LTSA 4.00 Overview, Frank Farance 17

 Bus Notation:
General Concepts

P - Common naming
- Common control flow
- Non-homogeneous
Each member Q data flow
of bus has
unique name - Dynamic, on-demand
and shares connections
R
namespace

1998-05-21 LTSA 4.00 Overview, Frank Farance 18

 Bus Notation:
General Concepts

P - Common naming
- Common control flow
- Non-homogeneous
Each member uses Q data flow
same protocol to
request or respond - Dynamic, on-demand
to transactions connections
R

1998-05-21 LTSA 4.00 Overview, Frank Farance 19

 Bus Notation:
General Concepts

P - Common naming

HTTP over FTP over - Common control flow

TCP/IP bus TCP/IP bus
- Non-homogeneous
Q data flow
Common control
protocol, but different - Dynamic, on-demand
data protocols connections
R

Example:
PQ uses FTP, but
S
PR uses HTTP
1998-05-21 LTSA 4.00 Overview, Frank Farance 20
 Bus Notation:
General Concepts

P - Common naming
- Common control flow
No permanent
connections among - Non-homogeneous
members of bus; Q data flow
each transaction is a
connection - Dynamic, on-demand
R connections
Examples:
1. Q sends to P
2. P broadcasts to all
3. R sends to Q S

1998-05-21 LTSA 4.00 Overview, Frank Farance 21

 Systems vs. Bus Notation

• Systems notation useful for:

– Small number of components
– Dedicated functions for each component
– Useful for LTSA layers 1, 3, and 4
• Bus notation useful for:
– Large number of components
– Dynamic, on-demand connections between components
– Useful for LTSA layer 5

1998-05-21 LTSA 4.00 Overview, Frank Farance 22

 Abstraction-Implementation Layers:
The LTSA Layers
• LTSA from high-level to low-level:
– Layer 1: Learner and environment interactions
– Layer 2: Human-centered features
– Layer 3: System components
– Layer 4: Stakeholder perspectives
– Layer 5: Operational components

1998-05-21 LTSA 4.00 Overview, Frank Farance 23

 Layer 1: Learner And
Environment Interactions
• Learner interacts with “environment” as part of a
“learning experience”

Internet/Web Lab Parent Collaboration*

Mentor Teacher Coach
SchoolComputers
Books Television
Other Students*
Employer Multimedia
Library Learner Newspaper
Other Employees* * See later slide: collaboration
is internal to Learner
1998-05-21 LTSA 4.00 Overview, Frank Farance 24
 Layer 1: Learner and Environment:
How Environment Effects Learner
• Primary Focus: the effect of environment on the learner,
not the effect of learner on the environment
• Learner can have important effects on environment
(research, teaching teachers, etc.), but is not primary
technical focus of learning technology

1998-05-21 LTSA 4.00 Overview, Frank Farance 25

 Layer 1: Learner and Environment:
Collaboration: Internal to Learner
• LTSA “learner” (abstraction) can represent one or more
“students” (humans).
• Collaboration among students is internal to learner
• Thus, from an information technology perspective, all
learner interaction is from the environment to the learner,
i.e., inbound information

1998-05-21 LTSA 4.00 Overview, Frank Farance 26

 Layer 1: Learner and Environment
Learning Interactions: Input to Learner
• Primary Focus: technology issues, i.e., the flow
of information, not pedagogy

Learner
Student

Learning
Environment Collaboration
Interactions Student
Student

1998-05-21 LTSA 4.00 Overview, Frank Farance 27

 Layer 2: Human-Centered Features
Limitations => Effect on Design
• Humans receive the learning experience (human learning),
not computers (machine learning)
• Human communication: fundamental limitations has
pervasive effect on design

Learner is not a computer.

Environment Learning Learner Human learners have
Interactions limitations -- complicates
receiving information from
learning interactions.

1998-05-21 LTSA 4.00 Overview, Frank Farance 28

 Layer 2: Human-Centered Features
Why humans aren’t computers
• Humans aren’t reliable receivers of information:
– Humans mishear or don’t hear at all
– Retransmission is necessary
• Humans aren’t predictable receivers of information:
– Humans hear different than desired
– Different strategies are necessary

1998-05-21 LTSA 4.00 Overview, Frank Farance 29

 Layer 2: Human-Centered Features
Why humans aren’t computers (cont’d)
• Humans learn differently (and over time):
– Humans differently from each other
– Same human may learn differently over time
– Need intelligent customization, configuration
– Need rich knowledge library for diverse humans
• Negotiate learner styles:
– Learner alone or system coach alone cannot determine best
learning style

1998-05-21 LTSA 4.00 Overview, Frank Farance 30

 Layer 2: Human-Centered Features
Constructing LTSA System Components
• #1: Start with human sensory input: multimedia delivery to
learner (similar to entertainment)
• #2: Add feedback loop for human unreliability
• #3: Add records database for varying teachers, infer
intelligently about student capability
• #4: Add rich knowledge library, search, and lookup for
diversity of humans
• #5: Add negotiated learning styles

1998-05-21 LTSA 4.00 Overview, Frank Farance 31

 Step #1: Transferring Information

Multimedia Learner

Delivery

• Transfer information to Learner

• Sensory input/interactions for humans
• Delivery of Multimedia: same as entertainment

1998-05-21 LTSA 4.00 Overview, Frank Farance 32

 Step #2: Feedback/Coaching Loop

Multimedia Learner Behavior

Delivery Evaluation

System
Coach

• Feedback necessary, humans are unreliable receivers, e.g.,

they don’t hear what you tell them
• Current state of feedback loop: Behavior, Evaluation,
Assessment

1998-05-21 LTSA 4.00 Overview, Frank Farance 33

 Step #2: Feedback/Coaching Loop (cont’d)

Multimedia Learner Behavior

Delivery Evaluation

System
Coach

• System Coach: determines corrective actions

• Locator Index:
– e.g., next lesson (Learning Content) towards objective
– directs Delivery of Multimedia to Learner
• Loop recovers from errors, focuses on target

1998-05-21 LTSA 4.00 Overview, Frank Farance 34

 Step #3: Records Database

Multimedia Learner Behavior

Delivery Evaluation

Performance
Performance (current)
(history)
System
Records
Coach
Database
Performance (new)

• Performance information
– helps infer corrective action over long periods of time
– nomadic students: allows student migration (handoff) to different
teachers (e.g, grade promotion) and different institutions
• Records Database: Learner’s history, objectives

1998-05-21 LTSA 4.00 Overview, Frank Farance 35

 Step #4: Knowledge Library

Multimedia Learner Behavior

Learning Content
Delivery Evaluation

Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Humans are unpredictable and diverse

• Knowledge libraries necessary to match human diversity
• System Coach:
– directs different learning styles and strategies
– based on Learner’s history, objectives, and Assessment
– supported via queries to Knowledge Library for appropriate
Learning Content
1998-05-21 LTSA 4.00 Overview, Frank Farance 36
 Step #5: Advice from the Learner

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Humans are self-aware: can give advice on learner styles

• Learning Styles:
– interaction between Learner and system to negotiate different
learning styles
– negotiation might actually be assertion: Learner may assert (e.g.,
experimentation), or system coach may assert (e.g., primary school
might dictate learning style)
1998-05-21 LTSA 4.00 Overview, Frank Farance 37
 Review: System Generated Solely
By Human-Centered Features

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

•  #1: Transferring Information

• #2: Feedback loop: unreliable Learners
• #3: Records Database: history supports intelligent inferences, history
allows Learner migration
• #4: Knowledge Library: varying/diverse student capabilities
• #5: Learning Style: negotiate between Learner/system

1998-05-21 LTSA 4.00 Overview, Frank Farance 38

 Review: System Generated Solely
By Human-Centered Features

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• #1: Transferring Information

•  #2: Feedback loop: unreliable Learners
• #3: Records Database: history supports intelligent inferences, history
allows Learner migration
• #4: Knowledge Library: varying student capabilities
• #5: Learning Style: negotiate between Learner/system

1998-05-21 LTSA 4.00 Overview, Frank Farance 39

 Review: System Generated Solely
By Human-Centered Features

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• #1: Transferring Information

• #2: Feedback loop: unreliable Learners
•  #3: Records Database: history supports intelligent inferences,
history allows Learner migration
• #4: Knowledge Library: varying student capabilities
• #5: Learning Style: negotiate between Learner/system

1998-05-21 LTSA 4.00 Overview, Frank Farance 40

 Review: System Generated Solely
By Human-Centered Features

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• #1: Transferring Information

• #2: Feedback loop: unreliable Learners
• #3: Records Database: history supports intelligent inferences, history
allows Learner migration
•  #4: Knowledge Library: varying student capabilities
• #5: Learning Style: negotiate between Learner/system

1998-05-21 LTSA 4.00 Overview, Frank Farance 41

 Review: System Generated Solely
By Human-Centered Features

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• #1: Transferring Information

• #2: Feedback loop: unreliable Learners
• #3: Records Database: history supports intelligent inferences, history
allows Learner migration
• #4: Knowledge Library: varying student capabilities
•  #5: Learning Style: negotiate between Learner/system

1998-05-21 LTSA 4.00 Overview, Frank Farance 42

 Layer 3: System Components

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Processes: Learner, Evaluation, System Coach, Delivery

• Stores: Records Database, Knowledge Library
• Flows: Behavior, Assessment, Performance (past, present, future), Indexes
(query, content, and locator), Learning Content, Multimedia, Learning Style

1998-05-21 LTSA 4.00 Overview, Frank Farance 43

 Learner
(process, human)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Abstraction of student
• Can represent more than one student
• Learner:
– Receives Multimedia
– Emits Behavior
– Negotiates Learning Style

1998-05-21 LTSA 4.00 Overview, Frank Farance 44

 Learning Styles
(data flow)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Learner’s preference as input to system

• System Coach can negotiate and/or restrict Learner’s preferences
• Surrogates: e.g., parentlearner, teacher system
• Other possible surrogates: e.g., parentsystem, student system,
mentorlearner

1998-05-21 LTSA 4.00 Overview, Frank Farance 45

 Behavior
(data flow)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Observable Behavior of the Learner, coded, and sent to the Evaluation

process

1998-05-21 LTSA 4.00 Overview, Frank Farance 46

 Evaluation
(process)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Evaluation:
– Receives Behavior
– Receives Learning Content (context of Learner’s Behavior)
– Stores Performance info based on Evaluation of Behavior
– Produces Assessment given to System Coach

1998-05-21 LTSA 4.00 Overview, Frank Farance 47

Performance Information Stored By Evaluation (data
flow)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Evaluation of Behavior: can emit Performance info

• Stored in Records Database

1998-05-21 LTSA 4.00 Overview, Frank Farance 48

 Records Database
(data store)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Stores Performance information as Learner history

• Retrieves and stores Learner history records for System Coach

1998-05-21 LTSA 4.00 Overview, Frank Farance 49

 Performance Information Received By
System Coach (data flow)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• System Coach gets Learner history by retrieving Performance

information from Learner’s Records Database
• Can select strategies based on history

1998-05-21 LTSA 4.00 Overview, Frank Farance 50

 Performance Information Stored By
System Coach (data flow)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• System Coach can store Performance information, too:

– “bookmarks” (incomplete sessions)
– certifications (milestones, completion of module)
– objectives (outlines for future sessions)

1998-05-21 LTSA 4.00 Overview, Frank Farance 51

 Assessment Information
(data flow)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Evaluation process can emit Assessments: “where there student is at”

• System Coach can use Assessment information, along with other
information, to select appropriate learning experience

1998-05-21 LTSA 4.00 Overview, Frank Farance 52

 System Coach
(process)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning 1
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• 1 Sends, receives, and negotiates Learning Style with Learner

1998-05-21 LTSA 4.00 Overview, Frank Farance 53

 System Coach
(process)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style 2
Learning Performance
Content Content Index Performance (current)
(metadata) 3 (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• 2 Receives current Assessment information and Performance

3
information (history and objectives) for future learning experiences

1998-05-21 LTSA 4.00 Overview, Frank Farance 54

 System Coach
(process)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index 4 Performance (new)

• 4 Queries to the Knowledge Library:

– Query Indexes sent to search for appropriate material
– Content Indexes (metadata) returned as “found” Learning Content
– Extracts the Locator Indexes (e.g., URLs) from the returned Content
Indexes (metadata)

1998-05-21 LTSA 4.00 Overview, Frank Farance 55

 System Coach
(process)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
5 Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Locator Indexes (similar to web URLs) are sent to Delivery

5
process to identify (not transfer) Learning Content
• Note: Delivery is responsible for retrieving and Knowledge Library is
responsible for transferring Learning Content

1998-05-21 LTSA 4.00 Overview, Frank Farance 56

 Query Indexes
(data flow)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Query Index specifies “what to search for” in Knowledge Library

• System Coach sends Query Indexes to search the Knowledge Library
for appropriate Learning Content, based on Assessment info and
Performance info from Records Database

1998-05-21 LTSA 4.00 Overview, Frank Farance 57

 Knowledge Library
(data store)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Database searched via Query Index

• Returns Content Indexes (metadata) of Learning Content that matches
the Query Index
• Emits Learning Content based on Locator Index from Delivery process

1998-05-21 LTSA 4.00 Overview, Frank Farance 58

 Content Index -- Metadata
(data flow)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Search of Knowledge Library returns Content Indexes (metadata) for

Learning Content that matches Query Index
• Content Indexes (metadata) are similar to card catalog entries in a
library
• Locator Indexes (e.g., URLs) can be extracted from Content Indexes
(metadata)

1998-05-21 LTSA 4.00 Overview, Frank Farance 59

 Locator Index
(data flow)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• System Coach extracts Locator Index from the Content Index for
appropriate learning experience
• Locator Index is similar to call number in a library, and similar to URL
of the World Wide Web

1998-05-21 LTSA 4.00 Overview, Frank Farance 60

 Learning Content Received By
Delivery (data flow)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Delivery Process sends a Locator Index to the Knowledge Library

pointing to Learning Content
• Knowledge Library returns Learning Content to Delivery process

1998-05-21 LTSA 4.00 Overview, Frank Farance 61

 Delivery
(process)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Receives Locator Index (e.g., URLs) that identify learning experiences

desired by the System Coach
• Transforms Learning Content to Multimedia
• Sends Learning Content to Evaluation to give context of Learner’s
Behavior
• Sends interactive Multimedia to the Learner

1998-05-21 LTSA 4.00 Overview, Frank Farance 62

 Learning Content Sent By
Delivery (data flow)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Delivery process sends Learning Content to Evaluation to give context

of Learner’s Behavior

1998-05-21 LTSA 4.00 Overview, Frank Farance 63

 Multimedia
(data flow)

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Delivery process generates Multimedia learning experience based on

Learning Content

1998-05-21 LTSA 4.00 Overview, Frank Farance 64

 Functional vs. Actual Learning
Technology Systems
• LTSA components identify functional boundaries and
critical interfaces
• Actual systems integrate several components into
individual products
• Functionality exists in actual systems but may be
embedded within larger system (several components
combined) or single component spread across several
systems

1998-05-21 LTSA 4.00 Overview, Frank Farance 65

 Actual Implementation Example #1:
Web-Based Learning

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)
Abstraction
Implementation Student
Records
Human Presentation (database)
Interface Tool
Human
(e.g., X Windows (browser)
Win95) Courseware
Database
1998-05-21 LTSA 4.00 Overview, Frank Farance (web servers) 66
 Actual Implementation Example #1:
Web-Based Learning

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)
Abstraction
Implementation Student
Records
Human Presentation (database)
Interface Tool
Human
(e.g., X Windows (browser)
Win95) Courseware
Database
1998-05-21 LTSA 4.00 Overview, Frank Farance (web servers) 67
 Actual Implementation Example #2:
Flight Simulator, Flight Instructor

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)
Abstraction
Implementation Flight
Pilot’s
Instructor
Logbook
Flight
Simulator
Pilot

1998-05-21 LTSA 4.00 Overview, Frank Farance 68

 Actual Implementation Example #3:
Non-Electronic, Traditional Classroom

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)
Abstraction
Implementation
Student
Student Uses Teacher Grades
Library Report
Teacher Points Student To
Cards,
Books (A Locator Index)
Transcript
School
Retrieves Teaching Materials
Library
1998-05-21 LTSA 4.00 Overview, Frank Farance 69
 Actual Implementation Example #4:
Home Study Course

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)
Abstraction
Implementation
Home
School Study Student
Course

1998-05-21 LTSA 4.00 Overview, Frank Farance 70

 Layer 4: Stakeholder Perspectives

• Each stakeholder has a view, but not complete picture

• Show each stakeholder perspective within LTSA
• Stakeholder perspectives used to validate views and
subsets of LTSA components
• “It’s in there!” (if not, let us know!)

1998-05-21 LTSA 4.00 Overview, Frank Farance 71

 Layer 4: Stakeholder Perspectives
(cont’d)
• Stakeholder perspectives analyzed by:
– Primary design issues: LTSA components that greatly affect
design and implementation for stake holder (shown as red/bold)
– Secondary design issues: LTSA components that affect design
and implementation of stake-holder, but not critical (shown as
blue/dash ---)
– Other LTSA components: insignificant design issue or not
applicable (no highlight/green fill)

1998-05-21 LTSA 4.00 Overview, Frank Farance 72

 Layer 4: Stakeholder Perspectives
(cont’d)
• Stakeholders, typically, have different and/or incompatible
priorities
• Differing priorities: obstacles to consensus
• Non-technical (political, business) issues just as important
as technical issues
• Cataloging stakeholder perspectives is critical for building
consensus

1998-05-21 LTSA 4.00 Overview, Frank Farance 73

 Abstraction-Implementation:
System Components ==> Stakeholder/Design Perspectives

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)
Abstraction
Implementation
Various Stakeholder/Design Perspectives
Learner, Assessment, Institution, Metadata, Intelligent Tutor,
Mentoring, Collaboration, Ontology, Task, etc.

1998-05-21 LTSA 4.00 Overview, Frank Farance 74

 “Isolated” LTSA Stakeholders

• Isolated LTSA subsets -- little interaction with neighboring

LTSA components
• Little overlap with other “isolated” stakeholders
• Relatively “simple” integration
• Can make use of isolated component standards
• Best stakeholder examples: “Records”, “Metadata”,
“Multimedia”

1998-05-21 LTSA 4.00 Overview, Frank Farance 75

 Learner-Centered

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: learner keeps his/her records, learner has influence on

negotiated learning style
• Secondary: assessment and performance info

1998-05-21 LTSA 4.00 Overview, Frank Farance 76

 Assessment-Centered

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: educational standards, behavior, evaluation, assessment

records, records database reporting
• Secondary: learning style, learner, system control (adaptive
administration)

1998-05-21 LTSA 4.00 Overview, Frank Farance 77

 Records, Certifications

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: records database, performance info

• Secondary: evaluation, assessment info, system coach

1998-05-21 LTSA 4.00 Overview, Frank Farance 78

 Task Model, School-To-Work

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: records database

• Secondary: performance info

1998-05-21 LTSA 4.00 Overview, Frank Farance 79

 Institution-Centered

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: system control, administration, records, reporting, negotiating

(many times: asserting) learning style
• Secondary: learner collaboration, teachers, evaluation (grading),
delivery infrastructure (including multimedia and behavior observation
correlated by learning content)

1998-05-21 LTSA 4.00 Overview, Frank Farance 80

 Content-Centered

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: delivery, multimedia, invocation via locator index (e.g.,

URLs), learning content format
• Secondary: assessment, content search (query index, content index),
knowledge library, learner’s user interface

1998-05-21 LTSA 4.00 Overview, Frank Farance 81

 Metadata

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: query index (search), content index (metadata), locator index

(e.g., URLs), and associating locator indexes with content indexes
• Secondary: knowledge organization, learning content (presentation)

1998-05-21 LTSA 4.00 Overview, Frank Farance 82

 Ontologies, Expert Systems

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: knowledge design and knowledge representation in

knowledge library, learning content (presentation)
• Secondary: query index (search), content index (metadata), and locator
index (e.g., URLs)

1998-05-21 LTSA 4.00 Overview, Frank Farance 83

 Learning Objects

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: knowledge library, query index (search), content index

(metadata), learning content (presentation)
• Secondary: evaluation, performance info and database, assessment
info, system coach, locator index (e.g., URLs)

1998-05-21 LTSA 4.00 Overview, Frank Farance 84

 Multimedia

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: delivery, multimedia, query index (search), content index

(metadata), locator index (e.g., URLs), hardware limitations
• Secondary: system coach, knowledge library, learning content format,
behavior

1998-05-21 LTSA 4.00 Overview, Frank Farance 85

 Collaboration, Asynchronous Learning

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: communication among students with same roles (collective

learner), communication modes (multimedia), delivery tools, and
differing “time zones”
• Secondary: system control by coordinator/moderator, learning style
negotiated among participants and coordinator/moderator

1998-05-21 LTSA 4.00 Overview, Frank Farance 86

 Multiple Role, Team Learning

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: communication among students, but different roles

(collective learner), communication modes (multimedia), delivery
tools, and differing “time zones”
• Secondary: system control by coordinator/moderator, learning style
negotiated among participants and coordinator/moderator

1998-05-21 LTSA 4.00 Overview, Frank Farance 87

 “Overlapping” LTSA Stakeholders

• Concerns many, most, or all LTSA components

• Overlaps with other stakeholders
• Differing, conflicting designs priorities
• Relatively “complex” integration
• Potential non-technical, political, or business obstacles to
consensus and interoperability
• Best stakeholder examples: “Experimentation”, “Intelligent
Tutoring Tools”, “Distance Learning”

1998-05-21 LTSA 4.00 Overview, Frank Farance 88

 Mentoring, Coaching

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: learner and mentor/coach jointly perform evaluation and

jointly choose direction
• Secondary: tools and delivery support mentoring, coaching,
experimentation, recordkeeping

1998-05-21 LTSA 4.00 Overview, Frank Farance 89

 Interactive Environment

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: learner directs style, learner is observed, delivery is

responsive to evaluation
• Secondary: assessment info, system coach, locator index (e.g., URLs)

1998-05-21 LTSA 4.00 Overview, Frank Farance 90

 Simulation

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: delivery, multimedia, learning content to correlate multimedia

and behavior, assessment info, system coach, locator index
• Secondary: behavior, performance info, records database

1998-05-21 LTSA 4.00 Overview, Frank Farance 91

 Learning Tool-To-Tool Communication

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: system coach, evaluation, assessment info, query index

(search), content index (metadata), locator index (e.g., URLs),
delivery, invocation
• Secondary: multimedia, learning style, performance info, records
database, knowledge library, learning content

1998-05-21 LTSA 4.00 Overview, Frank Farance 92

 Sequencing, Pre-Requisites, Co-Requisites

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: system coach, evaluation, assessment info, query index

(search), content index (metadata), locator index (e.g., URLs),
delivery, invocation
• Secondary: multimedia, learning style, performance info, records
database, knowledge library, learning content

1998-05-21 LTSA 4.00 Overview, Frank Farance 93

 Experimentation, Discovery

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: learner chooses direction, acquires knowledge during

experimentation, learner, system coach, query index (search), content
index (metadata)
• Secondary: learner does self evaluation, tools and delivery support
experimentation, behavior, assessment info, locator index (e.g., URLs),
delivery, multimedia

1998-05-21 LTSA 4.00 Overview, Frank Farance 94

 Intelligent Tutoring Tools

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: learner chooses direction, acquires knowledge during use of

tutoring tool, knowledge library may be implicit (not explicitly
defined) in tutor
• Secondary: tutor does evaluation, tools and delivery support
experimentation

1998-05-21 LTSA 4.00 Overview, Frank Farance 95

 Distance Learning, Distributed Learning

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: distributed and nomadic communication of all “flows”

(behavior, performance, multimedia, etc.)
• Secondary: processes and stores components
• NOTE: Take note of primary design issues (red/bold)
– How does this slide differ from all others?
– Answer: All flows are primary, everything else is secondary

1998-05-21 LTSA 4.00 Overview, Frank Farance 96

 “Parallel” LTSA Stakeholders

• Integration of multiple sessions, each with differing

learning experiences and interactions
• Synchronization of multiple sessions: starting, stopping,
coaching, learner interfacing
• Integration, collaboration, and synchronization of feedback
and coaching strategies
• Best stakeholder example: “Student Teacher”

1998-05-21 LTSA 4.00 Overview, Frank Farance 97

 Parallel Sessions for Same Learner

• Primary: multiple, simultaneous learning experiences; multiple flows of

behavior, learning style, multimedia
• Secondary: multiple evaluation, system coach, and delivery processes
• Example: flight simulator -- simultaneous training of flying skills, navigation
skills, communication skills, cockpit resource management
1998-05-21 LTSA 4.00 Overview, Frank Farance 98
 Student Teacher
Student

Teacher

• Primary: student teacher as a teacher

• Secondary: student teacher as a student

1998-05-21 LTSA 4.00 Overview, Frank Farance 99

 Multi-Tiered Process Improvement
Student

Teacher

Principal

• Student is evaluated by teacher, teacher coaches student

• Teacher is evaluated by principal, principal coaches teacher
• Principal is evaluated by school board, school board coaches principal
• Teacher is also evaluated by performance of his/her students
• Principal is also evaluated by performance of his/her students
1998-05-21 LTSA 4.00 Overview, Frank Farance 100
 Standards and Specification
Development Organizations
IEEE, ANSI, ISO: Accredited Standards
IEEE
(LTSC) ANSI ISO
1484

Specs From
Specs Consortia

AICC ADL IMS ARIADNE

Liaising
Requirements Requirements

1998-05-21 LTSA 4.00 Overview, Frank Farance 101

 IEEE 1484 As A Stakeholder
(IEEE 1484 Roadmap)
• IEEE 1484 working groups (WGs) and study groups (SGs)
each involve certain areas of LTSA
• Some LTSA components should not be standardized,
e.g., evaluation methods, delivery systems
• Some LTSA components should be standardized outside
1484, e.g., multimedia, education standards, cultural
adaptation
• More info: http://www.manta.ieee.org/p1484

1998-05-21 LTSA 4.00 Overview, Frank Farance 102

 IEEE 1484.1:
Architecture and Reference Model WG

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: all components

• Secondary: none

1998-05-21 LTSA 4.00 Overview, Frank Farance 103

 IEEE 1484.2:
Learner Model WG

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: learner, negotiated learning style, evaluation, performance

info, assessment info, system coach
• Secondary: behavior

1998-05-21 LTSA 4.00 Overview, Frank Farance 104

 IEEE 1484.3:
Glossary WG

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: none
• Secondary: all components

1998-05-21 LTSA 4.00 Overview, Frank Farance 105

 IEEE 1484.4:
Task Model WG

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: records database

• Secondary: performance info

1998-05-21 LTSA 4.00 Overview, Frank Farance 106

 IEEE 1484.6:
Course Sequencing WG

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: system coach, performance info, assessment info, query index

(search), content index (metadata), locator index (e.g., URLs),
invocation of learning content
• Secondary: learning style, records database, knowledge library,
delivery, learning content

1998-05-21 LTSA 4.00 Overview, Frank Farance 107

 IEEE 1484.7:
Tool/Agent Communication WG

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: system coach, behavior, evaluation, assessment info,

performance info, records database, query index (search), content
index (metadata), locator index (e.g., URLs)
• Secondary: query index (search), content index (metadata), knowledge
library, delivery

1998-05-21 LTSA 4.00 Overview, Frank Farance 108

 IEEE 1484.9:
Task Ontology WG

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: knowledge library, learning content (presentation)

• Secondary: query index (search), content index (metadata), locator
index (e.g., URLs)

1998-05-21 LTSA 4.00 Overview, Frank Farance 109

 IEEE 1484.10:
CBT Data Interchange WG

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: system coach, performance info, assessment info, query index

(search), content index (metadata), locator index (e.g., URLs),
invocation of learning content
• Secondary: learning style, behavior, records database, knowledge
library, learning content, delivery

1998-05-21 LTSA 4.00 Overview, Frank Farance 110

 IEEE 1484.11:
Computer Managed Instruction WG

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: performance info, assessment info, query index (search),

content index (metadata), locator index (e.g., URLs), invocation of
learning content
• Secondary: behavior, records database, system coach, knowledge
library, learning content, delivery

1998-05-21 LTSA 4.00 Overview, Frank Farance 111

 IEEE 1484.12:
Learning Objects Metadata WG

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: query index (search), content index (metadata), locator index

(e.g., URLs)
• Secondary: system coach, knowledge library, learning content,
delivery

1998-05-21 LTSA 4.00 Overview, Frank Farance 112

 IEEE 1484.13:
Student Identifiers SG

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: performance info, records database

• Secondary: learner, system coach

1998-05-21 LTSA 4.00 Overview, Frank Farance 113

 Other Specification Development
Organizations As Stakeholders
• Educom’s Instructional Management Systems (IMS)
Project
http://www.imsproject.org
• Aviation Industry CBT Committee
http://www.aicc.org
• DoD Advanced Distributed Learning
http://www.adlnet.com
• European Union ARIADNE Project

1998-05-21 LTSA 4.00 Overview, Frank Farance 114

 Educom’s Instructional Management
Systems (IMS) Project
• Main areas of development:
– Metadata: learning technology metadata
– Content objects: learning content, course structure
– Management systems: collaboration, reporting
– Profiles: student records
– External services: interface to institution
• More info: http://www.imsproject.org

1998-05-21 LTSA 4.00 Overview, Frank Farance 115

 IMS Metadata

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: content index (metadata) and associating locator indexes with

content indexes
• Secondary: query index (search), locator index (e.g., URLs),
knowledge organization in knowledge library

1998-05-21 LTSA 4.00 Overview, Frank Farance 116

 IMS Content Objects

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: knowledge library, content index (metadata), learning content

(presentation)
• Secondary: evaluation, performance info, assessment info, system
coach, locator index (e.g., URLs), delivery

1998-05-21 LTSA 4.00 Overview, Frank Farance 117

 IMS Management System

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: learner, system coach, performance info, locator index (e.g.,

URLs)
• Secondary: negotiated learning style, records database, content index
(metadata), knowledge library, delivery

1998-05-21 LTSA 4.00 Overview, Frank Farance 118

 IMS Profiles

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: records database, performance info

• Secondary: evaluation, assessment info, system coach

1998-05-21 LTSA 4.00 Overview, Frank Farance 119

 IMS External Services

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: records database, student registration, institution back office

• Secondary: performance info

1998-05-21 LTSA 4.00 Overview, Frank Farance 120

 Aviation Industry CBT Committee
(AICC)
• Develops guidelines and recommendations for CBT in
aviation applications
• Subject areas: Courseware Delivery Stations, Digital
Audio, Operating Windowing System, CBT Peripheral
Devices, Computer Managed Instruction (CMI),
Courseware Interchange, Digital Video, Icon Standards
• More info: http://www.aicc.org

1998-05-21 LTSA 4.00 Overview, Frank Farance 121

 AICC AGR003:
Digital Audio

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: multimedia
• Secondary: delivery

1998-05-21 LTSA 4.00 Overview, Frank Farance 122

 AICC AGR005:
CBT Peripheral Devices

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: behavior, delivery, multimedia

• Secondary: learner

1998-05-21 LTSA 4.00 Overview, Frank Farance 123

 AICC AGR006:
Computer Managed Instruction

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: performance info, assessment info, query index (search),

content index (metadata), locator index (e.g., URLs), invocation of
learning content
• Secondary: behavior, records database, system coach, knowledge
library, learning content, delivery

1998-05-21 LTSA 4.00 Overview, Frank Farance 124

 AICC AGR007:
Courseware Interchange

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: system coach, performance info, assessment info, query index

(search), content index (metadata), locator index (e.g., URLs),
invocation of learning content
• Secondary: learning style, behavior, records database, knowledge
library, learning content, delivery

1998-05-21 LTSA 4.00 Overview, Frank Farance 125

 AICC AGR008:
Digital Video

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: multimedia
• Secondary: delivery

1998-05-21 LTSA 4.00 Overview, Frank Farance 126

 AICC AGR009:
Icon Standards

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: learner, learning content

• Secondary: delivery, multimedia

1998-05-21 LTSA 4.00 Overview, Frank Farance 127

 AICC CRS002:
Glossary of Terms Related to CBT

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: none
• Secondary: all components

1998-05-21 LTSA 4.00 Overview, Frank Farance 128

 DoD Advanced Distributed
Learning (ADL)
• Research and development areas:
– Promote widespread collaboration.
– Exploit Internet technologies.
– Develop next generation learning technologies.
– Create reusable content, and lower costs, with object-based tools.
• Supplies requirements to IMS
• More info: http://www.adlnet.org

1998-05-21 LTSA 4.00 Overview, Frank Farance 129

 ADL Collaboration

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: all components, ADL addresses collaboration of learning

technologies and learning technology requirements
• Secondary: none

1998-05-21 LTSA 4.00 Overview, Frank Farance 130

 European Union - ARIADNE

• Collaborative browsing
• Learning technology metadata
• Working with IMS
• More info:
http://tina.lancs.ac.uk/computing/
research/cseg/projects/ariadne/

1998-05-21 LTSA 4.00 Overview, Frank Farance 131

 ARIADNE Metadata

Multimedia Learner Behavior

Learning Content
Delivery Evaluation
Learning
Style
Learning Performance
Content Content Index Performance (current)
(metadata) (history)
System
Knowledge Records
Coach
Library Database
Query Index Performance (new)

• Primary: content index (metadata) and associating locator indexes with

content indexes
• Secondary: query index (search), locator index (e.g., URLs),
knowledge organization in knowledge library

1998-05-21 LTSA 4.00 Overview, Frank Farance 132

 Layer 5: Operational Components

• Common components that build the system

• Operational components may serve several functions
• Several system components may be implemented via the
same operational components, e.g., performance info and
learning content both coded as XML (an operational
component)

1998-05-21 LTSA 4.00 Overview, Frank Farance 133

 Layer 5: Operational Components
(cont’d)
• Operational components are identified by:
– Systems, e.g., clients and servers
– Connectors (interface protocols), e.g., HTTP, PPP
– Busses (namespaces), e.g., URLs, telephone numbers
• Components (systems) on a bus can “talk” to each other
via interface protocols

1998-05-21 LTSA 4.00 Overview, Frank Farance 134

 Abstraction-Implementation:
Stakeholder/Design Perspective => Operational Components

Various Stakeholder/Design Perspectives

Learner, Assessment, Intelligent Tutor, Distance Learning,
Mentoring, Collaboration, Ontology, Task, etc.

Abstraction
Implementation
Human
Protocols Interface
Processes
Data Interchange
Storage
Control Flow
1998-05-21 LTSA 4.00 Overview, Frank Farance 135
 Information Busses:
Example: Internet Client-Server Bus, Client Perspective

Client - Client connects to many

servers
- On-demand connections
Server - Several servers may have
same purpose, but varied
resources; like libraries:
Server
same interface, but varied
collections of books
Server

1998-05-21 LTSA 4.00 Overview, Frank Farance 136

 Information Busses:
Example: Internet Client-Server Bus, Server Perspective

Server - Server handles requests

from multiple clients
- On-demand connections
Client - Clients may appear
homogenous, but have
different identities
Client

Client

1998-05-21 LTSA 4.00 Overview, Frank Farance 137

 Information Busses:
Example: Combining/bridging two busses

Web
User Server 1
Web Browser: “sits” on two
busses: both window manager
application & web application
World Wide Web
Window Manager GUI “Bus” of Browsers
Web
Application “Bus” App 1 Client and Web Servers
Examples:
GUI Web
X Windows, Win95 App 2 Server 2

GUI Web
App 3 Server 3

1998-05-21 LTSA 4.00 Overview, Frank Farance 138

 Information Busses:
Example: Combining/bridging three busses

World Wide Web

“Bus” of Browsers Web
User Server 1
and Web Servers
Chat “Bus”
Web Browser: of Sessions
Web
GUI<->Web Server 2
GUI Web
App 1 Client
Chat
Window Server
Manager
GUI Chat
Application App 2 Client 1
“Bus”
Chat Session: Chat
GUI<->Chat Client 2

1998-05-21 LTSA 4.00 Overview, Frank Farance 139

 Information Busses:
Commonly Used Busses/Protocols
• Layer 5 is: collection of common bus protocols, no specific
requirements
• Examples:
– Transport, Network, Data Link: TCP, UDP, IP, Ethernet, telephone
– Application, Presentation, Session: X11, HTTP, IRC, SMTP,
NNTP, FTP, TELNET
– Data, Control, Security: HTML, XML, RDF, RPC, CORBA,
DCOM, HLA, Kerberos, HTTPS, SSL, HLA

1998-05-21 LTSA 4.00 Overview, Frank Farance 140

 Summary of Five LTSA Abstraction-
Implementation Layers
Learner
Learner
Layer 1 Environment Environment
Interactions
Interactions

Layer 2 Five Human-Centered Features

M L B
LC
System D E
Layer 3 Components LC
LI LS A
P
CI P
KL SC RD
QI
Layer 4 40+ Stakeholder Perspectives
Protocol
Layer 5 & Interface
Busses

1998-05-21 LTSA 4.00 Overview, Frank Farance 141

 LTSA Layer Summary (cont’d)

• Layer 1:
– Learning experience: learners interacting with the environment
– A learner (abstract) may represent several students (humans)
– From information technology perspective, flow is one-way:
environment ==> learner
– Collaboration is within the learner abstraction

1998-05-21 LTSA 4.00 Overview, Frank Farance 142

 LTSA Layer Summary (cont’d)

• Layer 2:
– Because the learner abstraction is “implemented” with “humans”
(human learner), rather “implemented” with “computers” (machine
learning), the “nature of humans” has pervasive effect on system
design
– Fundamental limits of human communication

1998-05-21 LTSA 4.00 Overview, Frank Farance 143

 LTSA Layer Summary (cont’d)

• Layer 2 (cont’d):
– Humans aren’t reliable receivers of information: retransmission
– Humans aren’t predictable receivers of information: different
strategies
– Humans learn differently (and over time): customizable,
configurable

1998-05-21 LTSA 4.00 Overview, Frank Farance 144

 LTSA Layer Summary (cont’d)

• Layer 3:
– System notation: process, store, flow
– Human-centered feedback loop: learner, behavior, evaluation,
assessment, system coach, delivery, multimedia
– More “intelligence” from learning content: need records database
and performance records
– Learner has different “teachers” during his/her lifetime: records
database facilitates “hand-off”

1998-05-21 LTSA 4.00 Overview, Frank Farance 145

 LTSA Layer Summary (cont’d)

• Layer 3 (cont’d):
– Humans each learn differently (and learn differently over time):
rich knowledge library required; query indexes and content indexes
(metadata) make best use of knowledge library
– Learners and their surrogates (e.g, parent, mentor) negotiate
learning styles with system coaches and their surrogates (e.g.,
institution, teacher, courseware developer)

1998-05-21 LTSA 4.00 Overview, Frank Farance 146

 LTSA Layer Summary (cont’d)

• Layer 4:
– Stakeholders each have own, incomplete perspectives
– All stakeholders’ perspectives are incorporated in LTSA system
components diagram
– IEEE 1484: working groups specifying critical interfaces; some
technical standards outside of 1484; “education standards” are
necessary but outside scope of 1484 and “technical standards”

1998-05-21 LTSA 4.00 Overview, Frank Farance 147

 LTSA Layer Summary (cont’d)

• Layer 5:
– Bus notation: bus, connector, system
– Operational components: actual implementation
– One operational component (e.g., HTTP) might be used by several
system components (e.g., records database, knowledge library)
– One system component (e.g., delivery) might use several
operational components (e.g., HTTP, HTML, MPEG)

1998-05-21 LTSA 4.00 Overview, Frank Farance 148

 LTSA Layer Summary (cont’d)

• Layer 5 (cont’d):
– Bus may be used for several purposes
– Systems on bus share common namespace, e.g., URLs, process
IDs, object IDs
– Systems may be connected, simultaneously, to several busses

1998-05-21 LTSA 4.00 Overview, Frank Farance 149

 LTSA Layer Summary (cont’d)

• Layer 5 (cont’d):
– Systems dynamically connect, on-demand to other systems (e.g.,
clients to servers)
– System connectivity is not pre-arranged
– Connections (protocols) may serve several purposes (e.g., XML
used for performance records and learning content; MIDI used for
behavior and multimedia)

1998-05-21 LTSA 4.00 Overview, Frank Farance 150

 LTSA Conformance

• LTSA conforming products/services

– provide the critical interfaces (functionality) that conform to one or
more components of LTSA: Learner (Collaboration), Behavior,
Evaluation, Performance, Records Database, System Coach, Query
Index (search), Content Index (metadata), Locator Index (e.g.,
URLs), Knowledge Library, Delivery, Multimedia
– identify the protocols that describe the bus interface
– identify the interface point

1998-05-21 LTSA 4.00 Overview, Frank Farance 151

 LTSA Conformance

• LTSA conforming products/services

• Measuring conformance: the system under test is called the
“implementation under test”
• A system that conforms to the LTSA specification is called
a “conforming implementation”
• “A conforming implementation shall conform to all five
layers.”

1998-05-21 LTSA 4.00 Overview, Frank Farance 152

 LTSA Layer 1 Conformance

• Validating Learner-Environment interactions

• “The implementation under test shall identify the
boundaries of the Environment, the flows of Interactions,
and the Learner.”
• “The implementation under test shall identify the Learner's
relationship to human students.”
• Examples of Learner/Student relationship:
– Is the Learner represented by more than one Student?
– Is there collaboration among the Students?

1998-05-21 LTSA 4.00 Overview, Frank Farance 153

 LTSA Layer 2 Conformance

• Validating human-centered features

• “The implementation under test shall identify the methods
for addressing the human-centered features:”
– unreliable learners
– unpredictable and diverse learners
– nomadic learners
– influence on learning style

1998-05-21 LTSA 4.00 Overview, Frank Farance 154

 LTSA Layer 3 Conformance

• Validating decomposition into LTSA system components

• “The implementation under test shall identify the available
LTSA system components: Learner, Learning Style,
Behavior, Evaluation, Performance, Records Database,
Assessment, System Coach, Query Index, Content Index,
Locator Index, Knowledge Library, Learning Content,
Delivery, Multimedia.”

1998-05-21 LTSA 4.00 Overview, Frank Farance 155

 LTSA Layer 4 Conformance

• Identifying which LTSA subset (stakeholder) is applicable

-- stakeholders can be any subset, not limited to the
examples in the LTSA specification
• Mapping conceptual to actual system components
• “The implementation under test shall identify the subset of
applicable LTSA system components.”
• “The implementation under test shall identify the
boundaries and mapping of LTSA system components.”

1998-05-21 LTSA 4.00 Overview, Frank Farance 156

 LTSA Layer 5 Conformance

• Identifying the information busses -- bus protocols and

topology not limited to examples in LTSA specification
• “The implementation under test shall identify the
information busses in use, their namespace, control
protocols, and mapping to LTSA system components.”

1998-05-21 LTSA 4.00 Overview, Frank Farance 157

 Open Issues

• Verifying stakeholders perspectives

• Identifying and refining operating components: protocols,
interfaces
• Public review in IEEE 1484.1 and other fora

1998-05-21 LTSA 4.00 Overview, Frank Farance 158

 Summary

• Yourdon (systems notation): tried and true

• Bus notation used for larger number of component
• Five abstraction-implementation layers:
– Learner and environment interactions
– Human-centered features
– System components
– Stakeholder perspectives
– Operational components

1998-05-21 LTSA 4.00 Overview, Frank Farance 159

 Summary (cont’d)

• Wide range of implementations supported

• All stakeholders’ perspectives identified
• Reflects commercial systems

1998-05-21 LTSA 4.00 Overview, Frank Farance 160