Professional Documents
Culture Documents
Generalization
Generalization
Generalization
Generalization
Generalization
Properties
Generalization
Generalization
Applications
Classical Examples
Applications
Classical Examples
Classification
Autonomous driving
Yes/No
Function Approximation
[1, 1]
Video image
Trained to mimic the behavior of human drivers
Multidimensional Mapping
Steering
Generalization
Generalization
Classical Examples
Generalization
Biological Background
Classical Examples
NetTalk
Speech Synthesis
Dendrites Soma
"Hello"
Phonem
Written text
Dendrites
Passive reception of (chemical) signals
Soma (Cell Body)
Summing, Thresholding
Coded pronunciation
Axon
Aktive pulses are transmitted to other cells
Axon
Generalization
Biological Background
Generalization
Biological Background
ANN-caricatures
Generalization
Limitations
Generalization
Generalization
Generalization
Limitations
Generalization
Limitations
o = sign
!
xi wi
Geometrical interpretation
x2
x1
o Output
o = sign
!
xi wi
Separating hyper plane
Linear separability
Generalization
Generalization
Learning in ANNs
Perceptron Learning
Incremental learning
Generalization
Generalization
Generalization
Generalization
Possible Mappings
Generalization
Possible Mappings
Generalization
Backprop algoritmen
Generalization
Backprop algoritmen
Basic idea:
Minimize the error (E ) as a function of all weights (~
w)
Fundamental trick:
Use threshold-like, but continuous functions
Backprop algoritmen
Generalization
E
wi
Generalization
Backprop algoritmen
Normally one can use the error from each example separately
E=
1 X
(tk ok )2
2
kOut
Output layer:
1
(y ) =
1 + e y
wji wji + i xj
k = ok (1 ok ) (tk ok )
Hidden layers:
0.75
h = oh (1 oh )
1
1 + ex
0.5
wkh k
kOut
0.25
0
10
10
Generalization
Generalization
Generalization
Practical Problems
Generalization
Generalization
Generalization
Risk for overfitting (
overinlarning)!
The net normally interpolates smoothly between the data points
If the network has too many degrees of freedom (weights), the risk
increases that learning will find a strange solution
1
1
0.75
0.75
0.5
0.5
0.25
0.25
0
0
0.2
0.4
0.6
0.8
0.75
0.5
0.25
0
0
0.2
0.4
0.6
0.8
Generalization
0.2
0.4
0.6
0.8