BIG DATA AND SP THEORY OF

INTELLIGENCE

Presented By:
ATHIRA M RAJ
Roll No:22
Semester/Branch:S7/CSE

CONTENTS
Introduction
SP Theory of Intelligence
Problems of Big Data
Volume
Efficiency
Transmission
Variety
Veracity
Visualization
A Road Map
Conclusion
References
2

INTRODUCTION
SP theory of intelligence be applied to the management and
analysis of big data
Overcomes the problem of variety in big data.
Analysis of streaming data- velocity
Economies in the transmission of data
Veracity in big data.
Visualization of knowledge structures and inferential processes

SP THEORY OF INTELLIGENCE

The SP theory is
conceived as a
brain-like system
that receives New
information and
compresses it to
create Old
information

Designed to simplify and integrate concepts across artificial

intelligence, mainstream computing, and human perception and
cognition.
Product of an extensive program of development and testing via
the SP computer model.
Knowledge represented with arrays of atomic symbols in one or
two dimensions called patterns.
Processing are done by compressing information
Via the matching and unification of patterns.
Via the building of multiple alignments
5

BENEFITS OF SP THEORY
Conceptual simplicity combined with descriptive and
explanatory power across several aspects of intelligence.
Simplification of computing systems, including software.
Deeper insights and better solutions in several areas of
application.
Seamless integration of structures and functions within and
between different areas of application
6

Simplification of Computing system

MULTIPLE ALIGNMENT
The system aims to find multiple alignments that enable a
New pattern to be encoded economically in terms of one or
more Old patterns
Multiple alignment provides the key to:
Versatility in representing different kinds of knowledge.
Versatility in different kinds of processing in AI and
mainstream computing.

AN SP MULTIPLE ALIGNMENT

Compression difference:
CD = BN-BE
BN :total number of bits in those symbol in the
New pattern that are aligned with Old symbols in
the alignment
BE :the total number of bits in the symbols in
the code pattern
Compression ratio:
CR = BN/BE;
10

BIG DATA

11

PROBLEMS OF BIG DATA AND

SOLUTIONS
Volume: big data is BIG!
Efficiency in computation and the use of energy.
Unsupervised learning: discovering natural structures
in data.
Transmission of information and the use of energy.
Variety: in kinds of data, formats, and modes of
processing.
Veracity: errors and uncertainties in data.
Interpretation of data: pattern recognition, reasoning
Velocity: analysis of streaming data.
Visualization: representing structures and processes
12

Volume: Making Big Data Smaller

Very-large-scale data sets introduce many data management
challenges.
Information compression.
Direct benefits in storage, management and transmission.
Indirect benefits
efficiency in computation and the use of energy
unsupervised learning
additional economies in transmission and the use of
energy
assistance in the management of errors and uncertainties
in data
processes of interpretation.
13

Efficiency
Via Reduction in volume
Reducing the size of big data and size of search terms
Via Probabilities
Get out unnecessary searching
Via a synergy with data-centric computing
Close integration of data and processing

14

Transmission Of Information
Since so much of the energy in computing is required to
move data around, they have to discover ways to move the
data as little as possible.

The SP system can increase the efficiency of transmission

By making big data smaller (Volume).
By separating grammar (G) from encoding (E), as in
some dictionary techniques and analysis/synthesis schemes
Efficiency in transmission can mean cuts in the use of
energy.
15

Variety of Big Data

Diverse kinds of data and also there are often several
different computer formats for each kind of data.
Adding to the complexity is that each kind of data and each
format normally requires its own special mode of processing
Although some kinds of diversity are useful, there is a case
for developing a universal framework for the
representation and processing of diverse kinds (UFK)
The SP system is a good candidate for the role of UFK
because of its versatility in the representation and processing
of diverse kinds of knowledge.

16

Veracity
For any body of data, I, principles of minimum-length
encoding provide the key
Aim to minimize the overall size of G and E.
G is a distillation or essence of I, that excludes most
errors and generalizes beyond I.
E + G is a lossless compression of I including typos etc
but without generalizations.
Systematic distortions remain a problem.

17

Interpretation of Data
Processing I in conjunction with a pre-established grammar
(G) to create a relatively compact encoding (E) of I
Depending on the nature of I and G, the process of
interpretation may be seen to achieve:
Pattern recognition
Information retrieval
Parsing and production of natural language
Translation from one representation to another
Planning
Problem solving

18

Velocity: Analysis of Streaming Data

In the context of big data, velocity means the analysis of
streaming data as it is received.
This is the way humans process information.
This style of analysis is at the heart of how the SP system
has been designed.
Unsupervised learning.
19

Visualizations
The SP system is well suited to visualization for these
reasons:
Transparency in the representation of knowledge.
Transparency in processing.
The system is designed to discover natural structures in
data.
There is clear potential to integrate visualization with the
statistical techniques that lie at the heart of how the SP
system works.
20

A ROAD MAP
Develop a high-parallel, open-source version of the SP
machine.

This facility would be a means for researchers everywhere to

explore what can be done with the system and to create new
versions of it.

21

CONCLUSION
Designed to simplify and integrate concepts across artificial
intelligence, mainstream computing, and human perception
and cognition, has potential in the management and analysis of
big data.
The SP system has potential as a universal framework for the
representation and processing of diverse kinds of knowledge
(UFK), helping to reduce the problem of variety in big data
The great diversity of formalisms and formats for knowledge,
and how they are processed.
22

REFERENCES
www.cognitionresearch.org/sp.htm .
Article: Big data and the SP theory of intelligence, J G
Wolff, IEEE Access, 2, 301-315, 2014.
International Journal of Computer Engineering and
Technology (IJCET), ISSN 0976-6367,Volume 5, Issue 12,
December (2014), pp. 207-213 IAEME

23

24