INFORMATION IN THE MATHEMATICAL THEORY OF COMMUNICATION

silan tharakan

INFORMATION IN THE MATHEMATICAL THEORY OF COMMUNICATION

problems of information communication in information systems can be considered in terms of three levels: 1. Technical level. How accurately can information be transmitted? 2. Presentation level. How precisely do the transmitted symbols convey the desired meaning? 3. Effectiveness level (quality)- How suitable is the message as a motivator of human action?
silan tharakan

Historical Development

developed by Norbert Weiner According to Weiner, any organization is held together by the possession of means for acquisition, use, retention and transmission of information information theory has developed primarily as a mathematical theory of communication

silan tharakan

Model of a Communication System

purpose of a communication system is to reproduce at the destination a message selected at the source

silan tharakan

silan tharakan

Each sound, inflection, pitch, etc., of the human voice is a message in the telephone system Considerably more messages are required to relay the voice than the coded letters and numbers

silan tharakan

Mathematical Definition of Information

It is the average number of binary digits which must be transmitted to identify a given message from the set of all possible messages to which it belongs The message to be transmitted is encoded, the codes are sent over the channel, and the decoder identifies the message intended by the codes size of the code is dependent on the coding scheme and the number of possible messages coding scheme for information theory is assumed to be binary
silan tharakan

communication system is required to respond to only two responses, "yes" or "no," then the system needs to transmit one of only two possible signals1 and 0 E.g. a system used to transmit birthday greetings senders may not write their own greetings-they may only select from several standard messages Assume there are eight such messages

silan tharakan

The information content (or code size in bits) may be generalized as:

I log 2 n

where n = the total number of possible messages If n = 8, I = 3 bits

silan tharakan

Reduction of Uncertainty

Information reduces uncertainty Partial information will reduce the uncertainty but not eliminate it in the case of the eight birthday messages, three bits are required to eliminate uncertainty completely by identifying the exact message transmitting a single bit selects one of the two groups and reduces the possibilities from eight to four as shown in figure following
silan tharakan

silan tharakan

Redundancy

Redundancy reduces the efficiency of a particular transmission because more codes are transmitted than are strictly required to encode the message The transmission of redundant data allows the receiver to check whether the received message is correct and may allow the original message to be reconstructed redundancy means that the listener need not read and decode every letter in order to understand the message
silan tharakan