3 views

Uploaded by api-127299018

- Annual Ex001
- 1-s2.0-S003448771000025X-main.pdf
- DEACON, T. W. Steps to a Science of Biosemiotics
- A Practical Phyiscal Random Number Generator
- Homologous Hierarchy Modeling Exciting Logics Architectures
- Data Representation Worksheet 4 Images
- Primer_Self_Instruction_Manual_r2.5.pdf
- Multicriteria Decision Making Using Double Refined Indeterminacy Neutrosophic Cross Entropy and Indeterminacy Based Cross Entropy
- Week05_L02-3
- Math133 Unit 5 - how to prepare
- ERIC_ED143516-Calculus-Part II.pdf
- crc final.docx
- A New Probabilistic Transformation in Generalized Power Space
- Which University in USA can you study Computer Science in?
- EMEX 10 - Dr. Alex Hankey
- Base Conversion
- MCQ1 DC Number Systems and Codes
- Data Form
- Uganda Mathematics Scheme for S2
- Sc0x w1l2 Clean

You are on page 1of 4

This is a closed-book quiz, except that three 8.5 11 sheets of notes are allowed. There are three problems on the quiz of approximately equal value. The problems are not necessarily in order of diculty. A correct answer does not guarantee full credit and a wrong answer does not guarantee loss of credit. You should concisely indicate your reasoning and show all relevant work. The grade on each problem is based on our judgment of your level of understanding as reected by what you have written. If we cant read it, we cant grade it. If you dont understand a problem, please ask. Calculators are allowed, but probably wont be useful

Cite as: Robert Gallager, course materials for 6.450 Principles of Digital Communications I, Fall 2006. MIT OpenCourseWare (http://ocw.mit.edu/), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Problem Q1.1 (35 points) Consider a quaternary-valued source governed by a four-state Markov model that is op erating in its steady state.

As shown in Figure 1, when in a state i, a transition to state [(i + 1) mod 4] occurs with

probabity p, where p is a parameter, and such a transition causes a source symbol [(i + 1)

mod 4] to be produced; when no transition occurs, the source symbol i is produced.

(a) Suppose the memory in the source is ignored and a simple Human code is used to code each symbol. What average rate R1 in bits per source symbol is the source compressed to? (b) Determine the average rate R in bits per source symbol achievable by the best code for this source if the memory is taken into account. Express your answer in terms of the binary entropy function HB () = log2 () (1 ) log2 (1 ). Is there a value of p for which your answer is close to that of part (a)? Explain. (c) Suppose the source is partitioned into consecutive length-2 blocks, inter-block statisti cal dependencies are ignored, and a 2-to-variable length code of mininum expected length is used, which exploits the intra-block dependencies. The achievable average rate in bits per source symbol in this case is guaranteed to be at least as low as R2 = aHB (b) + c, where HB () is again the binary entropy function. Determine a, b, and c, expressing your answers in terms of the parameter p.

1;1p 2;1p

2;p

1;p

3;p

0 0;1p

0;p 3 3;1p

Cite as: Robert Gallager, course materials for 6.450 Principles of Digital Communications I, Fall 2006. MIT OpenCourseWare (http://ocw.mit.edu/), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Problem Q1.2 (35 points) Consider encoding the set of positive integers and assume that a probability distribution on these integers satises pi > pj for i < j. The table below gives the binary representa tion, the unary code and the unary-binary code for the rst few positive integers. (Recall that the unary-binary code is used in LZ77. This code works by viewing an integer j as a pair of integers (n, m) where n = log j + 1 and m = j 2n1 . Then j is mapped into the unary code for n concatenated with the n 1 bit binary representation for m.) j 1 2 3 4 5 6 binary rep. unary code unary-binary code 1 10 11 100 101 110 1 01 001 0001 00001 000001 1 010 011 00100 00101 00110

(a) Does the binary representation constitute a prex-free code? Explain. How about the unary and unary-binary codes? (b) Show that if an optimum (in the sense of minimum expected length) prex-free code is chosen for any given pmf (subject to the condition pi > pj for i < j), the code word lengths satisfy li lj for all i < j. Use this to show that for all j 1 lj log j + 1 (c) The asymptotic eciency of a prex-free code for the positive integers is dened to be limj log j . What is the asymptotic eciency of the unary-binary code? lj (d) Explain how to construct a prex-free code for the positive integers where the asymp totic eciency is 1. Hint: Replace the unary code for the integers n = log j + 1 in the unary-binary code with a code whose length grows more slowly with increasing n.

Cite as: Robert Gallager, course materials for 6.450 Principles of Digital Communications I, Fall 2006. MIT OpenCourseWare (http://ocw.mit.edu/), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].

Problem Q1.3 (True or False) 30 points For each of the following, state whether the statement is true or false and briey indicate your reasoning. No credit will be given without a reason. (a) Suppose X and Y are binary valued random variables with probability mass function given by pX (0) = 0.2, pX (1) = 0.8 pY (0) = 0.4 and pY (1) = 0.6. The joint probability mass function that maximizes the joint entropy H(X, Y ) is given by pX,Y (, ) X =0 X=1 Y =0 0.08 0.32 Y =1 0.12 0.48 (b) For a discrete memoriless source X with alphabet X = {1, 2, . . . , M }, let Lmin,1 , Lmin,2 , and Lmin,3 be the normalized average length in bits per source symbol for a Human code over X , X 2 and X 3 respectively. Then there exists a specic probability mass function 2 for source X for which Lmin,3 > 3 Lmin,2 + 1 Lmin,1 . 3 (c) Assume that a continuous valued rv Z has a probability density that is 0 except over the interval [A, +A]. Then the dierential entropy h(Z) is upper bounded by 1 + log2 A. Also h(Z) = 1 + log2 A if and only if Z is uniformly distributed between A and +A.

- Annual Ex001Uploaded byVinod Charles
- 1-s2.0-S003448771000025X-main.pdfUploaded byNelson Sandoval
- DEACON, T. W. Steps to a Science of BiosemioticsUploaded byLuís Filho
- A Practical Phyiscal Random Number GeneratorUploaded byΟλυμπίδης Ιωάννης
- Homologous Hierarchy Modeling Exciting Logics ArchitecturesUploaded bysfofoby
- Data Representation Worksheet 4 ImagesUploaded byUsman Malik
- Primer_Self_Instruction_Manual_r2.5.pdfUploaded byc350361
- Multicriteria Decision Making Using Double Refined Indeterminacy Neutrosophic Cross Entropy and Indeterminacy Based Cross EntropyUploaded byAnonymous 0U9j6BLllB
- Week05_L02-3Uploaded bySantiago Mejia
- Math133 Unit 5 - how to prepareUploaded byReuben Keith
- ERIC_ED143516-Calculus-Part II.pdfUploaded bybhulook
- crc final.docxUploaded byrachna009
- A New Probabilistic Transformation in Generalized Power SpaceUploaded byMia Amalia
- Which University in USA can you study Computer Science in?Uploaded byStudy Path
- EMEX 10 - Dr. Alex HankeyUploaded byBiofield Sciences
- Base ConversionUploaded byJamshed Ali
- MCQ1 DC Number Systems and CodesUploaded byarijitlgsp
- Data FormUploaded byriki2692
- Uganda Mathematics Scheme for S2Uploaded byVictor Omondi
- Sc0x w1l2 CleanUploaded byHaseeb Rehman
- README.txtUploaded byBenjamin Culkin
- 03. LogarithmsUploaded byDon Kabute
- Logical OperatorsUploaded bySwetha Nallapati
- It Lecture Notes2014Uploaded byRasha Ziyad
- LIS1-A.pdfUploaded byToshko Andreev
- MIT2_017JF09_design.pdfUploaded bypen2trinity3200
- Pra 3 IP AddressingUploaded byModi Mitulkumar
- Addmaths AssignmntUploaded byaeril72
- Learning CurvesUploaded byBarby Angel
- CTW_OpenEconAppendixUploaded byRodrigoOdónSalcedoCisneros

- Rm Chapter 1Uploaded byBrandon Ayala
- 23b8563760685fae18fea95b388f930edafaa7deUploaded byapi-127299018
- fd35a5c22ca4c8787de76166eded73a103ad02afUploaded byapi-127299018
- 9466be5cdea34cb0098f8e82846e2268fc3aca7dUploaded byapi-127299018
- 84cc54402d20f47cfb17a201658f04d3414edd9bUploaded byapi-127299018
- 244c1fc1f3981944e4c476320783462e2e050155Uploaded byapi-127299018
- 244c1fc1f3981944e4c476320783462e2e050155Uploaded byapi-127299018
- 244c1fc1f3981944e4c476320783462e2e050155Uploaded byapi-127299018
- 244c1fc1f3981944e4c476320783462e2e050155Uploaded byapi-127299018
- 244c1fc1f3981944e4c476320783462e2e050155Uploaded byapi-127299018
- 244c1fc1f3981944e4c476320783462e2e050155Uploaded byapi-127299018
- 702a75fe02ff62e5ae5b74646d18da512a724d02Uploaded byapi-127299018
- b151d72e4b9391412234cd9302dd07e679cc861aUploaded byapi-127299018
- f7895136c515d2e469f0a12e5c6c54190d8b4be1Uploaded byapi-127299018
- 0700244a1fd879268cc3abce8d4f02a44526f738Uploaded byapi-127299018
- 196040ff93bae9c9df8453dd90baad58c99544feUploaded byapi-127299018
- 196040ff93bae9c9df8453dd90baad58c99544feUploaded byapi-127299018
- 196040ff93bae9c9df8453dd90baad58c99544feUploaded byapi-127299018
- 196040ff93bae9c9df8453dd90baad58c99544feUploaded byapi-127299018
- 196040ff93bae9c9df8453dd90baad58c99544feUploaded byapi-127299018
- 196040ff93bae9c9df8453dd90baad58c99544feUploaded byapi-127299018
- 196040ff93bae9c9df8453dd90baad58c99544feUploaded byapi-127299018
- 196040ff93bae9c9df8453dd90baad58c99544feUploaded byapi-127299018
- 196040ff93bae9c9df8453dd90baad58c99544feUploaded byapi-127299018
- UntitledUploaded byapi-127299018
- fd35a5c22ca4c8787de76166eded73a103ad02afUploaded byapi-127299018
- fd35a5c22ca4c8787de76166eded73a103ad02afUploaded byapi-127299018
- 125c5da1e1b8536345a1c6a9876c51b5653bed34Uploaded byapi-127299018
- 187e9e5268ed74af09b10b60a622384cec2efa74Uploaded byapi-127299018
- b151d72e4b9391412234cd9302dd07e679cc861aUploaded byapi-127299018

- Algebra Question Paper 2Uploaded byAMIN BUHARI ABDUL KHADER
- Reviewing the Multiobjective Optimization Package of modeFrontier in Energy Sector.pdfUploaded byAmir Mosavi
- Dynamic Modelling and Advanced Control of Air Conditioning and Refrigeration SystemsUploaded bymalj1811
- MS5837-30BAUploaded byAbdelaziz Abdelghany
- DWTUploaded byhandaayush
- MCAMX5 Quick Ref CardUploaded bytobich
- EconomicsUploaded bydipesh_tamrakar
- Field Symbols NewUploaded byJaspal Kumar
- Design, Modelling and Implementation of Interleaved Boost DC-DC ConverterUploaded byInternational Journal of Innovative Science and Research Technology
- Summary - Strategic Database Marketing - Arthur M. HughesUploaded byjkivit
- Torque and speed harmonic analysis of a PWM CSI-fed induction motor driveUploaded byvalentinmuller
- Optics - Benjamin Crowell 5 of 6Uploaded byNige Danton
- Ambros Safety evaluation of Czech roundabouts.pdfUploaded byramadan1978
- Ruggeri-EMBC03-Poster.pdfUploaded byBalaji Narayanan N
- GCSE Maths 306547 Unit 2 Stage 1 FoundationUploaded bygcsemathstutor
- Thesis Carl1Uploaded byhajmunnisa
- algebra 1 syllabus ths - castaUploaded byapi-268317138
- 29 SaratogaUploaded byCork Graham
- HP-DSA Aplication Note 243-1Uploaded byAndres Felipe Loaiza Patiño
- Module 1Uploaded bydevraj subedi
- Design Optimization of a Metallic Package.pdfUploaded byerpixaa
- System Setup Mrs6101GB.0.UUploaded byH_JIMENEZ9934
- Gas Laws Animation Software - Ideal Gas Law Equation 2.PDFUploaded byGoldenKstar
- Contabilidade de Custos RealUploaded byWhtiaker
- itp120chapt192009binaryinputoutput-091011054706-phpapp01.pptUploaded byVikas Nagare
- Physics ReviewerUploaded byChristina de Sena-Que
- mech_sem6Uploaded byNidhi Gandhi
- Joseph Newman the Energy MachineUploaded bykishbud
- gcalUploaded byLucas
- CE 632 Earth Pressure PPTUploaded byBala Sutharshan