Professional Documents
Culture Documents
Abstract— In this paper extrinsic information transfer (EXIT) channel. This method is an extension of [9] to non-binary
charts are proposed to design non-binary low-density parity-check LDPC codes. To the extent of the authors’ knowledge EXIT
(LDPC) codes for the AWGN channel. A new metric is presented charts have not been applied to the analysis or design of non-
to describe the mutual information of the non-binary messages.
The a priori information is modelled using a Gaussian mixture binary LDPC codes. Good codes are designed by matching the
distribution. Analytical expressions are given for the EXIT curves EXIT curves of the variable and check node decoders, which
of the variable and check node decoders for both regular and is orders of magnitude more efficient than the design approach
irregular LDPC codes. The analytical expressions are shown employed in [11].
to agree well with simulation results. Finally, by matching the This paper is organised as follows. In section II the encoder,
variable and check node EXIT curves, it is shown that good non-
binary LDPC can be designed for the AWGN channel. channel model and decoder are described. EXIT charts for
regular LDPC codes are presented in section III where a
Index Terms— Code optimization, iterative decoding, low- model for the a priori information is proposed and closed form
density parity-check (LDPC) codes, mutual information.
expressions for the EXIT curves are given. This method is then
extended to irregular LDPC codes in section IV. In section V
I. I NTRODUCTION
non-binary LDPC codes are designed for the AWGN channel.
Binary low-density parity-check (LDPC) codes, originally Finally, conclusions are drawn in section VI.
proposed by Gallager [1] in 1963, have recently been redis-
covered [2], [3]. Irregular LDPC codes [4] have been shown II. S YSTEM M ODEL
to approach the Shannon limit and outperform the powerful LDPC codes are linear block codes which are defined by a
turbo codes [5] in the AWGN channel for long frame lengths. very sparse parity check matrix H with dimensions m × n.
Binary LDPC codes can be designed using density evolution With non-binary LDPC codes the elements of H are defined
[4], [6] which involves determining the code threshold by iter- over finite field GF (q = 2p ). Using Gaussian elimination H
atively propagating the message densities through the decoder. can be transformed into the k × n generator matrix G where
This method is computationally intensive, an issue which was k = n − m.
addressed in [7]. Recently a more efficient design method, The encoding of non-binary LDPC codes is similar to the
based on extrinsic information transfer (EXIT) charts [8], was encoding of binary LDPC codes. The input frame u consisting
proposed in [9]. Here optimal codes were designed by matching of K information bits is converted to a frame v of length
the EXIT curves of the variable and check node decoders. k = K/p symbols from GF (q). This frame is encoded to
Non-binary LDPC codes [10], [11] are defined over the give codeword w = Gv of length n symbols. The rate of this
finite field GF (q = 2p ) and have demonstrated improved encoder is R = k/n. The codeword w is converted to a binary
performance over binary LDPC codes of the same frame length codeword c of length of N = np bits.
and rate [11]. By encoding over GF (q) each parity check The encoded bits are BPSK modulated and transmitted over
becomes more complex while the decoding remains tractable. the AWGN channel. The received signal after matched filtering
Increasing the field size of a LDPC code is comparable to is expressed as
increasing the memory of a convolutional encoder. Non-binary rt = (2ct − 1) + ηt . (1)
LDPC codes have recently been applied to the multiple- Here t = {1, . . . , N } and ηt is the AWGN noise which
input, multiple-output (MIMO) channel [12] and the magnetic is modelled as an independent, zero-mean Gaussian random
recording channel [13] with promising results. The design of sequence with variance σn2 = 1/(2REb /N0 ) where Eb /N0 is
non-binary LDPC codes has only been addressed in [11] where the signal-to-noise ratio.
the average number of iterations taken to successfully decode a The LDPC decoder then calculates the most likely estimate
frame was proposed to assess the effectiveness of a given code. of the original codeword given the received information r. The
This technique worked well but is extremely computationally decoding of non-binary LDPC codes is not equivalent to the
intensive, especially for long frames. decoding of binary codes as the non-binary decoder operates
In this paper a new method, based on EXIT charts, is on the symbol level, not on the bit level. In this paper decoding
proposed to design non-binary LDPC codes for the AWGN is performed using the Fourier transform algorithm of [11]. To
This work is partially supported by Alcatel and Telkom SA as part of the avoid numerical instability issues this algorithm is implemented
Centre of Excellence Programme. in the log domain [13].
decoder can be determined without the need to run lengthly max(xj ) = log exj . (3)
j
BER simulations. j=1
In this section a new metric is presented to calculate the This can be computed recursively by setting δ1 = x1 and for
mutual information of the non-binary messages. A decoder j = {2, . . . , J}
model is proposed for both the VND and CND and analytical
expressions are presented for the corresponding EXIT curves. δj = max(δj , δj−1 ) + ln [1 + exp(−|δj − δj−1 |)] . (4)
Both the model and analysis are compared to simulation results On the final iteration δJ is the solution to (3).
to assess their accuracy for GF (4) LDPC codes. However, the
The pdfs p(λ|X = 0) and p(λ|X = 1) are obtained from
model and analytical expressions presented are also valid for
the histogram of λ. The mutual information between the bits
GF (q) LDPC codes.
X and the bitwise LLRs λ = B(L) is [8]
1 ∞
A. Calculation of the Mutual Information I(X; B(L)) = · p(λ|X = x) (5)
2 x=0,1 −∞
With binary LDPC codes a single message is passed along
2p(λ|X = x)
each edge of the bipartite graph while with non-binary codes · log2 dλ.
p(λ|X = 0) + p(λ|X = 1)
q messages are passed along each edge. EXIT charts have
been considered for non-binary convolutional codes [14], [15]
where the calculation of the mutual information required the B. EXIT Curves for the Variable Node Decoder
1) Decoder Model: Consider an arbitrary variable node of
1 In a serially concatenated decoder interleavers are present. degree dv which receives dv messages Lai,iv from the CND and
one message Lach from the channel. The variable node com-
putes the dv output messages as follows for i = {1, . . . , dv }
I ch I ev I ac I ec and a ∈ GF (q)
VND CND
Lai,ov = Lach + Lak,iv . (6)
I av k=i
P(L0iv | X=x)
P(Liv | X=x)
4 x=3 0.15
3
0.1
1
2
0.05
1
0 0
−15 −10 −5 0 5 10 15 −15 −10 −5 0 5 10 15
L0iv L1iv
0.2 0.2
P(L2iv | X=x)
P(Liv | X=x)
0.15 0.15
0.1 0.1
3
0.05 0.05
0 0
−15 −10 −5 0 5 10 15 −15 −10 −5 0 5 10 15
L2iv L3iv
Fig. 2. Simulated (markers) and modelled (solid lines) pdfs of the VND input messages for a regular (3,6) code over GF (4) after 8 decoding iterations with
R = 1/2 and Eb /N0 = 2 dB.
representation of x and l = {1, . . . , p}. Each channel message input messages of the VND:
Lach is calculated from the received signals r1 , . . . , rp where
0, if a = 0
µ + n0i + nai , if x = a, a = 0
Lach = 2rl /σn2 . (7) Lai,iv (σ) = (10)
−µ + n0i + nai , if x = 0, a = 0
l: al =1 0
ni + nai , otherwise
Here rl = (2xl − 1) + ηl (1) and al denotes the lth bit in the Here µ = σ 2 /2 and n0i and nai are Gaussian distributed with
binary representation of a. It is more convenient to express (7) mean zero and variance σ 2 /2 which ensures the variance of
in the form Lai,iv (σ) is σ 2 . The random variable n0i is introduced to model
the correlation between the messages L1i,iv , . . . , Li,iv
q−1
.
Lach = µch · (2xl − 1) + nch,l (8) The proposed model for the input messages is compared to
l: al =1 simulation results in Fig. 2. A regular (3,6) LDPC code over
GF (4) is used where N = 12 000 bits, R = 1/2, Eb /N0 =
2
where µch = σch /2 and the nch,l are Gaussian distributed with 2 dB and the decoder is allowed to run for 8 iterations. It
mean zero and variance can be seen that the proposed model, based on the Gaussian
Eb distribution, provides a good fit to the simulated results.
2
σch = 8R . (9) 2) Analytical Expressions: For binary LDPC codes the
N0
channel messages, input messages and output messages of the
The input messages of the VND are more difficult to model VND can all be modelled using a Gaussian distribution while
and their distribution was carefully studied by simulation. A for non-binary LDPC codes these messages are modelled using
number of observations were made. Firstly, the input messages different Gaussian mixture distributions given by (8), (10) and
are independent. Secondly, the messages approach a Gaus- (6) respectively. For this reason the analytical expressions for
sian like distribution with an increasing number of iterations the EXIT curves presented in [9] do not apply directly to non-
where the mean and the variance are related by µ = σ 2 /2. binary LDPC codes. Therefore, two functions
This approximation has been successfully made in [7], [9] Jv (σ) = I [X; B(Lch + Liv (σ))] (11)
for the case of binary LDPC codes. Thirdly, the messages
Jc (σ) = I [X; B(Liv (σ))] (12)
L1i,iv , . . . , Li,iv
q−1
are correlated due to the normalisation of the
VND output messages in the iterative decoder [11]. Based on are defined which relate the standard deviation of the pdf
these observations the following model is proposed for the to the mutual information of the messages. In [9] only one
For computer implementation the functions Jv (σ) and Here Pi (Li,ic ) is the permutation of Li,ic by hi and F(x)
Jc−1 (Iav ) can be computed using a lookup table or the curve denotes the Fourier transform of x as described in [11], [13].
fitting approach suggested in [9]. The notation L(s) and L(m) denote the sign and magnitude
of L.
3) Comparison with Simulation Results: In Fig. 3, EXIT
By simulating the distribution of the CND input messages it
curves for a number of variable node degrees are presented for
was observed that the messages are independent and their pdf
a GF (4) code with R = 1/2 and Eb /N0 = 2 dB. The EXIT
is a composite of the pdfs of the channel and input messages of
curves obtained from the model (6) and the analysis (13) are
the VND. It is important to note that each VND input message
compared with simulation results for a “free-running” decoder
corresponds to the same coded symbol x while each CND input
with a frame length of N = 12 000 bits. For the decoder model
message corresponds to a possibly different coded symbol xi .
the EXIT curve is averaged over 106 messages while for the
Based on these observations the following model is proposed
simulation the snapshot trajectory [8] for a single frame is
for the CND input messages:
taken. It is shown that there is good agreement between the
model, analysis and simulation. Lai,ic (σ) = Lai,c (σ) + µch · (2xi,l − 1) + nch,i,l (18)
l: a =1
l
0, if a = 0
µ + n0i + nai , if xi = a, a = 0
Lai,c (σ) = (19)
C. EXIT Curves for the Check Node Decoder
−µ + n0i + nai , 0
if xi = 0, a =
0
ni + nai , otherwise
1) Decoder Model: Consider an arbitrary check node of
where xi,l is the lth bit of xi , µ = σ 2 /2, nch,i,l and nai are
degree dc which receives dc messages Lai,ic from the VND. 2
Gaussian distributed with mean zero and variance σch and σ 2 /2
Define h as the vector of parity checks associated with this
respectively.
check node and x as the vector of encoded symbols which
2) Analytical Expressions: In [9] the EXIT curve of the
satisfy these checks. These length dc vectors contain elements
CND was expressed in terms of the EXIT curve of the VND
from GF (q) and satisfy the equality h x = 0 where the
based on the duality property of [17]. This approximation is not
arithmetic is performed over GF (q).
feasible for GF (q > 2) because the input and output messages
The check node computes the dc output messages Lai,oc as cannot be described by the same Gaussian mixture distribution.
However, the EXIT curve of a degree dc check node can be
well approximated by
1
Iec (Iac , dc ) = Iacα(dc )·Iac +β(dc ) (20)
5
=1
0.95 =9
d
v
d v
=5 where the constants α(dc ) and β(dc ) are dependent on the
0.9 d v =4
dv 3
0.85 d v= check node degree, field size and SNR. These constants can be
2 calculated by fitting (20) to the CND EXIT curves of interest.
d v=
0.8
3) Comparison with Simulation Results: EXIT curves for
ev
0.7 0.7
0.6 0.6
ac
I ,I
ec
0.5 0.5
I
ev
0.4 =3 0.4
d c
0.3 0.3
=6
=9
0.2 0.2
d
c
15
Trajectory
d
c
d=
0.1 0.1 VND − Analysis
c
CND − Analysis
0 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Iac Iav , Iec
Fig. 4. EXIT curves for the check node decoder of a GF (4) code with Fig. 5. EXIT chart for the irregular GF (4) code of [11] with R = 1/2 and
R = 1/2 and Eb /N0 = 2 dB. Eb /N0 = 2 dB.
[17], [18]. Using the proposed metric this was found to be true dB because at this SNR the channel capacity Jv (0) ≈ 0.524 is
for non-binary LDPC codes as well. fractionally larger than R. The check node degree dc is chosen
The averaging is done with respect to the edge fractions, so that the CND curve is a small distance from the y axis at
not the node fractions, because it is the edges which carry the Iac = 0.5. In this way, Iec is not too small on the first iteration
decoder messages. Let λi (ρi ) denote the fraction of variable which gives the decoder a good start. Due to the constraints
(check) nodes of degree i and let λei (ρei ) denote the fraction on the variable and check node degrees only one λi can be
of edges connected to a variable (check) node of degree i. chosen freely. In practice it was found that best results were
The conversion from node fractions to edge fractions is given obtained by minimising the variance of the distance
by λei = iλiλ i
and ρei = iρiρ
i
. The VND and CND EXIT
i i i i D(Iav ) = Iec [Iev (Iav )] − Iav (23)
curves for an irregular LDPC code are calculated as follows:
between the VND and CND EXIT curves. The EXIT curves
Iev (Iav ) = λei · Iev (Iav , i) (21) can be efficiently calculated using (21) and (22). The optimised
i
profiles for GF (4) and GF (8) LDPC codes are given in Table
Iec (Iac ) = ρei · Iec (Iac , i) (22) I and the corresponding EXIT chart for the GF (4) code is
i presented in Fig. 6.
In Fig. 5 the analytical EXIT curves are compared to the The performance of the optimised codes is evaluated in the
average decoding trajectory obtained from simulation where AWGN channel. Their performance is compared to the GF (4)
the axes for the CND EXIT curve are swapped. The irregular code of [11] which is denoted by ‘Davey’. Simulation results
GF (4) code from [11] is used which is described by the profile are presented in Fig. 7 for a frame length of N = 60 000
ρ7 = 0.341, ρ8 = 0.659, λ2 = 0.478633, λ3 = 0.4085, λ8 = bits since it is for this frame length that the code from [11] is
0.000067, λ11 = 0.045233, λ17 = 0.067567. The trajectory is optimised. The number of decoding iterations is set to 100. It
averaged over 20 frames, N = 12 000 bits and Eb /N0 = 2 dB. can be seen that the codes optimised using the proposed EXIT
It is observed that the analysis provides a good approximation chart approach achieve superior performance to the code from
of the decoding trajectory. [11] even though the optimised codes use fewer variable and
check node degrees. This indicates that the proposed method
V. D ESIGN OF N ON -B INARY LDPC C ODES is a promising approach for optimising non-binary LDPC
codes. Furthermore, the approach used here is considerably less
In [17] it was shown that to design LDPC codes which
computationally intensive, requiring only function evaluations
approach capacity the VND and CND EXIT curves must be
and table lookups, than the simulation based method used
matched exactly. This approach was applied in [9] to design
in [11] which requires averaging the number of decoding
LDPC codes for both the AWGN and MIMO channels and in
iterations.
[19] to design repeat-accumulate codes for the MIMO channel.
This approach is employed here to design non-binary LDPC
TABLE I
codes for the AWGN channel.
D EGREE PROFILES FOR RATE 1/2 LDPC CODES , OPTIMISED AT 0.5 D B.
The design approach now described closely follows that of
[9]. To limit the search space, codes with 3 variable node
degrees and 1 check node degree are considered. The code rate Code Profile
is R = 1/2 and the EXIT curves are matched at Eb /N0 = 0.5 GF (4) ρ7 = 1, λ2 = 0.50637, λ3 = 0.36783, λ11 = 0.12580
GF (8) ρ6 = 1, λ2 = 0.62763, λ3 = 0.26776, λ9 = 0.10461
0.5 matrices,” IEEE Trans. Inform. Theory, vol. 45, pp. 399–431, Mar. 1999.
[4] T. J. Richardson, M. A. Shokrollahi and R. L. Urbanke, “Design of
0.4 capacity-approaching irregular low-density parity check codes,” IEEE
0.3 Trans. Inform. Theory, vol. 47, pp. 619–637, Feb. 2001.
[5] C. Berrou, A. Glavieux and P. Thitimajshima, “Near Shannon limit error-
0.2 correcting coding and decoding: Turbo codes,” in Proc. IEEE ICC ’93,
VND − Analysis
Geneva, Switzerland, May 1993, pp. 1064–1070.
0.1
CND − Analysis [6] T. J. Richardson and R. L. Urbanke, “The capacity of low-density parity-
0 check codes under message-passing decoding,” IEEE Trans. Inform.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Theory, vol. 47, pp. 619–637, Feb. 2001.
I ,I
av ec [7] S. Chung, T. J. Richardson and R. L. Urbanke, “Analysis of sum-
product decoding of low-density parity-check codes using a Gaussian
Fig. 6. EXIT chart for the GF (4) code from Table I at Eb /N0 = 0.5 dB. approximation,” IEEE Trans. Inform. Theory, vol. 47, pp. 657–670, Feb.
2001.
10
0 [8] S. ten Brink, “Convergence behaviour of iteratively decoded parallel
concatenated codes,” IEEE Trans. Commun., vol. 49, pp. 1727–1737,
Oct. 2001.
−1
10 [9] S. ten Brink, G. Kramer and A. Ashikhmin, “Design of low-density
parity-check codes for modulation and detection,” IEEE Trans. Commun.,
−2 vol. 52, pp. 670–678, April 2004.
10
[10] M. C. Davey and D. MacKay, “Low-density parity-check codes over
GF (q),” IEEE Commun. Lett., vol. 2, pp. 165–167, June 1998.
BER
10
−3 [11] M. C. Davey, Error-correction using low-density parity-check codes,
Ph.D. dissertation, Univ. of Cambridge, Cambridge, U.K., Dec. 1999.
[12] G. Li, I. J. Fair and W. A. Krzymień, “Low density parity check codes for
−4
10 space-time wireless transmission,” in Proc. Intl. Conference on Wireless
Communications, Calgary, Canada, July 2002.
−5 [13] H. Song and J. R. Cruz, “Reduced-complexity decoding of Q-ary LDPC
10 GF(4) codes for magnetic recording,” IEEE Trans. Magn., vol. 39, pp. 1081–
GF(8)
Davey
1087, Mar. 2003.
10
−6 [14] A. Grant, “Convergence of non-binary iterative decoding,” in Proc. IEEE
0.4 0.45 0.5 0.55 0.6 GLOBECOM ’01, vol. 2, Nov. 2001, pp. 1058–1062.
Eb / N0 (dB) [15] C. Schlegel and A. Grant, “Concatenated space-time coding,” in Proc.
IEEE PIMRC ’01, Sep. 2001.
Fig. 7. Simulation results for the optimised codes from Table I. [16] S. Benedetto, D. Divsalar, G. Montorsi and F. Pollara, “Serial concate-
nation of interleaved codes: Performance analysis, design and iterative
decoding,” IEEE Trans. Inform. Theory, vol. 44, pp. 909–926, May 1998.
[17] A. Ashikhmin, G. Kramer and S. ten Brink, “Extrinsic information
VI. C ONCLUSIONS transfer functions: A model and two properties,” in Proc. Conf. Inform.
Sciences and Systems, Princeton, NJ, Mar. 2002, pp. 742–747.
[18] M. Tüchler and J. Hagenauer, “EXIT charts of irregular codes,” in Proc.
2002 Conf. Inform. Sciences and Systems, Princeton, NJ, Mar. 2002, pp.
This paper proposed the use of EXIT charts to design non- 748–753.
binary LDPC codes for the AWGN channel. Decoder models [19] S. ten Brink and G. Kramer, “Design of repeat-accumulate codes for
iterative detection and decoding,” IEEE Trans. Signal Processing, vol.
were developed for both the VND and CND where the a 51, pp. 2764–2772, Nov. 2003.
priori information was modelled using a Gaussian mixture
distribution. Analytical expressions were given for the VND
and CND EXIT curves for both regular and irregular codes.
The EXIT curves obtained from the model and analysis closely
matched simulation results. It was shown that by matching the
EXIT curves of the VND and CND, good non-binary LDPC
codes could be designed which achieved superior performance
to the code optimised in [11]. Furthermore, the proposed EXIT
chart approach is considerably less computationally intensive
than the simulation based approach used in [11].