Hamming Codes

How to Calculate them.

How to use them for error detection.
How to use them for single bit error correction.

R. Jesse Chaney 1 CS271 – Oregon State University

 Outline

• History
• Calculation
• Using Hamming codes for single bit error correction
• data bits
• parity bits

Heavily drawn from: http://users.cis.fiu.edu/~downeyt/cop3402/hamming.html

R. Jesse Chaney 2 CS271 – Oregon State University

 History: briefly
Cycle time in
seconds!!!

Richard Hamming worked at Bell Labs in the 1940s on the Bell Model V
computer, an electromechanical relay-based machine with cycle times in
seconds. Input was fed in on punched cards, which would invariably have read
errors. During weekdays, special code would find errors and flash lights so the
operators could correct the problem. During after-hours periods and on
weekends, when there were no operators, the machine simply moved on to
the next job.
Hamming worked on weekends, and grew increasingly frustrated with having
to restart his programs from scratch due to the unreliability of the card reader.
Over the next few years, he worked on the problem of error-correction,
developing an increasingly powerful array of algorithms. In 1950, he published
Punch card (not actual size).
what is now known as Hamming Code, which remains in use today in
applications such as ECC memory.
Credit to Wikipedia
R. Jesse Chaney 3 CS271 – Oregon State University
 Calculation

There is a lot of theory behind how the Hamming codes work. There are
numerous books and web pages that describe the theory.

I’m going to go very light on the theory and just jump into the calculation and
use of Hamming codes to correct single bit errors.

Okay, I think we are done with the theory part now.

That aside, though there a large number of examples Hamming Code use on
the internet, I never found a step-by-step example that quite fit with how we
have described Hamming Codes in this class.

R. Jesse Chaney 4 CS271 – Oregon State University

 Start with a number

We are going to start with a number. We’ll use 8-bit numbers (0-255)
for this example. We could use any number, but I’ll select:
154

There is nothing special about 154. It happens to be large enough that

it uses the left most bit in its 2-byte representation as a binary number.

R. Jesse Chaney 5 CS271 – Oregon State University

 Convert to binary

Next, we are going to convert the number 154 into its unsigned binary representation.

154 1 0 0 1 1 0 1 0

The number 154 requires 2 bytes. So that it makes it a little easier to

distinguish the bits in the left byte from the bits in the right byte, I’ve
colored them. All together, I’ll call these bits the data bits.

R. Jesse Chaney 6 CS271 – Oregon State University

 Don’t forget the (+ 1) part.

Introduce the parity bits

Now, we’ll introduce the location of the parity bits.
For 8 bits of data (2 bytes), the Hamming code must contain (log2 n) + 1 or parity bits. For
our example, 8 bits of data, that’s 4 parity bits.
A parity bit, or check bit, is a bit
added to a string of binary code
154 1 0 0 1 1 0 1 0 Data bits that indicates whether the
number of bits in the string with
the value 1 is even or odd.

Wikipedia

_ _ 1 _ 0 0 1 _ 1 0 1 0 Data bits and parity bits

1 2 3 4 5 6 7 8 9 10 11 12

Parity bits, for Hamming encoding

We number the bits for the now 12 bits (8 for data and 4 for parity) from left to right starting
at 1. The parity bits are assigned to the locations in the 12 bits based on the powers of 2: 20
= 1, 21 = 2, 22 = 4, 23 = 8. If we had more than 8 bits of data, this would continue. Correctly
placing the parity bits within the 12 bits is very important.

R. Jesse Chaney 7 CS271 – Oregon State University

 Parity bit 1

Now we’ll calculate the value for the first parity bit, bit position 1.

Notice the mnemonic Do we want even parity or odd parity?

_ _ 1 _ 0 0 1 _ 1 0 1 0
1 2 3 4 5 6 7 8 9 10 11 12
even odd
Position 1: check 1 bit, skip 1 bit, _ _ 1 _ 0 0 1 _ 1 0 1 0 = 4 0 1
check 1 bit, skip 1 bit…
bits 1, 3, 5, 7, 9, 11, …

Now we’ll calculate the value for the first parity bit, bit position 1. Count all the data bits
from the list. In the example, the indicated data bits count to 4 (positions 3, 7, 9, and 11).

If we are interested in even parity, the parity bit (position 1) is set to a zero. If we are
interested in odd parity, then the parity bit (position 1) is set to a one.
R. Jesse Chaney 8 CS271 – Oregon State University
 Parity bit 2

Now we’ll calculate the value for the second parity bit, bit position 2.

Notice the mnemonic Do we want even parity or odd parity?

_ _ 1 _ 0 0 1 _ 1 0 1 0
1 2 3 4 5 6 7 8 9 10 11 12
even odd
Position 2: check 2 bits, skip 2 bits, _ _ 1 _ 0 0 1 _ 1 0 1 0 = 3 1 0
check 2 bits, skip 2 bits…
bits 2, 3, 6, 7, 10, and 11

Now we’ll calculate the value for the second parity bit, bit position 2. Count all the
data bits from the list. In the example, the indicated data bits count to 3 (positions 3,
7, and 11).

If we are interested in even parity, the parity bit (position 2) is set to a one. If we are
interested in odd parity, then the parity bit (position 2) is set to a zero.
R. Jesse Chaney 9 CS271 – Oregon State University
 Parity bit 3

Now we’ll calculate the value for the third parity bit, bit position 4.

Notice the mnemonic Do we want even parity or odd parity?

_ _ 1 _ 0 0 1 _ 1 0 1 0
1 2 3 4 5 6 7 8 9 10 11 12
even odd
Position 4: check 4 bits, skip 4 bits, _ _ 1 _ 0 0 1 _ 1 0 1 0 = 1 1 0
check 4 bits, skip 4 bits…
We run out of bits. If there were more than 12
bits 4, 5, 6, 7, 12 bits in our data, we’d keep adding.

Now we’ll calculate the value for the third parity bit, bit position 4. Count all the data
bits from the list. In the example, the indicated data bits count to a lonely 1 (position 7).

If we are interested in even parity, the parity bit (position 4) is set to a one. If we are
interested in odd parity, then the parity bit (position 4) is set to a zero.

R. Jesse Chaney 10 CS271 – Oregon State University

 Parity bit 4

We now calculate the value for the fourth parity bit (last in our example), bit position 8.

Notice the mnemonic Do we want even parity or odd parity?

_ _ 1 _ 0 0 1 _ 1 0 1 0
1 2 3 4 5 6 7 8 9 10 11 12
even odd
Position 8: check 8 bits, skip 8 bits, _ _ 1 _ 0 0 1 _ 1 0 1 0 = 2 0 1
check 8 bits, skip 8 bits… Once again, we run out of bits long before we
bits 8, 9, 10, 11, 12, … get to 8. If there were more than 12 bits in our
data, we’d keep adding.

Now we’ll calculate the value for the fourth parity bit (the last one in this example), bit
position 8. Count all the data bits from the list. In the example, the indicated data bits
count to 2 (positions 9 and 11).

If we are interested in even parity, the parity bit (position 8) is set to a zero. If we are
interested in odd parity, then the parity bit (position 8) is set to a one.
R. Jesse Chaney 11 CS271 – Oregon State University
 Pulling all the parity bits together
Now we’ll put all the parity bits together.

154 1 0 0 1 1 0 1 0 Data bits

_ _ 1 _ 0 0 1 _ 1 0 1 0
1 2 3 4 5 6 7 8 9 10 11 12

0 1 1 1 0 0 1 0 1 0 1 0 Even parity Hamming Code

1 0 1 0 0 0 1 1 1 0 1 0 Odd parity Hamming Code

Parity bits, for Hamming Code

R. Jesse Chaney 12 CS271 – Oregon State University

 Encoding a 32 bit number

A number requiring 8 bytes: 1,234,567,890

Data bits 0 1 0 0 1 0 0 1 1 0 0 1 0 1 1 0 0 0 0 0 0 0 1 0 1 1 0 1 0 0 1 0

_ _ 0 _ 1 0 0 _ 1 0 0 1 1 0 0 _ 1 0 1 1 0 0 0 0 0 0 0 1 0 1 1 _ 0 1 0 0 1 0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

Parity bits

R. Jesse Chaney 13 CS271 – Oregon State University

 Error correction
Now we’ve seen how to calculate a Hamming encoding, but what can we do with it?

We are going to take our example from before, the number 154, and introduce an
error into the stream. We’ll do 2 examples. The first will have an error in the data bits.
The second will have an error in the parity bits. For both examples, we’ll use even
parity encoding.

It is important to know that Hamming codes can only correct single bit errors. There
are other more powerful error correcting codes, but they require more bits and are
more complex.

R. Jesse Chaney 14 CS271 – Oregon State University

 Introduce an error in a data bit
To introduce the error, we’ll just toggle bit 12 from zero to one.

0 1 1 1 0 0 1 0 1 0 1 0 Even parity, (our favorite number, 154)

0 1 1 1 0 0 1 0 1 0 1 1 Corrupt a single bit, a data bit

1 2 3 4 5 6 7 8 9 10 11 12
In reality errors can come from many sources. One that is mentioned often in the web
pages is from computer memory. Another source of errors is corruption that can come
from line noise over networks.

Though statistically, data errors are rare, you want to detect and correct when they do
occur and when they occur in clusters (such as memory or disk errors) you need to
detect and correct before you just have all garbage.

R. Jesse Chaney 15 CS271 – Oregon State University

 Parity bit 1

Check parity the first parity bit, bit position 1.

Remember the mnemonic for parity

bit position 1?
0 1 1 1 0 0 1 0 1 0 1 1 Single error
check 1 bit, skip 1 bit, check 1 bit,
skip 1 bit, … 1 2 3 4 5 6 7 8 9 10 11 12

0 1 1 1 0 0 1 0 1 0 1 1 = 4 this is fine

Add all the values represented by the first parity bit, position 1. Count all the data bits
from the list. In the example, the indicated data bits count to 4 (positions 3, 7, 9, and
11). The value 4 is already even (for our even parity example). So, the single error
must not be within that group of bits.

R. Jesse Chaney 16 CS271 – Oregon State University

 Parity bit 2

Check parity the second parity bit, bit position 2.

Remember the mnemonic for

parity bit position 2?
0 1 1 1 0 0 1 0 1 0 1 1 Single error
check 2 bits, skip 2 bits,
check 2 bits, skip 2 bits, … 1 2 3 4 5 6 7 8 9 10 11 12

0 1 1 1 0 0 1 0 1 0 1 1 = 4 this is fine

Add all the values represented by the second parity bit, position 2. Count all the data
bits from the list. In the example, the indicated data bits count to 4 (positions 2, 3, 7,
and 11). The value 4 is already even (for our even parity example). So, the single error
must not be within that group of bits.

R. Jesse Chaney 17 CS271 – Oregon State University

 Parity bit 3

Check parity the third parity bit, bit position 4.

Remember the mnemonic for

parity bit position 4?

check 4 bits, skip 4 bits, check 0 1 1 1 0 0 1 0 1 0 1 1 Single error

4 bits, skip 4 bits
1 2 3 4 5 6 7 8 9 10 11 12

0 1 1 1 0 0 1 0 1 0 1 1 = 3 this is wrong

Add all the values represented by the third parity bit, position 4. Count all the data
bits from the list. In the example, the indicated data bits count to 3 (positions 4, 7, and
12). The number 3 is not even (for our even parity example). So, one of the bits from
the parity bit 3 collection must be wrong.

R. Jesse Chaney 18 CS271 – Oregon State University

 Parity bit 4

Check parity the fourth parity bit, bit position 8.

Remember the mnemonic

for parity bit position 8?
0 1 1 1 0 0 1 0 1 0 1 1 Single error
check 8 bits, skip 8 bits,
1 2 3 4 5 6 7 8 9 10 11 12 check 8 bits, skip 8 bits

0 1 1 1 0 0 1 0 1 0 1 1 = 3 this is wrong

Add all the values represented by the fourth parity bit, position 8. Count all the data
bits from the list. In the example, the indicated data bits count to 3 (positions 9, 11,
and 12). The number 3 is not even (for our even parity example). So, one of the bits
from the parity bit 4 collection must be wrong.

R. Jesse Chaney 19 CS271 – Oregon State University

 Putting it together

We’ve identified that 2 parity bits are wrong, 4 and 8.

Each bit position in can be described as a unique

sum of the parity bits. This is one of the simple
0 1 1 1 0 0 1 0 1 0 1 1 Single error beauties of the Hamming code.

1 2 3 4 5 6 7 8 9 10 11 12

0 1 1 1 0 0 1 0 1 0 1 1 = 3 this is wrong

0 1 1 1 0 0 1 0 1 0 1 1 = 3 this is wrong

Now that we’ve identified that there are 2 parity bits that are wrong (and which bits
Wrong parity bits
they are), how do we correct the error? We add the location of the wrong parity bits.
In this case: 4 + 8 = 12. It is bit 12 that is corrupted. So, instead of the received value
1, it should be a zero.
R. Jesse Chaney 20 CS271 – Oregon State University
 Introduce an error in a parity bit
To introduce the error, we’ll just toggle bit 4 from zero to one.

0 1 1 1 0 0 1 0 1 0 1 0 Even parity, 154

0 1 1 0 0 0 1 0 1 0 1 0 Corrupt a single bit, a data bit

1 2 3 4 5 6 7 8 9 10 11 12

Instead of corrupting a data bit, we’ll corrupt a parity bit for this example. We’ll
continue to use even parity.

R. Jesse Chaney 21 CS271 – Oregon State University

 Parity bit 1

Check parity the first parity bit, bit position 1.

Remember the mnemonic for parity bit

position 1?

0 1 1 0 0 0 1 0 1 0 1 0 Single error check 1 bit, skip 1 bit, check 1 bit, skip 1 bit
1 2 3 4 5 6 7 8 9 10 11 12

0 1 1 0 0 0 1 0 1 0 1 0 = 4 this is fine

Add all the values represented by the first parity bit, position 1. Count all the data bits
from the list. In the example, the indicated data bits count to 4 (positions 3, 7, 9, and
11). The value 4 is already even (for our even parity example). So, the single error
must not be within that group of bits.

R. Jesse Chaney 22 CS271 – Oregon State University

 Parity bit 2

Check parity the second parity bit, bit position 2.

Remember the mnemonic for parity bit

position 2?

0 1 1 0 0 0 1 0 1 0 1 0 Single error check 2 bits, skip 2 bits, check 2 bits, skip 2

bits
1 2 3 4 5 6 7 8 9 10 11 12

0 1 1 0 0 0 1 0 1 0 1 0 = 4 this is fine

Add all the values represented by the second parity bit, position 2. Count all the data
bits from the list. In the example, the indicated data bits count to 4 (positions 2, 3, 7,
and 11). The value 4 is already even (for our even parity example). So, the single error
must not be within that group of bits.

R. Jesse Chaney 23 CS271 – Oregon State University

 Parity bit 3

Check parity the third parity bit, bit position 4.

Remember the mnemonic for parity bit

position 3?

0 1 1 0 0 0 1 0 1 0 1 0 Single error check 4 bits, skip 4 bits, check 4 bits, skip 4

bits
1 2 3 4 5 6 7 8 9 10 11 12

0 1 1 0 0 0 1 0 1 0 1 0 = 1 this is wrong

Add all the values represented by the third parity bit, position 4. Count all the data
bits from the list. In the example, the indicated data bits count to 3 (position 7). The
number 1 is not even (for our even parity example). So, one of the bits from the parity
bit 3 collection must be wrong.

R. Jesse Chaney 24 CS271 – Oregon State University

 Parity bit 4

Check parity the fourth parity bit, bit position 8.

Remember the mnemonic for parity bit

position 4?

0 1 1 0 0 0 1 0 1 0 1 0 Single error check 8 bits, skip 8 bits, check 8 bits, skip 8

bits
1 2 3 4 5 6 7 8 9 10 11 12

0 1 1 0 0 0 1 0 1 0 1 0 = 2 this is fine

Add all the values represented by the fourth parity bit, position 8. Count all the data
bits from the list. In the example, the indicated data bits count to 2 (positions 9, 11).
The value 2 is already even (for our even parity example). So, the single error must not
be within this group of bits.

R. Jesse Chaney 25 CS271 – Oregon State University

 Putting it together

We’ve identified a single parity bit is wrong, bit 4.

Each bit position in can be described as a unique

sum of the parity bits. This is one of the simple
0 1 1 0 0 0 1 0 1 0 1 0 Single error beauties of the Hamming code.

1 2 3 4 5 6 7 8 9 10 11 12

0 1 1 0 0 0 1 0 1 0 1 0 = 1 this is wrong

Now that we’ve identified that there is 1 parity bit that is wrong (and which bit it is),
Wrong parity bit
how do we fix the error? Like before, we add the location of the wrong parity bits. In
this example, it’s just 1 bit: 4. It is bit 4 that is corrupted. So, instead of the received
value zero, it should be a one.
R. Jesse Chaney 26 CS271 – Oregon State University
 Conclusion

• We touched a little on the history of the Hamming Code.

• We’ve gone through how to create Hamming Code encoded
data, and
• We’ve gone through how to use Hamming Code parity bits to
correct single bit errors in data.

R. Jesse Chaney 27 CS271 – Oregon State University