0.12 Lab 8 - number representation and quantization

Questions or comments concerning this laboratory should be directedto Prof. Charles A. Bouman, School of Electrical and Computer Engineering, Purdue University, West Lafayette IN 47907;(765) 494-0340; bouman@ecn.purdue.edu

Introduction

This lab presents two important concepts for working with digital signals. The first section discusses how numbers arestored in memory. Numbers may be either in fixed point or floating point format. Integers are often represented withfixed point format. Decimals, and numbers that may take on a very large range of values would use floating point.The second issue of numeric storage is quantization. All analog signals that are processed on the computer must firstbe quantized. We will examine the errors that arise from this operation, and determine how different levels ofquantization affect a signal's quality. We will also look at two types of quantizers. The uniform quantizer is the simpler of the two.The Max quantizer , is optimal in that it minimizes the mean square error between the original and quantizedsignals.

Review of number representations

There are two types of numbers that a computer can represent: integers and decimals. These two numbers arestored quite differently in memory. Integers (e.g. 27, 0, -986) are usually stored in fixed point format, while decimals (e.g. 12.34, -0.98) most often use floating point format.Most integer representations use four bytes of memory; floating point values usually require eight.

There are different conventions for encoding fixed point binary numbers because of the different ways of representing negative numbers.Three types of fixed point formats that accommodate negative integers are sign-magnitude , one's-complement , and two's-complement . In all three of these "signed" formats, the first bitdenotes the sign of the number: 0 for positive, and 1 for negative. For positive numbers, the magnitude simply follows the first bit.Negative numbers are handled differently for each format.

Of course, there is also an unsigned data type which can be usedwhen the numbers are known to be non-negative. This allows a greater range of possible numbers since a bit isn'twasted on the negative sign.

Sign-magnitude representation

Sign-magnitude notation is the simplest way to represent negative numbers. The magnitude ofthe negative number follows the first bit. If an integer was stored as one byte, the rangeof possible numbers would be -127 to 127.

The value +27 would be represented as

0 0 0 1 1 0 1 1 .

The number -27 would represented as

1 0 0 1 1 0 1 1 .

One's-complement

To represent a negative number, the complement of the bits for the positive number with the same magnitude are computed.The positive number 27 in one's-complement form would be written as

0 0 0 1 1 0 1 1 ,

but the value -27 would be represented as

1 1 1 0 0 1 0 0 .

Two's-complement

The problem with each of the above notations is that two different values represent zero. Two's-complement notation is a revision toone's-complement that solves this problem. To form negative numbers, the positive number is subtracted from a certain binarynumber. This number has a one in the most significant bit (MSB), followed by as many zeros as there are bits in therepresentation. If 27 was represented by an eight-bit integer, -27 would be represented as: