DoAssignment.ca

DoAssignment study guide

A1.4 · Explain finite precision and representation limits

Learn to explain finite precision and representation limits through clear examples and targeted practice.

Ontario Grade 12 Computer Science

Programming Concepts and Skills

How a computer’s finite number of bits limits the values it can store

Computers store information using patterns of bits, each of which has one of two values: 0 or 1. Because a computer value uses a limited number of bits, it cannot represent every possible number. Some representations have a maximum and minimum value; others also cannot store certain values exactly. These limits matter when interpreting a calculation: the displayed result may be rounded, or a value may be outside the range the representation can hold. This lesson focuses on those limits and how to reason about them using simple examples.

What you will learn

  • Explain why a finite number of bits can represent only a limited set of values.
  • Determine the range of an unsigned whole-number representation with a stated number of bits.
  • Explain why some decimal fractions are stored approximately.
  • Describe how limited range and precision can affect calculations and their results.

1. From bit patterns to a limited set of values

A bit is a binary digit: 0 or 1. A fixed group of bits has only a fixed number of possible patterns. With one bit there are two patterns; with two bits there are four. In general, nn bits can form 2n2^n different patterns. A number representation assigns meanings to those patterns.
For an unsigned whole-number representation, every pattern stands for a non-negative integer. The smallest value is zero. If there are nn bits, the largest value is one less than the total number of patterns. This gives a finite range. When a calculation needs a value beyond that range, that representation cannot store it as an ordinary value in its range.
Other representations assign some patterns to negative numbers or use part of the available information in a different way. Therefore, knowing only the number of bits is not enough to identify the exact range: you also need to know how the patterns are interpreted. The examples here state the representation being used.
2n2^n
  • A fixed number of bits has a fixed number of patterns.
  • A representation rule determines what each pattern means.
  • The bit count and representation rule together determine the available values.

2. Range is not the same as precision

Range describes how small or large a representable value can be. Precision describes how finely values can be distinguished or how closely a stored value can match the intended value. These are related but different limits.
For example, a whole-number representation can store each integer in its range exactly, but it cannot store fractions as whole numbers. A representation for numbers with fractions may cover very small and very large values, yet still store only selected values exactly. A value between two representable values must be approximated if the representation cannot encode it.
In many computer representations for numbers with fractions, the available bits are used to describe a scale and a limited number of significant binary digits. This can allow a wide range, but it does not make every decimal fraction exact. Rounding chooses a nearby representable value. The stored value and the intended value can therefore differ slightly.
  • Range concerns the extent of values that can be stored.
  • Precision concerns the closeness or detail with which values can be represented.
  • A number with a fractional part may be stored approximately even when it is within the representation’s range.

3. Why representation limits matter in calculations

A calculation uses the values available in the chosen representation. If an input has already been rounded when stored, later calculations use that stored approximation rather than the exact intended value. Further results may also need rounding to a representable value.
A result can also exceed a representation’s range. For a bounded whole-number representation, a mathematical answer larger than its maximum is not an ordinary value that the representation can hold. The specific behaviour depends on the representation and system; do not assume that every language handles this situation in the same way.
When reviewing a result, ask three questions: What representation is being used? Is the intended value within its range? Can that representation store the value exactly, or must it approximate it? These questions help distinguish an arithmetic mistake from a limitation of the representation.
  • Calculations operate on stored values, which may already be approximations.
  • A result outside the range cannot be represented as an ordinary value in that range.
  • Always identify the representation before making a claim about a value or calculation.

4. A practical way to explain a result

Start by stating the representation, including its bit count when known. Next, describe its available range or the fact that some values are approximations. Then trace the intended operation using the stored values and explain whether the result remains representable. Keep the claim specific: an example can demonstrate one representation and one calculation, but it does not establish how every computer or programming language behaves.
This reasoning is useful when a displayed answer has unexpected final digits, when a number is too large for a stated range, or when a value with a fractional part cannot be represented exactly. It also helps you communicate what you know and what depends on details that were not supplied.
  • Name the representation and the relevant limit.
  • Separate the intended mathematical value from the value the representation can store.
  • Avoid generalizing beyond the stated representation.

Worked example

Counting the values in a small unsigned representation

An unsigned whole-number representation uses 3 bits. How many patterns are available, and what is its range?
  1. Count the patterns
    Each of the three bit positions has two possible values. The number of patterns is therefore two multiplied by itself three times.
    23=82^3 = 8
  2. Find the range
    Because the representation is unsigned, its patterns represent non-negative whole numbers. The range begins at zero and ends one less than the number of available patterns. 0 to 7
Answer: There are 8 patterns, representing the whole numbers from 0 through 7.
Check: The conclusion depends on the stated unsigned interpretation. A different rule for assigning meanings to the patterns could give a different range.

Worked example

Testing whether a whole-number value fits

A counter uses the same 3-bit unsigned representation. Can it store the value 9 as an ordinary value in its range?
  1. Recall the stated range
    The 3-bit unsigned representation has values from zero through seven.
    0≤x≤70\leq x\leq 7
  2. Compare the proposed value
    The proposed value is greater than the maximum. It is not in the representation’s stated range, so it cannot be stored as an ordinary value within that range. The question does not specify what a particular system does when this limit is exceeded.
    9>79>7
Answer: No. The value 9 is outside the range 0 through 7.
Check: This answer identifies whether the value fits; it does not assume a particular system response to an out-of-range value.

Worked example

Understanding an approximate decimal fraction

A number representation cannot store the decimal fraction 0.1 exactly. Explain what it means to store a nearby value and why a later displayed calculation may differ slightly from the exact decimal calculation.
  1. Separate intended and stored values
    The intended value is the exact decimal fraction. If the representation has no pattern for that exact value, it stores a representable value close to it instead.
    stored value≈0.1\text{stored value}\approx 0.1
  2. Follow the calculation
    A later operation uses the stored approximation. Its result may therefore differ slightly from a calculation using the exact intended decimal value. The size and direction of the difference depend on the representation and the operation.
    result uses stored value\text{result uses stored value}
Answer: The stored value is a nearby approximation, not necessarily exact 0.1. A later calculation uses that approximation, so its result may have a small difference from the exact decimal result.
Check: No particular numerical error is claimed here because the representation details and operation are not specified.

Common mistakes and how to avoid them

Assuming a fixed number of bits can represent every number.
Correction: A fixed number of bits has only a finite number of patterns. The representation assigns meanings to those patterns, so only a limited set of values is available.
Treating range and precision as interchangeable.
Correction: Range describes the extent of values; precision describes how closely or finely values can be represented.
Assuming every decimal fraction is stored exactly.
Correction: A fractional value may not match any available pattern, so the representation may store a nearby value.
Claiming that every system responds to an out-of-range value in the same way.
Correction: State that the value is outside the specified range. The response depends on details that must be given separately.

Lesson summary

  • A fixed number of bits creates a finite set of patterns.
  • The representation rule determines which values those patterns mean.
  • Range and precision describe different limits.
  • Some fractional values must be approximated, and later calculations use the stored approximation.
  • Check the representation and its limits before interpreting a result.

Check your understanding

Question 1

How many different patterns can be made with 4 bits?
  1. 4
  2. 8
  3. 16
  4. 32
Show answer and explanation
16
Four bit positions each have two possible values, so the count is 24=162^4=16.

Question 2

A 3-bit unsigned whole-number representation is used. Which statement is correct?
  1. It represents values from 0 through 8.
  2. It represents values from 0 through 7.
  3. It represents only negative values.
  4. It can represent every whole number.
Show answer and explanation
It represents values from 0 through 7.
There are eight patterns, assigned to the non-negative values from zero to seven.

Question 3

What does it mean if a fractional value is stored approximately?
  1. The representation stores a nearby value because the exact value has no available pattern.
  2. The computer has changed the representation rule to include every decimal.
  3. The value is necessarily outside the largest representable range.
  4. The stored value must be exactly equal to the intended value.
Show answer and explanation
The representation stores a nearby value because the exact value has no available pattern.
Approximation means that the stored value is close to, but not necessarily equal to, the intended value.

Key terms

Bit
A binary digit with a value of 0 or 1.
Representation
A rule for assigning meanings to available bit patterns.
Range
The smallest and largest values a representation can store.
Precision
How finely values can be distinguished or how closely a stored value can match an intended value.
Approximation
A nearby representable value used when the exact intended value cannot be stored.

Continue through ICS4U

View the complete ICS4U Ontario Grade 12 Computer Science curriculum and lessons

About this lesson and its review

Published by DoAssignment. This reviewed lesson follows Ontario Grade 12 Computer Science (ICS4U), expectation A1.4. It is a study resource, not an official curriculum publication.

Before publication, content is checked for structure, mathematical or chemical notation, calculations, course boundaries, and readability. Errors can still occur, so corrections are welcomed.

Official curriculum reference

Report a correction or ask a question