ADC Resolution vs Accuracy: Why 16 Bits Isn't 16 Useful Bits

The board uses a 16-bit ADC, so the spec sheet for the product says “16-bit resolution”. A customer reads that as “accurate to 1 part in 65,536” and files a complaint when two units disagree by 2 mV on a 5 V range. Both units are working correctly. The difference between ADC resolution vs accuracy explains it: resolution is the size of one code step, accuracy is how close the code is to the true input, and the second is limited by offset, gain error, nonlinearity and noise, none of which the bit count describes. This article separates the specs, works through one example, and shows which errors calibration removes and which it never will.

Key Takeaways

• Resolution, precision and accuracy are three different things. The international vocabulary of metrology (VIM) is explicit that accuracy isn’t even a number; you state errors and uncertainties instead.
• For an uncalibrated converter, offset and gain error usually dominate total error. Nonlinearity matters after you calibrate.
• Combine independent static errors as a root-sum-square to get total unadjusted error (TUE). Adding worst cases overstates it.
• Noise sets two different “resolutions”: effective resolution from RMS noise and noise-free resolution from peak-to-peak noise, about 2.7 bits apart.
• Two-point calibration removes offset and gain error. It does not remove integral nonlinearity, noise or drift with temperature.

The question behind “why is my 16-bit ADC off by 2 mV?”

On engineering forums this comes up in several forms: “my ADC reads 30 counts high at full scale”, “two boards give different readings for the same voltage”, “what’s the point of 24 bits if the last 6 are noise?” Each one mixes up properties that datasheets list separately.

Part of the confusion is vocabulary. In everyday speech, resolution, precision and accuracy are used interchangeably. In measurement they aren’t. The JCGM’s International Vocabulary of Metrology (VIM, clause 2.13) defines measurement accuracy as the closeness of agreement between a measured value and the true value, and notes that accuracy is not a quantity and has no numerical value. Precision (clause 2.15) is the closeness of agreement between repeated measurements. Resolution (clause 4.14) is the smallest change in a quantity that causes a perceptible change in the indication.

For an ADC, that maps cleanly. Resolution is the step size, one least significant bit (LSB). Precision is how much repeated readings of the same input scatter, which is noise. Accuracy is how far the average reading sits from the true input, which is set by static errors. A converter can be precise and wrong at the same time: very low noise, consistently 30 counts high.

How ADC errors actually add up

The static errors: offset, gain and nonlinearity

Analog Devices’ tutorial MT-010 (Kester, 2008) defines the static specifications with a simple transfer equation, D = K + GA, where A is the analog input and D the digital output. Offset error is the deviation of K from its ideal value. Gain error is the deviation of G, usually given as a percentage of full scale. Differential nonlinearity (DNL) is how far any single code step departs from 1 LSB. Integral nonlinearity (INL) is the maximum deviation of the whole transfer curve from a straight line. MT-010 notes that the end-point method (a line through zero and full scale) gives an INL figure roughly twice the best-straight-line method, so check which one a datasheet uses before comparing parts.

MT-010 also says which errors you can trim: offset and gain usually can be adjusted by the user. INL is a property of the converter’s curve and isn’t removed by a gain-and-offset correction.

Combining them: total unadjusted error

Texas Instruments’ Precision Hub article on total unadjusted error (Agarwal, 2014, updated 2020) defines TUE as the maximum deviation between the actual and ideal transfer functions with no system calibration, combining offset, gain error and INL. Its recommendation is to combine the maximum values as a root-sum-square in the same unit (usually LSB) rather than adding them, because the worst-case offset, gain and linearity errors don’t normally occur at the same input voltage. The same article notes that in uncalibrated systems, offset and gain errors dominate TUE.

Noise: effective and noise-free resolution

Noise sets the precision limit. Analog Devices’ MT-004 (Kester, 2008) measures input-referred noise with a grounded-input histogram, whose standard deviation is the RMS noise in LSB. From it come two figures:

• Effective resolution = log₂(2ᴺ / RMS noise in LSB).
• Noise-free code resolution = log₂(2ᴺ / peak-to-peak noise in LSB), with peak-to-peak taken as about 6.6 × RMS.

The two differ by log₂(6.6), about 2.7 bits. Effective resolution tells you the statistical resolution; noise-free resolution tells you how many bits stay stable without flicker on a display.

Dynamic performance: ENOB

For AC signals, Analog Devices’ MT-003 (Kester, 2008) defines effective number of bits from the measured signal-to-noise-and-distortion ratio: ENOB = (SINAD − 1.76) / 6.02. ENOB includes harmonic distortion, so it usually falls as input frequency rises. IEEE Std 1241, the IEEE standard for terminology and test methods for ADCs (current revision 1241-2023), is the reference for how these quantities are measured.

A worked example (illustrative values)

Take a 16-bit converter on a 0–5 V range, so 1 LSB = 5 V / 65,536 ≈ 76.3 µV. Suppose the datasheet gives offset error ±4 LSB, gain error ±0.05% of full scale and INL ±2 LSB, and a grounded-input test shows 1.5 LSB RMS noise. These are round illustrative numbers, not a specific part.

• Gain error: 0.05% × 65,536 ≈ 32.8 LSB.
• TUE (RSS): √(4² + 32.8² + 2²) ≈ 33.1 LSB ≈ 2.5 mV, about 0.05% of full scale. Expressed as bits: log₂(65,536 / 33.1) ≈ 11.0 bits of accuracy before calibration.
• Effective resolution: log₂(65,536 / 1.5) ≈ 15.4 bits.
• Noise-free resolution: 15.4 − 2.7 ≈ 12.7 bits.

So the converter resolves about 15 bits, displays about 12.7 stable bits and, uncalibrated, is accurate to about 11. Each statement is true. The customer complaint in the opening is the difference between the first and the last.

Which ADC errors calibration can remove

The table summarizes the specs above and what happens to each after a standard two-point (zero and full-scale) calibration.

Specification	What it describes	Unit on datasheets	Removed by two-point calibration?	Source
Resolution (N bits)	Step size, 1 LSB = range / 2ᴺ	bits	Not an error	VIM 4.14
Offset error	Shift of the whole transfer curve	LSB or mV	Yes, at the calibration temperature	MT-010
Gain error	Slope error of the transfer curve	% FS or LSB	Yes, at the calibration temperature	MT-010
INL	Bow of the curve from a straight line	LSB or % FS	No; needs multi-point correction	MT-010
DNL	Unevenness of individual code steps	LSB	No	MT-010
RMS noise	Scatter of repeated readings	LSB rms or µV rms	No; averaging reduces it	MT-004
ENOB	Noise plus distortion at a given input frequency	bits	No	MT-003
TUE	Combined offset, gain and INL with no calibration	LSB or % FS	Mostly; INL remains	TI Precision Hub

The reading of this table that matters most: the uncalibrated accuracy figure is dominated by the two errors that are cheapest to remove. Once they’re gone, INL and noise become the limit, and those need either a better part or a different technique, such as multi-point correction or averaging.

What to actually do

Write the error budget in LSB before you choose a part

Convert every datasheet error to LSB at your range and combine offset, gain and INL as a root-sum-square. If the total is acceptable without calibration, you’ve saved a production step. If not, you know calibration is mandatory, and you can compare parts on INL and noise instead of headline bits.

Include the reference in the gain error

Gain error in the datasheet assumes an ideal reference. Any error in your reference voltage is a gain error for the whole system, and its temperature coefficient becomes a gain drift. Add the reference’s initial tolerance and drift over your temperature range to the budget as separate lines.

Calibrate offset and gain at the right points

A two-point calibration near zero and near full scale removes offset and gain at the calibration temperature. If your signal never reaches full scale, calibrate across the range you use; MT-010’s transfer equation is a straight line, and the line that fits your working range best is the one that matters.

Measure noise with the input shorted

Ground the input at the ADC pins (or apply a clean DC source), capture a few thousand samples and compute the standard deviation. That gives RMS noise in LSB, and from it effective and noise-free resolution. If the result is much worse than the datasheet, look at the reference decoupling, the layout and the input driver before blaming the converter.

Quote accuracy and resolution separately on product material

When a product spec says “16-bit”, say what that means and add the accuracy figure with its conditions: “16-bit resolution; total error ±0.05% of full scale after calibration at 25 °C”. It answers the engineer’s real question and prevents the complaint in the opening.

What not to do

Don’t add worst-case offset, gain and INL linearly and call it accuracy; the TI article explains why that overstates the error. And don’t assume averaging fixes accuracy. Averaging reduces noise, which improves precision, but a converter that reads 30 counts high reads 30 counts high on average too.

How to put it to work

If you have an hour

Take the datasheet of the ADC you use, convert offset, gain error and INL to LSB at your range, combine them as a root-sum-square and write the answer in bits. Compare it with the resolution printed on your product spec.

If you have a sprint

Build a small calibration fixture with two stable reference voltages, measure five boards before and after two-point calibration and record the residual error at five points across the range. The residual curve is your INL in practice.

If you want it to last the product’s lifetime

Store the error budget, the calibration procedure and the per-unit calibration constants with the production records. Re-check a sample of units across temperature once a year or when the reference or ADC part changes.

The question to ask yourself: “If a customer asked how far a reading can be from the true value, could I answer with a number, its conditions, and where it came from?”

Frequently asked questions

What is the difference between ADC resolution and accuracy?

Resolution is the smallest step the converter can represent, one LSB, set by the number of bits and the input range. Accuracy describes how close the reading is to the true input and is limited by offset, gain error, nonlinearity, reference error and noise. A 16-bit ADC has 16-bit resolution but may be accurate to only about 11 bits before calibration.

What is ENOB in an ADC?

ENOB, the effective number of bits, is the resolution an ideal ADC would need to match the real converter’s measured signal-to-noise-and-distortion ratio: ENOB = (SINAD − 1.76) / 6.02. It includes noise and harmonic distortion, depends on input frequency and amplitude, and is usually lower than the converter’s nominal bit count.

What is total unadjusted error (TUE)?

Total unadjusted error is the maximum deviation of an ADC’s actual transfer function from the ideal one with no calibration, combining offset error, gain error and integral nonlinearity. It’s usually calculated as the root-sum-square of the individual maximum errors in LSB, because the worst cases don’t normally occur at the same input voltage.

Can calibration fix INL?

Not with a standard two-point calibration, which corrects only offset and gain, the shift and slope of the transfer curve. INL is the curve’s bow, so correcting it needs a multi-point calibration or a lookup table, which only helps if the nonlinearity is stable with temperature and time.

What is noise-free resolution?

Noise-free code resolution is the number of bits that stay stable without flickering, calculated from peak-to-peak noise: log₂(2ᴺ / peak-to-peak noise in LSB). Peak-to-peak noise is taken as about 6.6 times RMS noise, so noise-free resolution is about 2.7 bits lower than effective resolution.

Related reading: the full sensor signal chain, sensor calibration for offset, gain and linearity and where sensor noise comes from.

Sources

1. Joint Committee for Guides in Metrology (2012). International Vocabulary of Metrology: Basic and General Concepts and Associated Terms (VIM, JCGM 200:2012), clauses 2.13, 2.15 and 4.14. BIPM. https://jcgm.bipm.org/vim/en/2.13.html
2. Kester, W. (2008). “The Importance of Data Converter Static Specifications: Don’t Lose Sight of the Basics!” Analog Devices Tutorial MT-010, Rev. A. https://www.analog.com/media/en/training-seminars/tutorials/MT-010.pdf
3. Kester, W. (2008). “The Good, the Bad, and the Ugly Aspects of ADC Input Noise: Is No Noise Good Noise?” Analog Devices Tutorial MT-004, Rev. A. https://www.analog.com/media/en/training-seminars/tutorials/MT-004.pdf
4. Kester, W. (2008). “Understand SINAD, ENOB, SNR, THD, THD + N, and SFDR so You Don’t Get Lost in the Noise Floor.” Analog Devices Tutorial MT-003, Rev. A. https://www.analog.com/media/en/training-seminars/tutorials/MT-003.pdf
5. Agarwal, V. (2014, updated 2020). “ADC Accuracy Part 2: Total Unadjusted Error Explained.” Texas Instruments Precision Hub. https://e2e.ti.com/blogs_/archives/b/precisionhub/posts/adc-accuracy-part-2-total-unadjusted-error-explained
6. IEEE (2023). IEEE Standard for Terminology and Test Methods for Analog-to-Digital Converters (IEEE Std 1241-2023). https://standards.ieee.org/ieee/1241/6797