Free shipping to the United States when you order $20+

Accuracy/Resolution
Login to post a new topic and to write a reply.
Thread Tools


Search within thread:

Rate This Thread
Login to rate this thread.
#1 (permalink)      2/27/2014 7:44:52 AM US Central   quote/reply + tips
FlavorFreak
Verified
  • JoinedNov 2013
  • Posts 2
  • Reviews 0
  • Kudos0
Accuracy/Resolution
Can anyone explain to me, what these bold marked digits stand for? Is it an accuracy or resolution value? Thank you!

Examples:
750V ±(1.2%+5)
400ohm ±(1.2%+2)
4A/10A ±(2.0%+5)
#2 (permalink)      2/27/2014 8:53:42 AM US Central   quote/reply + tips
Hrawk
Verified
  • JoinedJan 2014
  • Posts 1026
  • Reviews 156
  • Kudos1257
Say you are measuring around 500 volt. Precision is ±(1.2%+5).

This means you could be out by ± 11v ((500 x 1.2%)+5), giving you a reading between 489 and 511 volts.
#3 (permalink)      2/28/2014 8:47:59 AM US Central   quote/reply + tips
FlavorFreak
Verified
  • JoinedNov 2013
  • Posts 2
  • Reviews 0
  • Kudos0
Thank you for your answer. Sounds good to me, until I imagine a low range measuring scenario:

e.g I want to measure a resistance of around 1,5 ohms. Regarding the spec "400ohm ±(1.2%+2)", it would give me a reading between -0.518 and 3.518 ohms. This measurement would be useless. So is that number "2" really 2 "full" digit values which the result may differ?
#4 (permalink)      2/28/2014 9:12:21 AM US Central   quote/reply + tips
Hrawk
Verified
  • JoinedJan 2014
  • Posts 1026
  • Reviews 156
  • Kudos1257
Pretty much.

The +2 refers to '2 counts' or 2 units in your current range.

2 ohms when in the 400 ohm range or 2k ohm when in the 40k range.

It sux that even Fluke (Quality equipment) do this now as it makes those percentages smaller which traditionally, was all we looked at.

Fucking marketing.
#5 (permalink)      12/5/2014 7:40:48 AM US Central   quote/reply + tips
mbarbos
Verified
  • JoinedDec 2014
  • Posts 1
  • Reviews 0
  • Kudos0
Nope, it is not marketing, it's because of the A/D conversion.
Usually +-1 is when you don't have enough digits to display the value (5.6789 would be displayed as 5.678 on a 4 digit display), +-other value means that the A/D converter has less resolution than what you can display.