Are Measurements Subjective?

The Listing Room, Rene Magritte (1952)

If you frequent some of the hi-fi forums, you may have noticed a dust up over some measurements. No, not the endles Objective versus Subjective debate over the idea of measurements, rather there's some controversy surrounding some measurements generated by LH Labs, makers of the Geek Out DAC, comparing their DAC to a number of competitors. I've received numerous emails relating to these measurements since I reviewed two of the DACs represented in the LH Labs comparisons; the AudioQuest Dragonfly and the Meridian Explorer. "How can something that measures that poorly, sound so good?" being the standard question. "It doesn't." being my standard answer. Let me explain.

Thankfully, both the AudioQuest Dragonfly and Meridian Explorer were also reviewed in Stereophile where John Atkinson performed measurements of both DACs. I think its safe to say that JA's measurements are generally regarded as being industry standard both in the test equipment used and the associated results. The main issue with the measurements provided by LH Labs on their website is they do not jive with Stereophile's. Let's look at just one; Total Harmonic Distortion + Noise.

image credit: LH Labs

LH Labs shows the Dragonfly's THD+N to be > 5% (source). Here's what Stereophile's measurements show for the same THD+N (source):

With the first sample of the DragonFly, a full-scale 24-bit signal actually clipped the bottom halves of the waveform with the computer's volume control set to its maximum, giving a THD+noise level of 3.8%. Backing off the control by one click (–0.17dB) reduced the THD to 2.14%, by a second click (–0.34dB) to 0.627%, and by a third click (–0.51dB) to 0.054%, below which the THD+N percentage plateaued. The second sample didn't clip with a 0dBFS signal at maximum volume, and the THD+N was 0.041% rather than 3.8%. According to Gordon Rankin, the volume control offers 64 steps of less than 1dB to –60dB and then mute (–100dB), but he used only 60 of those steps in the DragonFly, as the top four steps suffered from significant clipping into high impedances. "In retrospect," he wrote of the first sample, "I could have changed the maximum volume down a few more steps and then this would not have been an issue."
So the older Dragonfly, not the newer V 1.2, in its most recent incarnation, 1.0c, had a THD+N of 0.041% according to Stereophile. That is a striking difference from the > 5% shown in the LH Labs graph which is even higher than the original Dragonfly, 1.0, that measured 3.8% according to Stereophile. I've also been told by AudioQuest that the improved version 1.0c was implemented as soon as this problem was discovered and they replaced any model 1.0 with a 1.0c free of charge on request. Why are these THD+N numbers a big deal? According to LH Labs:
Total Harmonic Distortion + Noise represents how well the device will play back "faithfully" to the original music. It generally will give you a good idea about how comfortable and natural you will feel when you listen to your music. We list measurements in both decibels and percentages here. With Dragonfly at maximum output volume, you can see 5% distortion and -25dB THD+N here. This will usually cause the music to sound unnatural.
According to my ears, I'd go with Stereophile's findings (see review). It's worth noting that LH Labs measured their own Geek Out at 0.0027/0.0025 THD+N.

iFI iDSD THD+N. image credit: iFi

I contacted iFi regarding this same issue since their DAC, the iFi iDSD, was also included in the LH Labs post. The LH Labs measurements show a THD+N of 0.152% while iFi claims a THD+N measurement of 0.02%. iFi also took issue with other LH Labs measurements of their iDSD DAC including Signal to Noise Ratio and the FFT Spectrum offering their own results which do not match those on the LH Labs website and in each instance the LH Labs results are worse than those from iFi. Since Stereophile has not performed measurements on this iFi DAC, I offer their measurements as rebuttal.

image credit: LH Labs

iDSD Signal-to Noise Ration (dB - no weighting) image credit: iFi

iDSD Signal-to Noise Ration (dB - A-weighted) image credit: iFi

iFi also took exception with the LH Labs Output Voltage measurements (1.481/1.482Vrms) of their iDSD DAC. From iFi:

The nano iDSD's normal operating voltage output is 1.65v. This is irrefutable.

Yet Competitor X's measured output voltage of 1.48v. The DAC operates directly from the Battery and so when the battery voltage drops near to the full discharge, the output voltage drops in-line with the decline of the battery voltage. So from this, one can work back that the on board battery was running near empty with the red-light warning of battery low on.

The iDSD shuts down at 3.7V and the battery normally is at 4.1V.

Battery drop - 3.7V/4.1V = 90.24%
Signal drop - 1.48V/1.65V = 89.7%

As can be seen the lower, measured signal level by Competitor X is very close to what one can expect based on the Battery Voltage drop from Full to shutdown.

When the battery is so low, its measurements of course are no longer meaningful.

There are other disparities between the LH Labs measurements and the Stereophile measurements. For example, if we look at the LH Labs measurements for the Dragonfly's Maximum Output Voltage, we are shown 2.106 and 2.105Vrms, where the Stereophile measurements came in at 1.84Vrms. Here's what Gordon Rankin, who provided the circuit design for the Dragonfly, had to say about the LH Labs numbers:
We could only attain this level of distortion when we raised the signal to 110%.
AudioQuest also took issue with the LH Labs 5kHz Square Wave measurements and sent along their own results.

image credit: LH Labs

AudioQuest Dragonfly 5kHz Square Wave image credit: Gordon Rankin

I also contacted Gavin Fish of LH Labs and asked him about the apparent discrepancies between their measurements, Stereophile's, AudioQuest's, and iFi's. I received this reply from Larry Ho, LH Labs Lead Engineer:

Few simple facts here...

1. Dragonfly do have clipping in -0dBFS. John Atkinson also mentioned in his review. He needs to dial down a few clicks to get normal result.

IF there is a need to do a special favorable treatment to Dragonfly. I could do that, actually I did it in our first test. The THD+N is still around -70dB or so.

Or if I do that, Meridian and iFi will jump out say: Not Fair? ;-)

2. JTEST is only a simulation of the simplest way jitter by toggle LSB... The JTEST doesn't have the 32 bit version yet, at least to my knowledge so I decide to skip it. Also, there are many professions claim that test is not accurate.

3. As an US manufacturer, I won't have any special favor for an UK vendor. We did Explore, iDSD and Dragonfly at the same time. Meridian Explorer is normal there... Machine won't discriminate.

4. We have the most advanced 1G bandwidth scope on hand. With the distortion I saw on Dragonfly's output wave, we really don't need the big gun scope there.... Again, even if Audio Precision is NOT good enough, and don't need to see Geek out's test result. Why iDSD is performing much better than Dragonfly? Do I need to favor iDSD? :-)

One friend from Google is sending me 3 more good DACs, including Herus, micro streamer and D3.

I'm gonna publish that with the original test results.

Don't worry. Truth is the daughter of the time.

Enjoy!

What does all of this add up to? I'd say we need to fully understand the methods behind the measurements in order to fully understand their outcome. If this is not possible, and I'm sure I don't understand the methods behind all of these measurements, its also not possible to make an informed judgement based on them.

COMMENTS
junker's picture

I'd say Light Harmonic is doing something right...

The early-entrants and competitors have taken notice to their news which was published internally on their company/product forum. I don not believe they published anything outside of their personal forum.

And beside who else published head-to-head DAC results? Certainly nobody in the US audiophile media that is for sure. I'd love to see some disinterested 3rd party publish the same tests. Perhaps someone like Anandtech.

For me I'm glad to see some hard numbers for a change. Let's see more of this!

Type35's picture

John Atkinson from Stereophile = industry gold standard for specs measurements. Says who? Audiostream which is part of the same publishing group!

The vast majority of the press avoids comparative analysis like the plague. 99% of the products are always highly recommended for consideration or competes well with products double/triple the price, etc.

I don't see why LH Labs would lie because as soon as their products are available every competitors will take them apart and will publish their own measurements.

Very recently, the Meridian Explorer launched with a subpar headphone output (with too high an impedance). Meridian being a well known established player, the whole press rated their portable DAC very highly. The only reviewer pointing this issue out was Tyll Hertsens. He is also one of a handful of guys to publish comparative headphone measurements. Coincidence? I think not.

Michael Lavorgna's picture
Here's what I wrote:

"I think its safe to say that JA's measurements are generally regarded as being industry standard both in the test equipment used and the associated results."

I don't see the word "gold" anywhere in there but if you'd care to dispute my statement, have at it.

I think Tyll does a great job too (he also works for the same publishing group ;-).

Larry Ho's picture

HI, Michael

First of all, thank you so much to bring this topic up. I have a lot to say, but I will keep it short here.

I believe both JA's and my tests are both objective. And I never hesitate to express my personal appreciation in public for his continuous effort on putting 'real numbers' out there.

Then why there is 5% (mine) vs 3.8% (his) THD difference from Dragonfly? While I believe both of us are putting exact number there.(I even put direct screen capture from Audio Precision with time stamp as you could see.)

Answer is simple. Different conditions...I'm trying very hard to reflect the real listening in testing condition which maybe will make the number doesn't look so pretty.

Here is my parameters setting, I'm open for any suggestion or discuss from you or JA. ;-)

* Output load is not 100k Ohm or 200K, which is usually not the real headphone load situation. I set 300Ohm in Audio Precision which is more real. By this alone, THD+N will be different.

* I set the bandwidth of measurement to 90K Hz. Not traditional 20K Hz.... Why? Because since everybody is talking about the important of HD music. Why limit the measurement bandwidth to 20k here. It's proven wrong to assume people could hear only below 20KHz.
Again, this single setting will change the SNR and THD results.

* I use 'un-weighted' numbers. A lot of people they use 'A-weighted'... why? Because A-weighted numbers looks better...

And in the same test reports, we tested not only iDSD and Dragonfly, Meridian's Explorer is there too. And their performance is better then iDSD and Dragonfly overall.

So if any one prefer, they could completely ignore my test to my own products. Just comparing the real test results between iDSD, Dragonfly and Explorers will help them a lot. And don't worry, Larry didn't get a penny from these competitors. So I have no discrimination or preference for any of them.... Also, it is fair to say the Dragonfly we bought is directly from market. I didn't know there is or will be an improvement from their V1.2, just like normal buyers.

And our friend in Google will send us quite a few more portable DACs and his numbers, we will use the same condition like above to do the whole test again!

Cheers,

Larry Ho
President/ Light Harmonic and LH Labs

CG's picture

Reading the original measurements over at geek.lhlabs.com, it's mentioned that the noise floor for the Geek Out DAC is -145 dB. I presume that is relative to the 4.2 Vrms output level. If that is the total noise power in a 90 KHz measurement, that is a rather astonishing result. If I did the calculation correctly, that is a noise voltage density lower than the lowest commercially available opamp that I know of, without even taking account of any increased noise caused by associated resistor thermal noise or even the intrinsic noise in the DAC device itself. How does this square with the 116 dB SNR quoted later in the article?

Also, how was the output impedance measured on each DAC sample?

How about jitter performance? There's comments under a number of the graphs suggesting poor jitter performance of the non-Geek DACs. Can you please explain this? What do you see that indicates that the problem is jitter? If you're pointing to tones at these high frequencies, could these not just as easily be "foldback" harmonics resulting from the way the individual DAC chip operates? These behave, and I presume sound, far different than any kind of deterministic jitter or more random phase noise. And, that is but one possibility.

What am I missing?

Beside that, it would seem to me that if anybody has any question about any of these results, from any of the measurers, it could be simply resolved. The manufacturers could again provide samples of each device - or, in fact, they are so inexpensive that they all could be purchased just as a consumer would, on the open market - to someone with the time and the means to measure the samples. Obviously JA is capable of doing that (if he has the time and/or inclination) as could Tyll (again, time and inclination...). I don't think anybody could reasonably assume any favoritism toward any of these DAC vendors - I see ads for each of their products every time I check the various Stereophile associated web pages. But, I suppose that could be possible. That's for the reader to decide. Or, find someone outside the Stereophile family to do the measurements. This is not that friggin hard. The test equipment is very expensive, but it isn't that hard to do the measurements. The test equipment guys have made the job pretty easy, especially for comparisons between pieces where all the test parameters would stay constant.

Honestly, I haven't heard any of these DACs, and have never even seen any of them, except for seeing a Dragonfly (my kid has one on her laptop). For all I know they all are spectacular. Or, they all could suck rocks. Or some combination. I don't know. But, something is fishy here.

One other thing... Just who in the world can use 4+ Vrms output into headphones? Isn't that not so good for the hearing health? It certainly would clip to Smithereens just about every commercially available power amp used in a home, with the possible exception of a couple First Watt products.

Larry Ho's picture

For the jitter performance, you could simply google out a lot of data there. Some articles from JA also mentioned about that wider skirt of central test frequency, if I remember it right.

Tyll will have our Geek Out for the test. I think he should get similar results under the same conditions (bandwidth, load... etc) I don't want to hijack Michael's forum by digging into more details.

There are still a lot nice sounding headphone could leverage the strength of 4V output. A lot of audiophile use Geek Out 1000mW, 4V version, with their big cans, including HD800, 650, LCX... etc. For IEM and other normal headphone, we have Geek Out 450mW there. Do no harm... ;-)

Larry

CG's picture

I appreciate you not wanting to take over Michael's thread.

My only comment is, based on Tyll's measurements, 4 Vrms into HD650 headphones produces a sound pressure level of about 110 dB. That's about the same as the level of a chain saw held at one meter distance. People who listen at that level enough will eventually need such high sound pressure levels in order to hear at all.

jim tavegia's picture

I have never understood the point of competitor bashing. I do remember those who participate in it and their obvious motives for it.

gavn8r's picture

We do have a lot of forums and publications on fire right now, all debating whether we're geniuses or loons. Either way, we win.

wendell's picture

IM new to the hifi seen I stumbled across this sight and read this bashing, publication. I would say this bashing is a turn off, and its not good for the hobby and it can not be good for your business in the long run.
Oh, I had my friend order me a GEEKOUT back in September of 2013 kick starter campaign that I can not get a refund. Six months later I still do not have my GEEKOUT DAC.
I THINK YOU SHOULD FOCUS ON GETTING YOUR PRODUCT OUT THE DOOR INSTEAD OF BASHING YOUR COMPETIORS, THANKS

deckeda's picture

Gavin's been the voice of the company. Appears in the videos and is the face on web ads. Any marketer's job is to sort through and deliver the clear message. Another rule of marketing is to not call into question competitors in such a way that incites (at best) controversy couched in differing standards of truth in the court of public opinion.

Mr. Fish, you were contacted in good faith. Don't push this off on the engineer to handle. It's not his role here. Respond.

Vigna ILaria's picture

Actually, Larry Ho does own the company :)

deckeda's picture

... but owners don't always make the best spokespersons. He hired Fish for what purpose, again?

Larry Ho's picture

No... Technically, company owns me.... ;-)

gavn8r's picture

I've never been one to play by the rules. Not always, anyway.

Larry's engineering skill is masterful. The measurements are what they are. I confidently stand behind them.

labjr's picture

I had a bad dream last night. This huge apple was taking over the bedroom. Suddenly, Larry Ho climbed though the window and saved me.

Vigna ILaria's picture

Larry doesn't climb through my bedroom window in my dreams :)

Larry Ho's picture

To test my theory is correct, I tweak the Audio Precision settings. By change the loading from 300 Ohm to 100K, limit the bandwidth from 90K to 20k... Now I saw the EXACT number like John Atkinson's: 3.8%...

So... both John and I are observing the same situation, that Dragon is clipping in -0dBFS. 5% is the results with my settings, 3.8% is the number with his settings. None of them are wrong. Either of them could prove the other one is wrong...

And the fact is bloody simple: It's Clipping! And both John's first sample and mine has the SAME issue with the SAME number.

Again, I want to limit this discussion to pure technical and regarding testing and results only.

I believe/hope Dragonfly V1.2 should improve it. Because after I hear these feedback about questioning my test results, I google a bit. There do have the news like this...

http://www.audiostream.com/content/audioquest-dragonfly-v12-usb-digital-...

Here is the quote: " the circuitry between the DAC chip and the analog output stage has been refined to create a more direct signal path, leading to even greater transparency and immediacy. Also, the DAC’s power supply has been fortified, which gives the sound more 'grip' and even greater dynamic contrast"

I'm not quite get it about what is more 'grip' of the sound. But I hope/believe they really make the THD and SNR better. Good marketing and communication with media is one thing. The responsibility to the people who bought it, is far beyond.

Cheers,

Larry

deckeda's picture

No one is disputing your numbers Mr. Ho. You misunderstand the context of what's happened. By including the competition's numbers, you've framed the discussion in a way that counterintuitively puts your company in a defensive role, instead of a "hey, we're doing it better" leading role as presumably intended.

It's clearly not an "engineering discussion" you want. You want vindication and validation, which is a very different thing related to ego and pride.

Is there another goal here? To see Mr. Rankin post [another] mea culpa? To see AudioQuest's customers (I'm not one, by the way) come out of the woodwork and say, "Gosh, I made a mistake by purchasing the DragonFly" ?

CG's picture

Well... What we've learned here is two things.

1. If you push any amplifier to clipping, the distortion goes up. Mark me as totally shocked. Now, you could possibly argue that the non-LH products should set their firmware up so that this can not happen. "Problem" solved. But, this also precludes someone from using that extra gain to make up for those recordings that aren't so compressed and never even approach 0 dBfs. (You could also raise the rails, so to speak, but this does come at a cost of needing higher current from the USB 5VDC supply or your battery to run the device. Pick your design approach - engineering is about compromise.)

2. This is all part of LH's marketing plan - see Mr. Fish's comment above. I guess the only bad publicity is no publicity. I'm a dope. I took the bait. Knowing what I now know, I'm leery of of what other tricks might be in store for a purchaser. Fool me once...

Michael Lavorgna's picture
From John Atkinson's measurements as quoted above:

"The second sample didn't clip with a 0dBFS signal at maximum volume, and the THD+N was 0.041% rather than 3.8%"

So you must have measured an original 1.0 version, not the improved 1.0c version. And this improved version was offered to all owners of the original version free of charge (which was also stated above).

Larry Ho's picture

Really nice to know that.

dallasjustice's picture

Larry,

Which version Drafonfly did you measure?

Michael.

labjr's picture

The real question is, did they choose to test the version which they knew had a problem even though they knew the flaw was corrected in subsequent versions? Judging by Gavin's attitude, what do you think?

dallasjustice's picture
labjr's picture

To me, this isn't a legal issue but an ethical one. If they did do knowingly do that, then I have a problem with it because it means they'll do or say anything to make money and profit.

Saying "We bought it from a retailer" is no excuse. Because it means they probably thought about it and did it anyway. I wouldn't do business with a company that acts in such a ruthless manner regardless of how good their product is gonna be which, BTW, we still don't know.

dallasjustice's picture

I think your standard is too lenient. The law doesn't concern itself with motive. For example, in a Murder case, the prosecution doesn't need to prove motive to prevail.

If Larry did measure a non-current model and his current marketing claims about another's goods aren't accurate as to the current model, then Audioquest ought take a closer look at the Lanham Act, IMO. 15 USC section 1125(a)(1)(B).

I agree with a previous poster that we should see more measurements. However, there is a reason why most manufacturers are reticent to post specification about their competitors' gear.

Larry Ho's picture

HI, there...

Thanks for all these information.

Few information: Before we published the test result of Dragonfly. I don't know there is V1.2 exists. Nor did a lot of people, I believe. For example, I also don't know if there is iDSD 1.5 or Explorer 1.05... (enlighten me if there is...)

For business purpose, with or without that testing results, Dragonfly is the only one there still stuck with 96K/24 sampling rate. So really, any people who have some business sense will know there is no need to attack it. Even Dragonfly could reach its THD as their specification, it is still inferior to other competitors (don't need to consider Geek Out there).

So I hope we could go back to the basic facts. I don't need to intentionally attack Dragonfly. People has its own judgement. And we use the same testing conditions for all these DACs, which is not set specifically for any one. And the Dragonfly we have on hand has the clipping issue. And we also publish the FFT of Dragon when down to -10dBFS there. It is better.

Magnum Innominnandum's picture

Dear Mr Ho,

Let us just presume that this whole affair is a honest mistake (just as all the delays in delivering your products to pre-aid customers are honest delays caused by QC problems).

The kind one may easily make when handling a complex test system such as an AP2. We all are prone to such. One can easily rectify such by simply coming clean and admitting to what actually goes on.

Why not go ahead and turn down the the volume on the Dragonfly (after all, it is an analogue volume control) and confirm the rest of the Measurements per JA. When you have learned how to correctly test the Dragonfly to replicate a third party's result, you can correct your results and post an update that shows the true performance (and maybe a few words of contrition).

Maybe you should re-measure all DAC's according to established industry standard, your product should gain as much as the competition. You can of course still show the results of different settings. But by first showing measurements that are in agreement with those of others before showing any made with different methods you demonstrate there is no chance your measurements are in error or indeed skewed to favour your own product, as some might be tempted to allege.

You see, it is all about openness and honesty and responsibility to your customers.

Magnum innominandum, signa stellarum nigrarum

junker's picture

Can you read or did not notice that their measurements confirmed that found by the 3rd part test measurements from JA @ Steorophile?

In addition, testing at 100k is a more realistic load for headphones vs. 100k or 200k ohms.

FACTS
1) JA's Stereophile testing also revealed clipping of this product and 3.8% THD at full-output into 100k ohms
2) When tested at "real world" loads LH test results were 5% THD
3) When LH retested at easier loads used by Stereophile they reported the EXACT same THD measurement of 3.8% at full-output
3) The designer (Gordon Rankin) of the DragonFly admitted to a design error on ComputerAudiophile that resulted in the clipping issue. He admitted the design issue. How would this have been released if it were tested under typical headphone loads??
4) Audioquest subsequently released an updated version to fix this design flaw, as per Steve Silberman
4) Was the volume issue and/or workaround published with the product? If not than it was tested under full-output conditions as ever other product in this test. Was this issue ever disclosed to customers? Was a recall ever initiated?
5) According to LH the DragonFly test unit was purchased recently from an authorized reseller and I have to assume they have a sales receipt to supports this.
6) LH ordered the version 1.2 model from Amazon.com and will update their tests with the results from the newest updated version shortly.
7) Both ComputerAudiophile and AudioStream - who appear to be publishing pre-canned talking points and collateral - both media outlets are paid directly by AudioQuest for advertising
9) Miska on ComputerAudiophile just published a second set of magazine test results which show that the DragonFly 1.2 measured more poorly than the built-in audio output on an Apple Mac Pro:

http://www.computeraudiophile.com/f6-dac-digital-analog-conversion/someo...

I actually like having some actual test measurements under more realistic loads for a change and look forward to DragonFly v1.2 Audio Precision test results, and to see how they dovetail with the results referenced by Miska.

junker's picture

Correction: It was tested at a lower load than 100k (~300ohms I believe).

Common headphones like the Sennheiser Momentum headphones only have a load of 18 ohms - even more difficult than that tested by LH.

http://en-us.sennheiser.com/over-ear-headphone-momentum-stereo

Michael Lavorgna's picture
7) Both ComputerAudiophile and AudioStream - who appear to be publishing pre-canned talking points and collateral - both media outlets are paid directly by AudioQuest for advertising
Look to the right side of this screen then explain to me how the fact that Light Harmonic also advertises on AudioStream doesn't make your point appear to be completely ridiculous. Let me help - you can't because your point is completely ridiculous.
Magnum Innominnandum's picture

Junker,

Yes, I can read...

I read that Mr. Ho replicated ONE measurement to match JA's at Stereophile. Yet he failed to match Stereophile's THD & N at 0.041%, 100 times lower than the measurement he was able to replicate. How, this is another story.

FACTS

1) JA's Stereophile testing also revealed that if either the new version of the Dragonfly was used or the original version had it's analogue volume control adjusted to avoid clipping this product produced 0.041% at 0dBFS into 100k ohms

2) When tested by LH Labs the Dragonfly nearly 2.1V output. According to Stereophile it produces around 1.85V. According to Mr. Rankin, the designer it is impossible to get this output level. According to Mr. Rankin the dragonfly an only produce the levels recorded by LH Labs if it was a send a signal that had been boosted digitally and hence clipped BEFORE it was send to the Dragonfly.

4) [next time please keep count correctly]

The designer (Gordon Rankin) of the DragonFly admitted that it was possible to turn up the analog volume control on the Dragonfly so far that Dragonfly output stage would clip.

This is a common trait of many analog amplifiers, they have some gain reserve and the volume control must be set correctly to avoid clipping.

5) Audioquest subsequently released an updated version. This simply prevented anyone incapable of using a volume control correctly from doing so.

6) The Stereophile Article clearly discusses this issue and mentions this issue and it mentions that setting the volume control around 0.5dB below maxium resolved the issue.

Further, Audioquest released a fix before John Atkinson's measurements went to publication (you would know all that had you read the original article).

As LH Labs first instalment of "Geek vs. Others" published August 16 2013 explicitly references the Stereophile article with full URL and includes the 0.041% THD figure from this article.

So LH-Labs where quite aware of the whole situation and all details.

7) If LH Labs made this purchase they should be able to provide the serial number and as you mention the receipt. So why not post them?

10) I responded to Miska on some of the incongruities in the numbers he posted.

Let us be clear, what is at debate here is not if the Draognfly is better than a MacBook Audio, or the Geek Out or anything under the sun.

What is at debate is if the measurements published by LH-Labs where done fairly and represent truthful advertising.

As I took up law after realising that Engineering was not a career to become well off, I would suggest that under existing laws in the USA AudioQuest likely has a good case (as others have pointed out).

I am sure any number "no win - no fee" Lawyers in the US would love to take such a case to court, especially if someone showed the the Kickstarter and Indiegogo campaigns...

This much publicity would make it trivial for them to prove the lot and to show both damage to AQ and monetary gain to LH-Labs and nothing motivates lawyers more than money. I'd offer my service but I suspect Montreal will not be accepted as Venue.

I sure wish I was that US Lawyer who gets to try this one.

labjr's picture

Sorry but this isn't about what Stereophile does with Audioquest or why they released it with a flaw. Or semantics about technical terms and test results or how they test it. You think The Geekout DAC is going to be without issues? Really!

They knew about the flaw and used it to exploit them. They also knew Audioquest updated the unit since John Atkinson received a second sample which tested without the problem, in time for the September 2012 review.

Gavin's attitude shows he doesn't give a shit about anything but the bottom line and will do anything to try to stomp the competition if he thinks it will bring more business his way. If they want to prove their products are better, how about delivering them to buyers?

I don't like the way the system works any more than the next guy but the magazines are for entertainment. Something has to pay for the cost of printing, salaries etc. The $1 per month subscription fee certainly doesn't. They aren't gonna have a head to head competition because everyone would be crying about unfair treatment, nobody would advertise and the publication couldn't exist. Readers can use their imaginations to draw their own conclusions. That's the way it is. If they object so much they can always go somewhere else. So far, I don't see Light Harmonic pulling their ads from this site.

Michael Lavorgna's picture
...iDSD including measurements:

Audio-Creative iFi nano iDSD Review (translated version)

"The distortion is low, just excessively low. At -0.5 dBFS, the THD + N (total distortion plus noise and hum) 0.066% and 0.024% linear weighted measured. Is at -14 dBFS output latter 0.0096%."
Magnum Innominnandum's picture

Larry Ho, LH Labs Lead Engineer wrote in response to this collective dropping the mits:

"Don't worry. Truth is the daughter of the time."

So she is.

The limeys have just published a review of the stone of contention and, marvel of all marvell's - it includes the first independent measurements of it...

http://www.anthemavs.co.uk/wp-content/uploads/HFN_Budget-Esoterica_Geek-...

Now one hears Paul Miller is a real Measure-Nazi (yes - that is a word, like Soup-Nazi) who gives the Gostapo a bad name.

So what did he find?

"Rated at 1000mW, our Geek Out EM achieved 605mW/25ohm at 0.027% THD"

"the A-wtd S/N ratio, which would necessarily enjoy a boost relative to the 2V competition, is rather weaker
at 94.3dB"

Do I smell something burning?

Pants perchance?

Oh, that would be the beans roasting downstairs at the dep, rather nice smell, time to head there for a freshly roasted triple-triple.

Magnum Innominnandum's picture

Larry Ho, LH Labs Lead Engineer wrote in response to this collective dropping the mits:

"Don't worry. Truth is the daughter of the time."

He also wrote:

"Good marketing and communication with media is one thing. The responsibility to the people who bought it, is far beyond."

Very wise words by Mr. Ho, indeed.

Meanwhile the good book in Job 15:6 states:

"Thine own mouth condemneth thee, and not I: yea, thine own lips testify against thee."

Why so?

Looking at the specifications posted for the LH-Labs Geek out (the actual shipping model) on LH-Labs website we find something surprising:

http://marketplace.lhlabs.com/products/geek-out-1000-usb-dac-and-headpho...

Signal to Noise Ratio (SNR): >101 dB unweighted

Now if we compare that to the claims made in the comparison test that started this whole dropping of mits:

http://lhlabs.com/force/geekout/898-geek-out-vs-the-others-3rd-installme...

"Geek outs SNR is an impressive 116dB"

Woah Nelly!

Somehow, between the production model Geek Out shipped to the many backers of LH Labs and still on sale and the prototype tested against the competiotion, the Geek Out LOST 15dB of performance. And for DAC's SNR is a key performance metric, not unlike horsepower for cars. Of course, those who read this quirky British publication called HiFi-News already knew that there was something amiss.

It is also worth checking other performance metrics how much production units are worse than the original "measurements" LH-Labs presented. For example THD & N (another key specification for DAC's) is also almost five times worse.

If we put this into the correct metrics, it means that the production Geek Out is 5.6 times noisier than originally promised to the backers. In fact, the Geek Out delivered to customers has a worse SNR than the much maligned Audioquest Dragonfly

In a more real context, put into a context like cars, we might say that the Backers of LH-Labs were promised a plain looking car with high performance 150 Horsepower Engine, but the delivered actual product barely manages 27 Horse Power!

I think that Mr. Ho should explain precisely what happened here, because: "The responsibility to the people who bought it, is far beyond." and it looks like this responsibility to LH Labs customers has been failed grossly and the trust of said customers has been abused.

wellington12's picture

Sorry to use the wrong comment section, but I am
confused as to how to ask a new question to ask
audio stream. Please explain how, thanks Jim

Michael Lavorgna's picture
mlavorgna@enthusiastnetwork.com