Making Excuses for Consumer Reports

January 9th, 2013

So a big fuss is being raised in the media over the fact that the iPhone 5 is not listed as the number one smartphone in the latest rankings from Consumer Reports. Based on product ratings of smartphones sold by the major U.S. carriers, the iPhone 5  is slipping.

Some members of the media continue to make excuses for the curious reviews from CR, claiming that the competition keeps getting better, while, I suppose, Apple stands still. So even though the iPhone 5 was once at the top of the heap, some mysterious changes on the part of HTC, LG, Motorola and Samsung have pushed Apple’s gadget from the top and supplanted it.

So is that true? Since the introduction of the iPhone 5 in September, have the ensuing four months brought worthy improvements in the competition? How so? Do they offer better screens? Not that I can see? What about speedier performance? Well, here the iPhone 5 and its A6 processor rate with the best of the competition. And there hasn’t been a major upgrade to Google’s Android software that would somehow result in a sudden improvement of the OS’s interface or performance.

So what’s really going on? Has Apple suddenly been downgraded by CR?

Well, ratings that are no more than a few points apart are regarded by CR as not significant. It seems that the Android phones, for some reason, rate better than the iPhone at “Messaging” and “Phoning.” You’d think the features are fairly obvious by the description, but it appears that CR blurs email and texting, and doesn’t give Apple any brownie points for iMessages. When it comes to Phoning, Android handsets are somehow regarded as superior in handling calls. But they both rate the same in that ephemeral “Ease of use” factor, which seems doubly confusing, since the iOS tends to fare better on that criteria according to most independent reviewers. Overall, they are all quite close, and CR would have you believe that you can buy any of them and be assured of a great user experience. They are, after all, all recommended.

But the truth is that these ratings have been out for weeks. The only change is the appearance of “The Hot List” in the February 2013 issue, which summarizes the best products in several categories that include smartphones, TVs, E-book readers and home appliances. To CR, Apple has always been at or near the top, with the highest-rated products receiving ratings that are extremely close.

You can certainly agree or disagree with CR’s rating methodology, or the results. But to assume something has suddenly changed is just plain wrong. But that position simply advances the meme on the part of some members of the media that Apple is in hot water, and that explains the stock price’s erosion in recent weeks. Apple has, some claim, lost its mojo.

If anything, the iPhone 5 rates better than the infamous iPhone 4, from 2010, in one key respect: CR recommends it. In the midst of the trumped up Antennagate scandal, such as it was, CR tried to convince readers that the iPhone 4 was the one and only smartphone to exhibit severe signal degradation when held the “wrong way,” the so-called “Death Grip.” In the real world, there were YouTube videos showing other popular smartphones exhibiting similar symptoms when held in ways that covered their antenna systems. CR also ignored the telltale labels on some smartphones and the references in user guides warning customers not to hold them in certain ways because of possible signal strength problems. But I suppose CR doesn’t believe in reading manuals, because how could they miss the warnings? I guess they never saw the Apple videos or the ones on YouTube showing how other phones had reception problems.

Don’t forget how CR tried to ding the third generation iPad when some reported it ran hot under load. CR did its level best to make the iPad fail, without success. It got hot, but not too hot to recommend.

But I am not going to suggest that CR is engaged in a deep-seated conspiracy to downgrade the iPhone 5, and really wants the companies who sell Android gear to succeed. Despite the shortcomings in the magazine’s testing, they appear to be going about these reviews with good intentions. That you aren’t seeing real usability testing of the various mobile platforms probably just means that CR thinks they are very much the same in the scheme of things, or the differences aren’t significant enough to mention. To them, “Ease of use” is very much about using the various features, but nothing is said about interface niceties or even possibly significant variations from one platform to another.

So, does the tiled interface of Windows Phone make it easier to check notifications on a Nokia Lumia 920, so you can quickly get back to your work? Is the iOS better than Android, equal, or not as good? You can’t tell, because Windows Phone, Android and the iOS are all regarded by CR as “Excellent” in “Ease of use.”

Regardless of how you interpret CR’s ratings, it is not true that the iPhone 5 is not faring so well compared to the competition. However you regard CR’s ratings scheme, and not every category gets equal importance or consistent importance, the top-rated phones are mostly equal in key respects. That’s not such a bad thing, even if you disagree with the totals.

Now there is another tidbit in the February issue of CR worth mentioning: According to their testing, Windows 8 PCs are actually slower than Windows 7 PCs, though the magazine doesn’t say how much, nor do they explain how much less battery life you’ll get on a note-book. They attribute the performance disadvantage and inferior battery life to the lack of drivers for the new OS. But since developers had the prerelease versions since 2011, you wonder why it’s taking them so long — assuming that is the real reason. Shades of Windows Vista!

| Print This Article Print This Article

2 Responses to “Making Excuses for Consumer Reports”

  1. rgmenke says:

    CR will almost always rate cheaper products higher. Also, they tend not to rate on quality as much as features.

Leave Your Comment