17-55 looking dubious

Image Stabilization Tested

For the EF-S 17-55mm f/2.8 IS USM, Canon claims a three-stop advantage for its image stabilization over no image stabilization. After exhaustive testing involving well over 500 photos, I've determined that the real advantage is between 1 and 2 stops, depending on focal length as well as the acceptable hit rate. For essentially 100% success, there is only a 1-1/3-stop advantage at 55mm and a 1-stop advantage at 17mm. For a 50% or higher hit rate, there is a 2-stop advantage at 55mm and a 1-1/3-stop advantage at 17mm.

Here's the data:

10 shots each taken every 1/3rd-stop interval of shutter speed from 100% hit rate (10 for 10) down to 0%.

55mm IS Off
100% at 1/125s or faster
50% hit rate at 1/40s or faster
0% at 1/15s or slower

55mm IS On
100% at 1/50s or faster
50% at 1/10s or faster
0% at 1/5s or slower

17mm IS Off
100% at 1/30s or faster
50% at 1/13s or faster
0% at 1/5s or slower

17mm IS On
100% at 1/15s or faster
50% at 1/5s or faster
0% at 0.3s or slower


Thoughts

Altogether disappointing, but not entirely unexpected. For a 100% handheld hit rate, this lens on a Canon Digital Rebel @ ISO 3200 should be able to properly expose EV 2; at 50% hit rate, EV 0.3. I was hoping for EV -0.3.

More importantly, this seriously undermines the Canon's advantages over the Tamron. Given that the optical quality is not that different (except the Tamron looks to have significantly more CA at 17mm), the Canon's $750 premium must be justified by the extra 5mm, Ring USM, and Image Stabilization.

As I've posted before, the 5mm advantage completely disappears at close focussing distances, where the Canon becomes an 18-50mm lens. Of course, the Tamron could also have this effect, but judging from its minimum focus distance versus maximum magnification, I don't think it does (or at least not much).

As I've also posted before, there are some focus accuracy issues: it seems that this lens is more willing than the 50mm f/1.4 to report success even when it has nowhere near the correct focal distance. (Recall that the 50mm f/1.4 itself is no great shakes when it comes to autofocus accuracy.) So far, it's only happened on low contrast test charts, and I haven't found evidence of it in real world shots yet, but it begins to explain all the early shots from this lens that were soft. So far, the behavior is disturbingly reminiscent of the Tamron 28-75mm f/2.8 (the full-frame equivalent of the new Tamron 17-50mm f/2.8), and so there seems to be no Canon advantage when it comes to autofocus (at least, on a Digital Rebel).

That leaves Image Stabilization. But we've just seen that, at the most critical 17mm end, the advantage is one measly stop. That's a $750 stop. A stop that adds over 50% to the weight and volume of the lens.

Here's one person that's going to do some hard thinking about ordering a Tamron 17-50mm f/2.8 tonight.