L-MOUNT Forum

Register a free account now!

If you are registered, you get access to the members only section, can participate in the buy & sell second hand forum and last but not least you can reserve your preferred username before someone else takes it.

Lenstip and L-Mount

SimonC

Well-Known Member
I've just noticed that Lenstip.com have started testing L-mount lenses. The first one tested is the Panasonic S Pro 50mm F1.4.

 
That's good to hear. Lenstip can be pretty acerbic, but their testing is pretty extensive and technical so I trust them.
 
The issue is they only test one sample each time (like many reviewers).

By example I had two copies of the Sigma 28-45mm DG DN, neither copies were similar to the test of Lenstip. By exemple at 45mm my second copy is much sharper than on the Lenstip Sample, it's not even close, it's even sharper at 45mm than at 28mm, and very close to the sharpness at 35mm.

I can see the same with the 50mm S Pro, my current sample is much sharper wide open than on their test, sharper than my Nikon 50mm Z f1.2 at f1.4 and my Nikon 50mm Z f1.8 wide open.
I know there are lot of "so so" copies of the 50mm S Pro as I owned one slightly soft wide open. But a good sample of the 50mm S Pro is much sharper than on their review.

My 35mm GM is also very similar to my 50mm S Pro wide open and slightly sharper than my Nikon 50mm f1.8 Z, so yes they make good tests, but sample variation is a real thing and it's hard to really test a lens sharpness with only one copy, especially when lenses are subject to sample variation like the S Pro line.
 
The issue is they only test one sample each time (like many reviewers).

...
This is an important point. I can see why most reviewers only test a single sample--there are budget and time constraints--but it limits the usefulness of the reviews, particularly as regards sharpness.

Some decades ago the U.S. Air Force ran tests on fast 50mm lenses from the major manufacturers to see which lens would work best for a specific surveillance application. Their budget was less constrained than that of the typical lens reviewer, and so they ordered a large number of lenses from each brand. They found a very large amount of sample variation within each brand, which exceeded the variation between brands. I suspect that these were all standard double-gauss designs common to that era, so perhaps this result is unsurprising. This didn't stop the lens testing divisions of camera magazines from declaring that a particular brand's fast 50 was superior to some other brand's fast 50, based on a single sample of each lens.

Most reviewers also fail to perform preliminary testing of the test lenses they receive to see if they have an obviously bad sample. Fred Miranda is an exception. To his great credit he tests each sample he receives for proper centering before subjecting the lens to the full battery of tests. If the lens is badly decentered--a common problem--then he rejects it and waits until he gets a properly centered copy. I have seen reviewers present their findings, and then mention that the lens is badly decentered. So what are these results supposed to tell us? That you can get a bad copy of that particular lens? Most of us already know that, which is why we test for decentering (and other obvious flaws) when we first receive a lens, before the return window has closed. What I would like to know is how a decent copy performs. The best way to determine that is with a large sample, but barring that it would be helpful if the reviewer at least conducted an initial screening to weed out the worst copies. They can then report on how many copies they had to examine to find one that isn't decentered.
 
The issue is they only test one sample each time (like many reviewers).
If one as a tester met problems, they have to ask for another -or more copy sample lenses-, before posting a review.

As @Andreas wrote, check for "decentered" lens errors in first place.
There are many "faulty" lenses.

Over the years (about 60 years), in general over the years personally I met many "faulty" copies of lenses. No brand excepted.
As also well known reviewers, that are precise and exact in their testing procedure met one or the other "copy" problems.
(E.g. Lloyd Chambers ----> https://diglloyd.com ).
-
 
That you can get a bad copy of that particular lens? Most of us already know that, which is why we test for decentering (and other obvious flaws) when we first receive a lens, before the return window has closed.
What test setup and chart do you use to test for decentering?

Also, my issue is how to determine if a corner focus is bad enough to warrant returning the lens. Maybe a more quantitive chart would help?
 
This is an important point. I can see why most reviewers only test a single sample--there are budget and time constraints--but it limits the usefulness of the reviews, particularly as regards sharpness.

Some decades ago the U.S. Air Force ran tests on fast 50mm lenses from the major manufacturers to see which lens would work best for a specific surveillance application. Their budget was less constrained than that of the typical lens reviewer, and so they ordered a large number of lenses from each brand. They found a very large amount of sample variation within each brand, which exceeded the variation between brands. I suspect that these were all standard double-gauss designs common to that era, so perhaps this result is unsurprising. This didn't stop the lens testing divisions of camera magazines from declaring that a particular brand's fast 50 was superior to some other brand's fast 50, based on a single sample of each lens.

Most reviewers also fail to perform preliminary testing of the test lenses they receive to see if they have an obviously bad sample. Fred Miranda is an exception. To his great credit he tests each sample he receives for proper centering before subjecting the lens to the full battery of tests. If the lens is badly decentered--a common problem--then he rejects it and waits until he gets a properly centered copy. I have seen reviewers present their findings, and then mention that the lens is badly decentered. So what are these results supposed to tell us? That you can get a bad copy of that particular lens? Most of us already know that, which is why we test for decentering (and other obvious flaws) when we first receive a lens, before the return window has closed. What I would like to know is how a decent copy performs. The best way to determine that is with a large sample, but barring that it would be helpful if the reviewer at least conducted an initial screening to weed out the worst copies. They can then report on how many copies they had to examine to find one that isn't decentered.
This is a great post. Copy variation is a real problem if you're picky about lens performance. It frustrates the hell out of me that apart from Sigma (and maybe Leica), manufacturers don't do individual final inspection optical testing on their lenses. It's no surprise that there is so much variation if there is no final QA test on every copy.

My experience with Panasonic lenses has been very mixed. I immediately got good ccopies of the 14-28, 20-60, and the 50mm f1.8. But I needed multiple goes with the 70-300 and 70-200 f4. I also had to buy two used copies of the 24-105 before I got a good one.

For comparison, both Sigma lenses I have are perfect - the 35mm f2 i-Series and the 105mm f2.8 macro (although the latter I bought used). Of course, the sample size is way too low to come to any conclusion.
 
What test setup and chart do you use to test for decentering?

Also, my issue is how to determine if a corner focus is bad enough to warrant returning the lens. Maybe a more quantitive chart would help?
I have tried a lot of different methods to test for decentering, all of which work to some degree. I have settled on a method that does not use a chart. Instead, I focus on a distant object (more on this later), and then take a series of images. For each shot I recompose--without changing the focus--so as to place the same object in each of the corners. I then compare the relative sharpness of the same object in all four corners. With one form of decentering the subject will be much sharper in one corner than in the opposing corner. With another form of decentering there is what looks like a parallel tilt, where the left-side corners are much sharper than the right-side corners (or vice-versa), or where the top corners are much sharper than the bottom corners. With a well-centered lens all the corners are equally sharp, or very nearly so.

A few notes on this method:
- I choose a subject that has good contrast and small details so that differences are easy to see. The size of the chosen object depends on the focal length of the lens.
- If the lens has a lot of field curvature then choice of the initial fixed focus point will need to be near the edge (or corner) of the frame, or else none of the corners will be in focus.
- If the lens has no sharp corners at any f-stop then you need to change the comparison zones closer to the center.
- It is critically important that the focus does not change between shots.
- It is important to use a tripod (and perhaps pre-fire, depending on the shutter speed). It's best to do this on a calm day.
- If I find a problem with a lens I repeat the test at least once, to make sure I am not seeing an artifact from a botched test. This is important.
- With zoom lenses you need to test at multiple focus lengths, since the lens might be well centered but not over the entire range.
- It is also useful to test at different f-stops. If buying a lens for landscape use you might be satisfied if the lens shows adequate corner consistency from, say, f/4.

With a really well-centered lens it is stunning how similar the corners look. That's always exciting to see! On the other hand, it can be quite stomach-turning to see how far off a lens can be. Of course we can't expect perfection. So how bad is too bad? That's subjective. After testing hundreds of lenses I have a sense of what I can reasonably expect to see. I feel that my standards are not too high. Retailers never give me a hard time when returning a lens for decentering (they know it's an issue), and even private sellers are typically very cooperative (and even apologetic) when I show them the photographic evidence.

While I haven't become less discerning over time I have become somewhat more understanding of the difficulties faced by manufacturers. It's hard to get this right. I have this learned as a result of trying to re-center old lenses that I adapt. The use of adapters can throw off a well centered lens. In that case I modify the adapter with thin shims to bring the lens-adapter set back into a centered state. These shims are SUPER thin. For my most commonly used shim it takes about 4-5 shims to equal the thickness of a piece of standard cellophane tape. One or two shims of this thickness can throw off the lens, or correct it. The tolerances are mind blowing. On the other hand, some manufacturers do a much better job than others at meeting this challenge. As Paul notes, Sigma and Leica are particularly praiseworthy. I have been extremely impressed with the modern Sigma lenses I have purchased, to the point where I am surprised when I see an issue. Obviously they are doing something right, both in the manufacturing process and with their post-assembly testing. It almost seems like they care. :)
 
Here is another way to test a lens:

There is a lot of great information on the lensrentals site. I've learned a lot from Robert Cicala through the years, but I prefer shooting real-world objects when testing for decentering because it gives me a better sense of the implications for the final image.
 
While I haven't become less discerning over time I have become somewhat more understanding of the difficulties faced by manufacturers. It's hard to get this right.
True. In the past I've been involved in having lenses manufactured for use in induatrial equipment, so I've experienced how hard this can be. I have a little pet peeve that this is always called decentering in the photography world, but there are so many things that can go wrong with lenses other than decentering. Nevertheless the way to measure whether a lens is good or bad is pretty much the same no matter the manufacturing defect(s).

... but I prefer shooting real-world objects when testing for decentering because it gives me a better sense of the implications for the final image.

I like your method and will give it a try. Steps beyond that would be to measure things like line pair resolution, contrast, radial optical distortion, etc. We do a lot of this testing with industrial lenses, with test charts, because overall system performance depends on lenses within defined specifications. But with photography I agree, I like real-world test objects.
 
Back
Top