Stay focused: Researchers sharpen photographs by capturing multiple low-quality images
September 30, 2009 by Larry Hardesty
In the image on the bottom, the eye is in the foreground and the text is in the background — and both are blurry because the photographer has focused on a point between the two. A new MIT system instead captures multiple images at several focal depths and stitches them into a sharper composite (top). Courtesy Sam Hasinoff
(PhysOrg.com) -- For photographers, it's sometimes difficult to keep both the foreground and background of an image in focus. Focusing somewhere between the two can ensure that neither is blurry; but neither will be particularly sharp, either. On Friday, at the IEEE Conference on Computer Vision in Kyoto, Japan, members of the MIT Graphics Group will show that combining several low-quality exposures with different focal depths can yield a sharper photo than a single, higher-quality exposure.
Given enough time, a digital camera could take a dozen well-exposed photos, and software could stitch them into a perfectly focused composite. But if the scene is changing, or if the photographer is trying to hold the camera steady by hand, there may not be time for a dozen photos. When time is short, says postdoc Sam Hasinoff, lead author on the paper, "there's a trade-off between blur, on the one hand — not having an image which is in focus — and noise, on the other. If you take an image really fast, it's really dark; it's not going to be of high quality."
Hasinoff, MIT professors Fredo Durand and William Freeman, and Kiriakos Kutulakos of the University of Toronto devised a mathematical model that determines how many exposures will yield the sharpest image given a time limit, a focal distance, and a light-meter reading. Hasinoff says that experiments in the lab, where the number and duration of digital-camera exposures were controlled by laptop, bore out the model's predictions.
A digital camera could easily store a table that specifies the ideal number of exposures for any set of circumstances, Hasinoff says, and the camera could have a distinct operational setting that invokes the table. The multiple-exposure approach, he says, offers particular advantages in low light or when the scene covers a large range of distances.
Clustered at the center of an ordinary lens filter are 12 tiny lenses with different focal depths, which project images onto different parts of a camera sensor. Credits - Courtesy Sam Hasinoff
For the time being, however, the technique is limited by the speed of camera sensors. Today's fastest consumer cameras can capture about 60 images in a second, Hasinoff says. If the MIT researchers' model determined that, under certain conditions, the ideal number of exposures in a tenth of a second would be eight, the fastest cameras could manage only six. "But there's still a big gain to be had," Hasinoff says.The Graphics Group's work on multiple-exposure composites uses an analytical approach first presented at this summer's Siggraph — the major conference in the field of computer graphics. There, Anat Levin, who was a postdoc at the time, Durand, Freeman, and colleagues described their "lattice-focal lens," an ordinary lens filter with what look like 12 tiny boxes of different heights clustered at its center. Each box is in fact a lens with a different focal length, which projects an image onto a different part of the camera's sensor. The raw image would look like gobbledygook, but the same type of algorithm that can combine multiple exposures into a coherent composite can also recover a regular photo from the raw image.
"Only time will tell whether that new, proposed piece of hardware will be better than the others, but I think their way of analyzing the whole thing is brilliant," says Marc Levoy, a professor of computer science and electrical engineering at Stanford University. "There's been a lot of work on different ways of extending the depth of field, and what this paper did was, it tried to analyze all of them together. And I actually think that it's a seminal paper. I think it's a landmark paper."
Provided by Massachusetts Institute of Technology (news : web)
Mar 19, 2008 |
not rated yet |
0
Voila yet another feature that most users will not understand and leave set at the default value. And if enough customers don't like the effect -- which they probably will not -- it will be defaulted to "off".
While we can see the details in a tunnel and the details outside of it on both sides, a digital camera is horrid at this. If this process would more focus on building cameras to see as we see, this will be great news. If all they are doing is trying to make things sharper... It's a waste of time.
I think that resolution and sharp focus are somewhat different issues here. If you have multiple out-of-focus images you could increase the resolution, but it would still be out of focus.
One problem is: how fast can a camera CCD take two pictures in a row? What they'd want to do is leave the shutter open, wipe one image, and take the next. I don't know how possible that is. The two images should be just as close in time as they can possibly be. (1/100th of a second would be a good start.)
But the problem LKD is talking about is much, much more difficult. There, it's a matter of taking photos at many exposures. The dynamic range of the human eye is something like 1 to 1,000,000. I don't know what a CCD's range is, but film was, as I remember well over 100 times less than that. Say a CCD could capture a 10th of that dynamic range at once. That means potentially taking 10 different shots. But there's a worse problem. Some of those shots are going to have a nice comfortable shutter speed, but others, 1/4 second? 1 second? 4 seconds? Blur city! Combining very dissimilar images!
Hollywood solves this problem by illuminating whole sets in the range of a film or CCD. That's a great solution, as long as one can control the lighting on the entire scene.
Adaptive optics in telescopes attempt to focus change in real time. Perhaps some of this technology could be adapted.
Just open the lens up and use as big an F number as possible, that'll make everything from 1m to Infinity in sharp focus. The real solution I'd like to see is more focal range so you can make parts of a digital picture out of focus, it's almost impossible to do that with even the most expensive non-SLR digital cameras. More exposure range would also be sweet.
Sorry but this is totally the wrong way round, big F number is correct but that means you have the lens stopped down as far as possible i.e. the aperture is very small. This means you need either a longer exposure or a much higher ISO (sensitivity) setting to take the picture, both of which bring their own problems as far as image quality goes.
I think on something as simple as barrel distortion? This is problematic in every single camera ever produced, and has yet ever to be corrected. I am not sure in my lifetime I will ever see this solved. But it would be nice! If they just made the CCD's on a convex surface, they could probably solve barrel distortion, but I don't think that is even possible with our technology.
To get a program to even fathom the intricacy of light and dark space while recording light at 1/500th of a second? I don't see it, but this is what is needed to be developed. As was stated earlier, you can already open up the depth of field by turning the camera aperture into a 19th century pin hole. Wasting time on this is not where I would like to see technology focused. But, it's not my money either. :/
It would be cool if similar technology could be applied to Digital Cameras. I don't need an 18mp sensor or whatever if I can take pictures at 5 or 6mp in natural light.
A simular idea is speckle imaging where the light output of the target is very low or passing through atmospheric polution. The problem is producing multiple images without distotion other than depth of field. I would imagine a tripod or other support would have to be used. Using large apertures uses the greater curved section of the lens with light taking longer to reach the sensor from the outer portions of the lens.