What's better than 1 camera? A whole bunch of them! A lightfield camera is basically that: a grid of cameras that capture the same image but with slightly different data, in order to have a higher dimensionality to play with later on (allowing changing of aperture and focus post-capture-of-the-image). Here we try out to do that post processing to get artsy. Photos are taken from the The (New) Stanford Light Field Archive rectified set of chess photos. The grid consists of 289 cameras arranged in a 17x17 grid.
Depth Refocusing
The first thing we can play with is the depth of our focus. Across the many images taken by our grid of cameras, discrepancies will be more noticeable between objects closer to the camera than those further away. When we average all the color values together, it results in a very blurry foreground and a sharper background. In order to manipulate this, we shift the images by their offset from the center image (located in position 8,8, or index 144 in my list or images). To play around with this shift further (moving the focus forward and back), I multiply it by a constant f which controls how much the images are shifted. The results are as follows:
|
|
|
|
|
|
|
|
|
|
|
Aperture Adjustment
We can also play with the "aperture" of the image, or at play with its effects. A smaller aperture results in a deeper depth of focus, and a larger aperture a shallower depth of focus. We can simulate this by choosing how many pictures to include in the final image, starting from the center image again. The more we include, the blurrier the parts not "in focus" become, thus simulating a photo with a larger aperture. Radii were calculated simply as the Euclidean distance between any image's indices and the middle image's indicies (resulting in a max radius of around 11.3ish). Below are the results as I increment the radius for included images by 1 each step:
|
|
|
|
|
|
|
|
|
|
|
|
|
Bells and Whistles: My Own Image
I was curious to make my own image using these techniques. But my hands are too shaky from the excess amount of caffeine that I drink every day. The solution? I tried doing it in a video game! I play a online RPG called Final Fantasy XIV that has a flight mechanic as well as a camera mode. All I had to do was shift my character a little bit either up or sideways and I could simulate a grid of cameras somewhat decently, using a sticky note I put on my computer screen as a center reference. And in this game of dragons and magic, there is nothing more interesting than this lamp!
|
I had to go with this boring lamp for a few reasons. Time and weather, and therefore lighting, is dynamic inn FFXIV. I would not have consistent lighting if the pictures were taken "outside". Characters also frequently roam around, and there's a lot of movement, so I restricted my field of view to a small, static object. At least it glows!
Using a game definitely minimized the amount of discrepancies in between pictures. The only problem is the blur itself is not that fine-grain, because of time constraints. I only had time to take 100 screenshots (a 10x10 grid of images), so the data is not nearly as complete as those from the Stanford dataset. With more granularity and more data, the results could actually be quite nice. It's cool seeing something with "real-life" photography, namely focus, be simulated in a virtual environment. 'f' values ranged from -2 to 3 for these photos.
|
|
|
|
|
|
I also try to do the aperture effect, with radius from 0 to 5. There is a barely noticeable drift towards the left as aperture increases; this is due to the fact that there are an even number of photos, and thus no true center (and thus the more images we add, the more the data shifts slightly in one direction). But I am too lazy to take another set of pictures (it took forever :( ).
|
|
|
|
|
|
I like seeing the difference from the single screenshot to the average! In a game, there's no need to blur the background, so everything is perfectly rendered and crisp all at once. To simulate the blur effect is a new perspective that's interesting to me.
This project was fun! I was interested with Professor Ng's lightfield camera work in the past, but never realized how nice the results could be considering how the process to get them is fairly clever and simple.