Archive for the 'photography' Category

Spherical panoramas: photographs without edges

Most lay-people see photography as the cut-and-dried act of capturing the world “as it is”. Ironically, the simple and necessary action of framing a photograph instantly renders the result subjective (’un-photogenic’ subjects are typically pushed out-of-frame). This creative choice built into the camera’s design is part of the appeal of photography. However, we occasionally encounter entirely photogenic scenes that we wish to convey in their entirety. Luckily, there is a type of photograph that has no edges at all; the spherical panorama.

Moreton island sand-blow (summit)29th Dec 08 map HD (2.6MB)

This spherical panorama captures the complete scene; from the zenith (above my head) to the complete horizon to the the nadir (under my feet). No photographic paraphernalia (camera, tripod, photographer) distracts the viewer from the scene. What kind of magical camera does this? Well, with the right know-how, software and technique any camera will do. This is because spherical panoramas are simply an amalgamation of many regular photographs. Here’s the kit I used to make my example panorama.

tripod, nodal ninja panoramic head and Canon 5D mark II
Tripod fitted with a Nodal Ninja 5 panoramic tripod head supporting a Canon 5D mark II with Canon 16 - 35mm f2.8L lens

To get started we need a sequence of regular photographs which cover the entire scene. Typically this involves taking several ‘rows’ of images, each containing a constant number of images (columns). The number of rows and columns required depends on the lens used but nevertheless the angle between rows should be consistent, as should that between each column (made easy by using a panoramic tripod head, e.g. Nodal Ninja 5). In addition, a zenith shot (directly up) and two (complimentary) ‘nadir’ shots (directly down) are required. Here’s the sequence of images I used to produce my sample panorama.

zenith rowzenithsky rowsky rowzenith rowhorizon rowzenith rowground rowzenith rownadir (taking two at opposites enables later removal of feet & shadows)

The rest of the spherical panorama creation process is all in software. I plan to cover the details in a future screen-cast, but for now here’s a quick overview:

  1. Canon Digitial Photo Professional (needs a better logo)Ensure ‘consistent colour’ across shots by correcting white-balance discrepancies (tip: remember to lock white-balance, exposure, shutter-speed, focal length and focus while shooting). Also correct any vignetting (dark edges) and chromatic aberration (coloured fringing on detail). For these tasks I use Canon’s bundled DPP software (if Lightroom’s vignetting correction tools were better, I’d use it instead).
  2. PTGuiImport the corrected images into a stitching tool (after trying a few apps I finally ponied up for a copy of PTGui). By creating control-point pairs (i.e. locating similar features in different photographs) I give PTGui the necessary information to correctly align, warp, composite and blend the all images (except for the zenith and nadir) into a single ’stitched’ equirectilinear result.
  3. In photoshop I work a little magic to combine the two original nadir shots (which both contain my legs and feet) into a single feet-free nadir image
  4. Using PTGui again I produce another equirectilinear image, this time containing only the warped ground (from the composited nadir) and sky (from the zenith shot)
  5. Photoshop logoBack in photoshop I combine both equirectilinear images into a single image (’patching’) and apply any necessary retouching (e.g. making horizons dead-level and apply colour-corrections) to produce the pièce de résistance: the final (70+ megapixel) image.

patching the warped nadir and zenith onto the stitched panorama in photoshop
patching the warped nadir and zenith onto the stitched panorama in photoshop

At first glance this appears to be a regular photo, but in fact it covers 360° horizontally and 180° vertically (i.e. the entire scene). So how does it justify the ’spherical’ and ‘edge-less’ tags? Well, the most effective way to present such an image is to wrap the rectangular image around an imaginary sphere and give the viewer control of a virtual camera placed at the orb’s centre. Rotating the camera simulates ‘looking’ in the original scene. There are no limits on where the camera may look and no visible edges or seams.

the distortion in the equirectilinear image disappears when the image is wrapped around a sphere
the distortion in the equirectilinear image disappears when the image is wrapped around a sphere (inside which the viewer’s virtual ‘camera’ rotates)

Spherical panoramas really take on a new dimension when combined with software like the PangeaVR iPhone app I’ve mentioned before. Wherever I am I can show friends exactly what my favourite places are like by handing them the phone and saying “drag the image with your finger”. No other explanation, instruction or intervention is required for me to show friends overseas what the beach I learned to surf at is “really like”. Of course, the panoramas can also be viewed in a web browser. Below are links to a few panoramas I’ve created in the past few months from images taken on my Christmas holiday back home.

view 'moreton island sand blow (summit)' panorama view 'moreton island sand blow (saddle)' panorama view 'Moreton island sand blow (crest, north)' panorama
view 'Moreton island sand blow (crest, middle)' panorama view 'Moreton island sand blow (crest, south) view 'Moreton island eastern beach (dune)' panorama
view 'Moreton island eastern beach (shore)' panorama view 'Palm beach avenue beach (1)' panorama view 'Palm beach avenue beach (2)' panorama

Hopefully I’ll have more to add to my panoramas page soon.

Immersive photographs on the iPhone with PangeaVR

It was all I could do to stay true to my anti-iPhone stance so long as my 3G demands remained unmet, so when apple finally relented and released the iPhone 3G I too was forced to concede (it would have been unreasonable of me not too!).

Most curious onlookers are sold after a whistle-stop tour of SMS, Web and Google Maps. However some stubborn folks haughtily demand what makes ‘it so special anyway?’, so I do my solemn best to give them the personalised excuse they need to justify their inevitable purchase. Twitterific seems to be the killer app for folks on twitter (twits?) and the common craft intro to twitter video quickly converts the non-twits. Productivity obsessives (guilty) have trouble restraining themselves when they see remember the milk and reqall. Some nuts are tougher to crack.

The final trick I have up my sleeve (besides games, but that’s cheap) is PangeaVR (app-store link).

PangeaVR iPhone ap displaying 'boat house' panorama from panedia.com
PangeaVR iPhone app displaying ‘boat house‘ panorama from panedia.com

It’s not the most snappily-named app, nor the most seductively dressed but it’s the app that’s come closest to eliciting an audible ‘wow’ from me. In short PangeaVR allows you to view immersive photographs by rubbing your finger on the touch-screen. Drag left to look left, up is up etc. Spin around, zoom in, stare at a foreign sky. See the world not as the photographer frames it, but as they saw it (there’s a big difference). Revisit the places of your youth. Take a peek at that fantasy vacation spot. Show your friends the home town you never shut up about.

example list of panoramas within a PangeaVR portfolio PangeaVR is (amazingly) free, though I reckon that with a less imposing UI metaphor (PangeaVR’s current UI is based around an intimidatingly long list of ‘portfolios’) and a Koi Pond-like price tag, PangeaVR could be a major money spinner for Pangea software (should they choose to go that route). The coolest part about PangeaVR as it stands though is that publishing your own panoramas through the system is also free and (theoretically) within reach of anyone with a digital camera (preferably with a wide lens and a bracket head), a PC and time! Brian Greenstone and the Pangea software guys responsible for this app/system deserve major kudos for their open-handed approach.

Now I should point out that this isn’t the first time I’ve encountered panoramic photography, so my praise isn’t just that of the VR ingénue. I’ve toiled to stitch my own panoramas in the past only to have users unable to load the Quicktime/flash/java plug-in necessary to view them or confused that they must ‘click & drag’ the photo (to which they are unaccustomed) to ‘pan’ the image (some very unhelpful lingo). In the end I gave up and started outputting stitched images as static JPEGs. What makes PangeaVR worth mentioning is that it harnesses people’s natural tendency to touch screens (even when they really shouldn’t) and the ability of even the most timid digital immigrant to comprehend the ‘move the picture with your finger’ concept.

My next move is to devise a workflow for producing my own spherical panoramas without splashing out on any specialist equipment.

Photos — trip to Salzburg & Munich

flickr set: Salzburg and Munich

Just a quick post (to offset my last ridiculously long post on stitching panoramas with Hugin). I’ve finally posted the photos from my recent trip to Salzburg & Munich. It took a few weeks (spare-time) to cull (from ~400 to 85), caption, tag, geocode, post-process and upload the images.

I’m still using the workflow described in Photography ‘08 (ditching iphoto) with a few minor modifications:

  • Re-reading the metadata into Lightroom after geocoding in PictureSync. Otherwise Lightroom is unaware of the geotags and may subsequently overwrite them.
  • I’ve also started exporting temp images from Lightroom (which have the ‘develop’ settings applied) as PictureSync seems unable to accurately apply the crops/tone adjustments defined in Lightroom. The temp images are uploaded and deleted.

Enjoy the pics. Also check out Robert’s pictures of the hike.

Making nodal panoramas that don’t suck

Update: 6th Feb 2010, changed ‘linear’ in title to ‘nodal’ due to comment feedback

Most modern digital cameras (even phones) have a ‘panorama photo’ mode, allowing you to take multiple photos which are then magically combined into a single large photograph. Sounds great, but there’s a problem. These composite images (whether produced in-camera or using ‘automatic stitching’ programs) look OK at first glance, but are typically full of smudges, ghosts, morphing people and objects sprouting from thin air.

Issues with automatic panorama stitching software

So where exactly is is all going wrong? In theory, the basic steps required to create convincing composite images (including panoramas) are:

  1. Align the images — Ordering and placing images by examining their overlaps.
  2. Correct for angle of view — Think ‘fisheye’, but subtler.
  3. Remap images to compensate for varying distance — Closer subjects ‘move’ more between shots than distant ones. This is fixed by identifying common ‘control points’ then remapping the images so they overlap perfectly.
  4. Blend images to hide image seams — Obscuring (typically softening) the boundaries between images to create the illusion of a single image.

Photostitching software typically gets steps 1 and 2 spot on, but can’t hold a candle to humans when it comes to pattern recognition (the secret sauce behind steps 3 and 4). If we want good composite images we have to hijack the identification of control points and eventually produce a layered photoshop file so we can blend manually (to perfection).

Pitfalls during shooting

First we need to keep an eye out for things that often go pear-shaped during the shoot.

  • Creeping Horizon — Caused by gradually rotating the camera up/down between shots (when doing a horizontal pano).
  • Wavy horizons — If the camera’s not perfectly level, the horizon will see-saw up & down.
  • Moving foreground subjects — People/cars/trees/etc moving between shots.
  • Moving photographer — Wind, terrain and rampaging wild-life may interrupt your plans.
  • Rubbish/damaged lenses — Cheap cameras have cheap lenses whose variations become glaringly obvious during stitching.
  • Varying ‘brightness’ — Not locking exposure results in adjacent shots with lighter or darker skies.
  • Varying ‘colour’ — Not locking white-balance (when multiple light sources present) can result in dramatically different colour between shots.
  • Changing light — If clouds arrive halfway through shooting your panorama, give up or start again.

Tips for shooting

  • Use a tripod
  • Level your tripod (camera’s rotation axis should typically be perpendicular to the horizon).
  • Be aware of moving objects near the camera (remember to check 360°). Fast moving objects are generally OK. It’s the slow moving object that cause headaches later on.
  • Take more shots than the camera recommends (only if using manual stitching). Using 2/3 overlap will give you a spare photoshop layer during blending.
  • Be quick. Light and subjects will change during your shoot. Plan your panorama, do a dry run, then fly like the wind (but don’t forget the other tips).
  • Lock any camera settings you can, leave the rest alone. Some cameras lock exposure and white balance in ‘panorama’ mode (check the manual), but it’s up to you to keep focal length (zoom), aperture, focus and ISO constant.

Playing with pixels — stitching an ‘unstitchable’ panorama

All the apps mentioned in this section are available for Windows and OS X (linux too, though using gimp instead of photoshop). Also it’s worth pointing out that even though this post is about horizontal panoramas, these techniques can be be used to create any composite image (blended tiled images, unblended tiled images, spherical panoramas, gigapixel images etc).

  1. Shoot the images & collect on PC — I’m going to use images taken atop Gaisberg mountain near Salzburg, Austria. The images were shot on my Canon Powershot G7 in it’s Stitch Assist mode (exposure & white balance locked). To make things interesting I’ve chosen a set of images with a few of the common issues you may come across (no tripod, inconsistent position).

  2. hugin logoDownload and extract Hugin — Hugin is an ambitious open source (donation-ware) panorama creation GUI (cross-platform, written in c# under mono). It’s still beta so you may find the UI a little cryptic and glitchy, but you can’t complain given the price-tag (free).

  3. Download and install autopano-sift (optional) — Autopano-sift is a helper program for Hugin which identifies control points in your sub-panorama images using some pretty hefty maths. We will use autopano-sift to kick-start the control point detection process, but you can bypass autopano-sift and create all control points manually if you wish.

  4. Open Hugin and load images — When Hugin loads click the ‘load images’ button and locate the images which make up your panorama (some camera naming schemes make this difficult so it’s best to collect them in a folder beforehand). Hugin may ask you to specify your cameras stated focal length (usually silk-screened around the lens itself) and 35mm equivalent (try dpreview or the manual).

    Hugin startup screen

    Once Hugin has loaded your images, autopano-sift will appear (on first-run, Hugin may ask you to locate the autopano.exe).

  5. Use autopano-sift to identify control points (optional) — The autopano-sift UI has a few tweak-able parameters for the adventurous but for now just click ‘compute’.

    autopano-sift UI

    Autopano-sift’s ‘PTO generation’ process will kick off. This process may take a few minutes (depending on the size & number of your images). Once completed, click ‘OK’ and close autopano-sift.

  6. The rough draft — Hugin will use the control points generated by autopano-sift to align and distort your images. It will present its results in the ‘preview’ window.

    initial panorama results

    As you can see, there are some misaligments. These are to be expected (especially with dodgy source images like the ones I’m using) but unlike the automatic stitching software which would just blend over these cracks, we have the power to fix them.

  7. Get to know the ‘control points’ tab — Close the preview window and navigate to the ‘control points’ tab.

    hugin control points tab

    The ‘control points’ tab enables you to review, edit, delete and add control points to improve the stitching & blending processes. Additionally, it gives a unique perspective on exactly why automated control point detection is so underwhelming. Main points:

    1. Existing control-points on both images (automatically generated by autopano-sift).
    2. We’re only really interested in the area of overlap. Ideally there would be control points evenly distributed througout this entire overlap region.
    3. The list of control points (automatic and manual).
  8. Manually tweaking control points — The ‘control points’ tab in the current build of Hugin (0.7 beta 4) has some frustrating UI glitches, but the following workarounds helped me:

    • When editing existing control points, set the zoom factor to ‘200%’ (be sure to click them dead centre or Hugin will try to create a new control point, frustrating).
    • When adding new control points, set the zoom factor to ‘fit to window’ then click on both images in turn, position the control points, click the ‘fine-tune’ button and then the ‘add’ button. repeat.

    Add a few control points for each troublesome image pair (evenly throughout the overlap area) then go to the ‘optimize’ tab, choose ‘everything’ and click the ‘optimize’ button. Once the optimization process is complete, check the preview (ctrl-p on windows, cmd-P on mac) to see whether it’s made an improvement to the panorama. If there are still major breaks, tweak the control points as necessary.

    improved stitch

    Here you can see the panorama preview is less ‘wavy’. Don’t worry too much about the rotation, we’ll fix this in photoshop.

  9. Get PTStitcher — Although Hugin comes with a stitching engine (nona), we’ll use PTStitcher as it has better blending plus .psd output. Download the PanoTools zip and extract it. Copy the pano12.dll from the extracted folder into your Hugin folder. Copy the PTStitcher.exe app (from the Helpers folder in the extracted PanoTools folder) to the Hugin folder.

  10. Creating a stitched (layered) psd file — In Hugin, go to the ‘Stitcher’ tab, change the ’stitch the images’ option to ‘into a layered photoshop file’ and click ’stitch now’ (beware the psd file may be several hundred megabytes, hope you’ve got the RAM). Hugin may ask you to locate PTStitcher. If so point it to the copy you put in the Hugin folder.

    Hugin stitcher options
  11. Final blending & cropping in photoshop — Upon opening the psd file in photoshop you’ll notice that the blend lines are pretty conspicuous. However, as each shot is a separate (masked) layer you can tweak each blend independently by editing the mask.

    blend issues

    At this point you also have the opportunity to mask in/out moving foreground subjects and make other creative edits (i.e. duplicate people). If you took my advice and shot with 2/3 overlap you’ll have a spare layer to run wild with too. A few blending tips:

    • Use broad diameter ’soft’ brushes to feather the masks in areas of sky.
    • Avoid straight lines in your masks (especially horizontal & vertical lines). Human eyes will spot those straight away.
    • In areas of high detail use small diameter ‘harder’ edged brushes (this avoid ghosting in the final image).
    • Hard edged subjects in the foreground make great (hard) mask boundaries

    After tweaking each of the seams the whole panorama becomes pretty convincing. This is the most rewarding part of the process, and you run the risk of wasting a lot of time ;). In this example I decided to merge foreground characters from two different shots, but I left some easter eggs in there for anyone who’s really looking.

    resolved blend issues
  12. The final cut — To get rid of the ‘rising horizon’ problem in this image set (a constant issue when shooting panoramas sans tripod) I’ve selectively cloned and distorted the middle foreground grass and left sky. The final image captures all the things I remember from that instant in one single image (the ultimate aide mémoire).

    final image

The length of this post might make the process of manually assembling composite/panoramic images appear complex, but trust me, the second time’s a snap. Oh, and you’ll be the only kid on your street who’s panoramas don’t suck.

dpreview.com lens review widget sallies forth

Today’s a big day at dpreview.com as it sees the launch of the first dpreview.com lens reviews. I’m excited as it also sees the launch of my latest and greatest project, the humbly-titled lens review widget (or see it embedded in a lens review).

Screen shot of the lens review widget in sharpness mode

The lens review widget is a flash component embedded into our lens reviews responsible for presenting the results of our studio tests. The challenge of the project is that the lens review studio test output is intricate, multi-dimensional and in acute danger of misinterpretation.

The widget’s goal is to provide users with a constructive mental model of this dataset and an intuitive/responsive/encouraging interface with which to navigate it.

The broad data categories initially displayed are:

  • sharpness demonstrationSharpness is the top of the list for most lens geeks. Not all lenses are created equal, especially considering the difference in the ’sharpness’ of the image they produce. A single lens can vary greatly between different focal length/aperture combinations and even varies (non-linearly) from centre to corner within a single shot!
  • chromatic aberration demonstrationChromatic aberration is the colour ‘fringing’ you see around light-on-dark or dark-on-light objects in your photographs. Generally purple or green (but occasionally blue/yellow) this effect varies with focal length/aperture and distance from centre.
  • distortion demonstrationDistortion is what causes real world straight lines to appear curved in your photographs. Zoom lenses in particular are prone to distorting ‘inwards’ (pincushion) at one extreme and ‘outwards’ (barrel) at the other. Distortion varies by focal length.
  • falloff demonstrationFalloff is the phenomenon whereby the corners of the frame appear ‘darker’ than the centre (i.e. the light ‘falls off’ from centre to edge). Though often used as an intentional effect, falloff is generally avoided if possible and varies with focal length, aperture and obviously, distance from centre.

In practice these 4 effects are all present (to some extent) in every shot you take, so representing their individual effects is more complicated than simply displaying the raw images (which we also do).

Designing the user interface

With so much data to navigate and several dimensions to deal with (angle from centre, distance from centre, focal length and aperture) the process of designing a useful and intuitive UI was difficult. Eventually a standard hierarchy emerged:

  1. Lens review. It may seem the obvious choice, but I’ve included it on this list as (if allowed in configuration) the user is able to change the review data being viewed within the widget.
  2. Visualization. Sharpness & Chromatic Aberration, Distortion or Fall-off.
  3. Focal Length. Or ‘zoom’ to non-photographers, is generally marked on lenses in arbitrary integer values (generally 4 - 7 per lens).
  4. Aperture. The variable size of the ‘hole’ which regulates the amount of light entering the camera body. Stated in f numbers, an approximately logarithmic semi-standardized decimal value. The aperture values available for a lens depend on the focal length chosen.
  5. Comparison lens review. Users reading reviews are constantly making comparative decisions, so the widget allows users to compare two lens reviews (or two data-points within the same review) cheek-by-jowl, hopefully improving comprehension. Of course, this requires user interface chrome to allow the user to choose.

picker picker controlThere was much internal debate (which still rages) about this UI element which allows users to change/compare reviews, but the eventual decision was to filter reviews by system, lens and finally camera body. Note: Our lens tests are performed on a camera body (instead of an optical test bench) which has relevance upon the results.

The ’slider’

The real challenge from a UI perspective was creating a UI element which allowed users to change focal length / aperture in an intuitive way, was brand-neutral and could cope with the intricacies of the focal length /aperture systems and their inter-relationship. After initially experimenting with flash’s slider component, I decided to abandon it and start from scratch (before you shout Not Invented Here, be aware that flash’s slider is a major usability dud).

GrabSlider demo animation

Trim

Some neat little features that I pushed into the widget (usually late at night):

  • Full screen mode: At any time a user can click the full screen button at the bottom of the widget to view in a new window/tab (selected reviews, visualization, focal length & aperture are retained).
  • Permalinking: At any point, users can obtain a permalink to the current state of the widget to bookmark and/or link to (again the selected reviews, visualization, focal length & aperture are retained).
  • Keyboard controls: A bit of a power-user feature requested by the guys in the office, the keyboard arrow keys can be used to control focal length and aperture. If multiple reviews are visible, they will all receive the inputs, allowing users to change the visualizations in unison (not currently possible via the sliders).

Though a few minor issues persist, I’m very happy with the current implementation. I feel it strikes the right balance between aesthetics, usability, accuracy, brand-neutrality and ‘fun’.

Time (and the forums) will ultimately tell.