Most curious onlookers are sold after a whistle-stop tour of SMS, Web and Google Maps. However some stubborn folks haughtily demand what makes ‘it so special anyway?’, so I do my solemn best to give them the personalised excuse they need to justify their inevitable purchase. Twitterific seems to be the killer app for folks on twitter (twits?) and the common craft intro to twitter video quickly converts the non-twits. Productivity obsessives (guilty) have trouble restraining themselves when they see remember the milk and reqall. Some nuts are tougher to crack.

The final trick I have up my sleeve (besides games, but that’s cheap) is PangeaVR (app-store link).

PangeaVR iPhone app displaying ‘boat house‘ panorama from panedia.com

It’s not the most snappily-named app, nor the most seductively dressed but it’s the app that’s come closest to eliciting an audible ‘wow’ from me. In short PangeaVR allows you to view immersive photographs by rubbing your finger on the touch-screen. Drag left to look left, up is up etc. Spin around, zoom in, stare at a foreign sky. See the world not as the photographer frames it, but as they saw it (there’s a big difference). Revisit the places of your youth. Take a peek at that fantasy vacation spot. Show your friends the home town you never shut up about.

PangeaVR is (amazingly) free, though I reckon that with a less imposing UI metaphor (PangeaVR’s current UI is based around an intimidatingly long list of ‘portfolios’) and a Koi Pond-like price tag, PangeaVR could be a major money spinner for Pangea software (should they choose to go that route). The coolest part about PangeaVR as it stands though is that publishing your own panoramas through the system is also free and (theoretically) within reach of anyone with a digital camera (preferably with a wide lens and a bracket head), a PC and time! Brian Greenstone and the Pangea software guys responsible for this app/system deserve major kudos for their open-handed approach.

Now I should point out that this isn’t the first time I’ve encountered panoramic photography, so my praise isn’t just that of the VR ingénue. I’ve toiled to stitch my own panoramas in the past only to have users unable to load the Quicktime/flash/java plug-in necessary to view them or confused that they must ‘click & drag’ the photo (to which they are unaccustomed) to ‘pan’ the image (some very unhelpful lingo). In the end I gave up and started outputting stitched images as static JPEGs. What makes PangeaVR worth mentioning is that it harnesses people’s natural tendency to touch screens (even when they really shouldn’t) and the ability of even the most timid digital immigrant to comprehend the ‘move the picture with your finger’ concept.

My next move is to devise a workflow for producing my own spherical panoramas without splashing out on any specialist equipment.

Just a quick post (to offset my last ridiculously long post on stitching panoramas with Hugin). I’ve finally posted the photos from my recent trip to Salzburg & Munich. It took a few weeks (spare-time) to cull (from ~400 to 85), caption, tag, geocode, post-process and upload the images.

I’m still using the workflow described in Photography ‘08 (ditching iphoto) with a few minor modifications:

• Re-reading the metadata into Lightroom after geocoding in PictureSync. Otherwise Lightroom is unaware of the geotags and may subsequently overwrite them.
• I’ve also started exporting temp images from Lightroom (which have the ‘develop’ settings applied) as PictureSync seems unable to accurately apply the crops/tone adjustments defined in Lightroom. The temp images are uploaded and deleted.

Enjoy the pics. Also check out Robert’s pictures of the hike.

So where exactly is is all going wrong? In theory, the basic steps required to create convincing composite images (including panoramas) are:

1. Align the images — Ordering and placing images by examining their overlaps.
2. Correct for angle of view — Think ‘fisheye’, but subtler.
3. Remap images to compensate for varying distance — Closer subjects ‘move’ more between shots than distant ones. This is fixed by identifying common ‘control points’ then remapping the images so they overlap perfectly.
4. Blend images to hide image seams — Obscuring (typically softening) the boundaries between images to create the illusion of a single image.

Photostitching software typically gets steps 1 and 2 spot on, but can’t hold a candle to humans when it comes to pattern recognition (the secret sauce behind steps 3 and 4). If we want good composite images we have to hijack the identification of control points and eventually produce a layered photoshop file so we can blend manually (to perfection).

Pitfalls during shooting

First we need to keep an eye out for things that often go pear-shaped during the shoot.

• Creeping Horizon — Caused by gradually rotating the camera up/down between shots (when doing a horizontal pano).
• Wavy horizons — If the camera’s not perfectly level, the horizon will see-saw up & down.
• Moving foreground subjects — People/cars/trees/etc moving between shots.
• Moving photographer — Wind, terrain and rampaging wild-life may interrupt your plans.
• Rubbish/damaged lenses — Cheap cameras have cheap lenses whose variations become glaringly obvious during stitching.
• Varying ‘brightness’ — Not locking exposure results in adjacent shots with lighter or darker skies.
• Varying ‘colour’ — Not locking white-balance (when multiple light sources present) can result in dramatically different colour between shots.
• Changing light — If clouds arrive halfway through shooting your panorama, give up or start again.

Tips for shooting

• Use a tripod
• Level your tripod (camera’s rotation axis should typically be perpendicular to the horizon).
• Be aware of moving objects near the camera (remember to check 360°). Fast moving objects are generally OK. It’s the slow moving object that cause headaches later on.
• Take more shots than the camera recommends (only if using manual stitching). Using 2/3 overlap will give you a spare photoshop layer during blending.
• Be quick. Light and subjects will change during your shoot. Plan your panorama, do a dry run, then fly like the wind (but don’t forget the other tips).
• Lock any camera settings you can, leave the rest alone. Some cameras lock exposure and white balance in ‘panorama’ mode (check the manual), but it’s up to you to keep focal length (zoom), aperture, focus and ISO constant.

Playing with pixels — stitching an ‘unstitchable’ panorama

All the apps mentioned in this section are available for Windows and OS X (linux too, though using gimp instead of photoshop). Also it’s worth pointing out that even though this post is about horizontal panoramas, these techniques can be be used to create any composite image (blended tiled images, unblended tiled images, spherical panoramas, gigapixel images etc).

1. Shoot the images & collect on PC — I’m going to use images taken atop Gaisberg mountain near Salzburg, Austria. The images were shot on my Canon Powershot G7 in it’s Stitch Assist mode (exposure & white balance locked). To make things interesting I’ve chosen a set of images with a few of the common issues you may come across (no tripod, inconsistent position).

2. Download and extract Hugin — Hugin is an ambitious open source (donation-ware) panorama creation GUI (cross-platform, written in c# under mono). It’s still beta so you may find the UI a little cryptic and glitchy, but you can’t complain given the price-tag (free).

3. Download and install autopano-sift (optional) — Autopano-sift is a helper program for Hugin which identifies control points in your sub-panorama images using some pretty hefty maths. We will use autopano-sift to kick-start the control point detection process, but you can bypass autopano-sift and create all control points manually if you wish.

4. Open Hugin and load images — When Hugin loads click the ‘load images’ button and locate the images which make up your panorama (some camera naming schemes make this difficult so it’s best to collect them in a folder beforehand). Hugin may ask you to specify your cameras stated focal length (usually silk-screened around the lens itself) and 35mm equivalent (try dpreview or the manual).

   

   Once Hugin has loaded your images, autopano-sift will appear (on first-run, Hugin may ask you to locate the autopano.exe).

5. Use autopano-sift to identify control points (optional) — The autopano-sift UI has a few tweak-able parameters for the adventurous but for now just click ‘compute’.

   

   Autopano-sift’s ‘PTO generation’ process will kick off. This process may take a few minutes (depending on the size & number of your images). Once completed, click ‘OK’ and close autopano-sift.

6. The rough draft — Hugin will use the control points generated by autopano-sift to align and distort your images. It will present its results in the ‘preview’ window.

   

   As you can see, there are some misaligments. These are to be expected (especially with dodgy source images like the ones I’m using) but unlike the automatic stitching software which would just blend over these cracks, we have the power to fix them.

7. Get to know the ‘control points’ tab — Close the preview window and navigate to the ‘control points’ tab.

   

   The ‘control points’ tab enables you to review, edit, delete and add control points to improve the stitching & blending processes. Additionally, it gives a unique perspective on exactly why automated control point detection is so underwhelming. Main points:

   1. Existing control-points on both images (automatically generated by autopano-sift).
   2. We’re only really interested in the area of overlap. Ideally there would be control points evenly distributed througout this entire overlap region.
   3. The list of control points (automatic and manual).

8. Manually tweaking control points — The ‘control points’ tab in the current build of Hugin (0.7 beta 4) has some frustrating UI glitches, but the following workarounds helped me:

   • When editing existing control points, set the zoom factor to ‘200%’ (be sure to click them dead centre or Hugin will try to create a new control point, frustrating).
   • When adding new control points, set the zoom factor to ‘fit to window’ then click on both images in turn, position the control points, click the ‘fine-tune’ button and then the ‘add’ button. repeat.

   Add a few control points for each troublesome image pair (evenly throughout the overlap area) then go to the ‘optimize’ tab, choose ‘everything’ and click the ‘optimize’ button. Once the optimization process is complete, check the preview (ctrl-p on windows, cmd-P on mac) to see whether it’s made an improvement to the panorama. If there are still major breaks, tweak the control points as necessary.

   

   Here you can see the panorama preview is less ‘wavy’. Don’t worry too much about the rotation, we’ll fix this in photoshop.

9. Get PTStitcher — Although Hugin comes with a stitching engine (nona), we’ll use PTStitcher as it has better blending plus .psd output. Download the PanoTools zip and extract it. Copy the pano12.dll from the extracted folder into your Hugin folder. Copy the PTStitcher.exe app (from the Helpers folder in the extracted PanoTools folder) to the Hugin folder.

10. Creating a stitched (layered) psd file — In Hugin, go to the ‘Stitcher’ tab, change the ’stitch the images’ option to ‘into a layered photoshop file’ and click ’stitch now’ (beware the psd file may be several hundred megabytes, hope you’ve got the RAM). Hugin may ask you to locate PTStitcher. If so point it to the copy you put in the Hugin folder.

    

11. Final blending & cropping in photoshop — Upon opening the psd file in photoshop you’ll notice that the blend lines are pretty conspicuous. However, as each shot is a separate (masked) layer you can tweak each blend independently by editing the mask.

    

    At this point you also have the opportunity to mask in/out moving foreground subjects and make other creative edits (i.e. duplicate people). If you took my advice and shot with 2/3 overlap you’ll have a spare layer to run wild with too. A few blending tips:

    • Use broad diameter ’soft’ brushes to feather the masks in areas of sky.
    • Avoid straight lines in your masks (especially horizontal & vertical lines). Human eyes will spot those straight away.
    • In areas of high detail use small diameter ‘harder’ edged brushes (this avoid ghosting in the final image).
    • Hard edged subjects in the foreground make great (hard) mask boundaries

    After tweaking each of the seams the whole panorama becomes pretty convincing. This is the most rewarding part of the process, and you run the risk of wasting a lot of time ;). In this example I decided to merge foreground characters from two different shots, but I left some easter eggs in there for anyone who’s really looking.

    

12. The final cut — To get rid of the ‘rising horizon’ problem in this image set (a constant issue when shooting panoramas sans tripod) I’ve selectively cloned and distorted the middle foreground grass and left sky. The final image captures all the things I remember from that instant in one single image (the ultimate aide mémoire).

    

The length of this post might make the process of manually assembling composite/panoramic images appear complex, but trust me, the second time’s a snap. Oh, and you’ll be the only kid on your street who’s panoramas don’t suck.

The lens review widget is a flash component embedded into our lens reviews responsible for presenting the results of our studio tests. The challenge of the project is that the lens review studio test output is intricate, multi-dimensional and in acute danger of misinterpretation.

The widget’s goal is to provide users with a constructive mental model of this dataset and an intuitive/responsive/encouraging interface with which to navigate it.

The broad data categories initially displayed are:

• Sharpness is the top of the list for most lens geeks. Not all lenses are created equal, especially considering the difference in the ’sharpness’ of the image they produce. A single lens can vary greatly between different focal length/aperture combinations and even varies (non-linearly) from centre to corner within a single shot!
• Chromatic aberration is the colour ‘fringing’ you see around light-on-dark or dark-on-light objects in your photographs. Generally purple or green (but occasionally blue/yellow) this effect varies with focal length/aperture and distance from centre.
• Distortion is what causes real world straight lines to appear curved in your photographs. Zoom lenses in particular are prone to distorting ‘inwards’ (pincushion) at one extreme and ‘outwards’ (barrel) at the other. Distortion varies by focal length.
• Falloff is the phenomenon whereby the corners of the frame appear ‘darker’ than the centre (i.e. the light ‘falls off’ from centre to edge). Though often used as an intentional effect, falloff is generally avoided if possible and varies with focal length, aperture and obviously, distance from centre.

In practice these 4 effects are all present (to some extent) in every shot you take, so representing their individual effects is more complicated than simply displaying the raw images (which we also do).

Designing the user interface

With so much data to navigate and several dimensions to deal with (angle from centre, distance from centre, focal length and aperture) the process of designing a useful and intuitive UI was difficult. Eventually a standard hierarchy emerged:

1. Lens review. It may seem the obvious choice, but I’ve included it on this list as (if allowed in configuration) the user is able to change the review data being viewed within the widget.
2. Visualization. Sharpness & Chromatic Aberration, Distortion or Fall-off.
3. Focal Length. Or ‘zoom’ to non-photographers, is generally marked on lenses in arbitrary integer values (generally 4 - 7 per lens).
4. Aperture. The variable size of the ‘hole’ which regulates the amount of light entering the camera body. Stated in f numbers, an approximately logarithmic semi-standardized decimal value. The aperture values available for a lens depend on the focal length chosen.
5. Comparison lens review. Users reading reviews are constantly making comparative decisions, so the widget allows users to compare two lens reviews (or two data-points within the same review) cheek-by-jowl, hopefully improving comprehension. Of course, this requires user interface chrome to allow the user to choose.

There was much internal debate (which still rages) about this UI element which allows users to change/compare reviews, but the eventual decision was to filter reviews by system, lens and finally camera body. Note: Our lens tests are performed on a camera body (instead of an optical test bench) which has relevance upon the results.

The ’slider’

The real challenge from a UI perspective was creating a UI element which allowed users to change focal length / aperture in an intuitive way, was brand-neutral and could cope with the intricacies of the focal length /aperture systems and their inter-relationship. After initially experimenting with flash’s slider component, I decided to abandon it and start from scratch (before you shout Not Invented Here, be aware that flash’s slider is a major usability dud).

Trim

Some neat little features that I pushed into the widget (usually late at night):

• Full screen mode: At any time a user can click the full screen button at the bottom of the widget to view in a new window/tab (selected reviews, visualization, focal length & aperture are retained).
• Permalinking: At any point, users can obtain a permalink to the current state of the widget to bookmark and/or link to (again the selected reviews, visualization, focal length & aperture are retained).
• Keyboard controls: A bit of a power-user feature requested by the guys in the office, the keyboard arrow keys can be used to control focal length and aperture. If multiple reviews are visible, they will all receive the inputs, allowing users to change the visualizations in unison (not currently possible via the sliders).

Though a few minor issues persist, I’m very happy with the current implementation. I feel it strikes the right balance between aesthetics, usability, accuracy, brand-neutrality and ‘fun’.

Time (and the forums) will ultimately tell.

1. Take photo, download images (using canon software) to mac and wipe CF card.
2. Import images into iPhoto 6 then:
   • Battle against iPhoto’s terrible data-entry interface (helped slightly by Ken Ferry’s great Keyword assistant for iPhoto).
   • Complain bitterly when I realized that iPhoto does not write EXIF/XMP, instead it keeps metadata in own proprietary database.
   • Lament iPhoto’s ignorance of Geo-tagging (storing latitude and longitude in photos to allow them to be represented on maps).
3. Avoid backing up iPhoto library (not so straight-forward) whilst panicking that i’d lose my photos to a disc failure.
4. Ignore requests to email photos due to effort required (we all do it!) or resize and email if feeling keen (rare).
5. Occassionally upload tiny subset to facebook (and receive angry-mails from non-facebookers who can’t see the images).

The New way

• Signed up for flickr pro account. $25/year for unlimited storage/bandwidth etc.
• Downloaded Lightroom (metadata editor, neat tone curve editor).
• Downloaded PictureSync. It turns iPhoto data into EXIF/IPTC/XMP/Spotlight tags, pulls lat/long from google earth (optional) and uploads (full-size) photos flickr. Also plays nice with lightroom. Great rules engine (for fixing metadata).



• Added My Flickr app to my facebook profile, canned default photo app.

The new status quo

1. Images metadata now embedded in images (where it ought to be) instead of in an app’s database.
2. Photos backed up (hurrah) in the cloud (even better) at full resolution/quality with powerful privacy control.
3. Can view/search/share entire photo library over web (including all metadata).
4. Friends/family can download images (configurable) in a variety of sizes without any additional effort on my part.



5. Ready-made RSS (and GeoRSS) feeds of my images thanks to flickr (friends and family can subscribe).
6. My Geo-tagged photos can be viewed on ready-made map UI. Cool!
7. Significantly more web 2.0 street-cred (easy on the 2.0 cynicism people!)

So next time someone asks me to send them a copy of a photo, there’s now a more-than-slim chance I may actually do so.

Next on my list is getting my G7 and my new GPS working together to automatically geocode my images.

Supplementing the Australian portion of the Garmin USA base map

The Australian portion of the inbuilt US base-map appears to have been drawn on an etch-a-sketch by a blind monkey. This is a problem, especially as Garmin’s range of Australian map-packs are expensive and useless to anyone but soccer-mums ‘navigating’ to the local shopping centre.

Luckily the free Shonkymaps Australian map-set for Garmin contains full topographic 1:250,000 maps for the whole of Australia. Downloading these maps into your GPS is reasonably straightforward:

1. Ensure you have Garmin MapSource for windows (mac version coming soon apparently).
2. Download Shonkymaps direct or legally via bittorrent (you’ll save them 340mb of bandwidth).
3. Install Shonkymaps (needs to write to registry to comply with Garmin’s cockamamy MapSource system).
4. Shonkymaps can now be downloaded to your Garmin handset via MapSource just like any off-the-shelf Garmin map-set.

So how do Shonkymaps shape up? Well, see below for a comparison of their respective coverage of Moreton Island.

Google satellite image (left), Garmin base map (middle) & ‘Shonkymaps Full Topo’ (right)

Having a crack at geocaching

Wandering through the 60csx’s menu system (a habit with all new gizmos) I discovered a few features relating to geocaching (a never-ending decentralized global treasure-hunt game). As I’ll have some time on my hands over the break, I thought I’d give it a go. It turns out that setting up a Garmin GPS for geocaching is remarkably easy.

1. Install the Garmin Communicator browser plug-in (supports firefox, woot!).
2. Create a free account at geocaching.com, an online community which lists, manages and discusses everything related to geocaching.
3. The geocaching.com site can be a little hairy at times, but if you head straight to their geocache map search page you’ll find a dead-simple UI for locating geocaches in your area.
4. Once you’ve found a geocache you want to add, click it’s icon and a Google map balloon will appear containing the relevant details, including a ‘Send to Garmin link.

   

5. Click the ‘Send to Garmin’ link, ensure your GPS is connected to your PC and click ’send’ on subsequent page.
6. Done! Here’s what the geocache waypoint info screen looks like. All the hard work’s done for us (except finding the actual cache)

In 60 seconds I had  half a dozen geocaches loaded on my GPS and due to the fact I wasn’t involved in typing the lat/long, there’s a fair chance they’ll be reliable. Now my only excuse for not finding them will be a lack of navigation ability ;)

I’ve said goodbye to the consulting world and taken a position working with the gang at dpreview.com as a full-time public-facing web-developer. It’s a huge, popular and well-established site (at 9 years old it can be considered the gray lady of digital photography) recently acquired by amazon. I now find myself working in central London (frequently cursing the tube) with a young and enthusiastic team covering an equally dynamic industry.

Note: I’m retiring my rule regarding blogging about my day-job, mainly because my role has changed (more public development) and I’m keen to get more involved in the development community.

With so much going on in digital photography and the web-savvy nature of our readership, there’s plenty of exciting stuff we can (and will) do. Some of the smaller stuff I’ve been working on has already started to appear on the site, more details to follow.

an IM icon created for a friend (whose name is a homonym for che’s)

In the process I decided to adorn my IM profiles with the mighty ‘Pico’ icon, something that had slipped my mind for years.

 
My new IM icon (based on my Pico character)

The original artwork dates is from the flash-based pico game I made for a uni assignment at Hong Kong University in 2002. As I spent all my free (and a fair chunk of my uni) time exploring Hong Kong with my friends, the assignment was developed in a marathon 48 hour cram session in the HKU multimedia labs. 

The game wasn’t always called Pico 

The game itself is a pretty basic affair, but still makes me laugh when I see the artwork and animations. As a bonus, it’s fiendishly difficult (being alone in a CS lab at 4am does tend to warp a coder’s mind). Give it a go, it’s a great cure for apathy.

 
Hong Kong Island (including HKU) is an festival of stairs and elevators

The whole 2-day stint was surreal; the deadlines, the long walk along pok fu lam road, the long hours, the great music, the septuagenarian tai-chi club which trained (complete with weapons) right outside the labs at dawn (their existence was unknown to me previously and seems unreal still).

 
Pok Fu Lam road, connecting HKU campus to the halls

My previous attempts had all involved reducing each image to a weighted list of ‘popular’ colours. The search UI let the user pick from the web (216) palette, with only images possessing a healthy portion of that colour being returned in the results. At one time I even allowed users to select a secondary colour along with RGB/HSV/CMYK (synchronised) sliders, but that was as complicated to use as it sounds, so I let it go. The problem was that this ‘popular colours’ approach ignored the fundamental truth that people just don’t search for drab browns, faded yellowy-greens or most of the other colours which make up these palettes. From a colour-space perspective, when people say ‘colour’, they mean ‘hue‘.

So earlier this week (on the train) I set about re-writing the whole colour search feature to actually be a hue-based search feature (the word colour remains because ‘hue search’ just doesn’t light up anyone’s eyes, not even mine). After a few train trips and most of Saturday I had the hue-parsing algorithm implemented and had written a console app which re-indexed all the existing images in the library. I also updated the relevant portions of the batch uploading tool and the admin web site to be hue-aware. Things were going well, but I had only a vague idea of how the actual search UI would work.

Initially I thought another slider would be the best approach, but decided it was overkill after looking around at existing html slider implementations (yahoo UI library as a nice slider, but I didn’t want the added dependency for what is essentially a minor feature). Eventually I set about rolling my own control for the job. I think the hue picker widget and the hue search feature as a whole turned out really well, and will continue to improve as the number of images in the library grows.

Now I’m thinking of packaging up the  Hue Picker for free distribution. You know, just for kicks.

Hi-lights include

1. .Net 1.1 to .Net 2.0 upgrade (long live the MasterPage)
2. Amazon S3 for all image storage (dramatically reduced bandwidth and hosting costs)
3. Google map-based location search.
4. Creation of “Pusher”, a fire-and-forget image watermarking, batch uploading and data-entry smart client app (to create the 12+ thumbnail and preview versions of each image).
5. Creation of “S3 Viewer”, a smart client app for viewing/maintaining the thousands of S3 items and buckets which support the site.
6. ‘New Images’ atom feed (surprisingly easy to get going).
7. SQL Server 2000 to SQL Server 2005 upgrade (Common Table Expressions are the goods).
8. GeoRSS feed to (hopefully) ensure photographs appear in search results as geo-tagged objects.
9. Tag-clouds for hierarchical image categories.
10. Colour search (still beta, currently rewriting the search algorithm to favour colour coverage rather than colour fidelity).
11. Cleaner UI design.
12. Xhtml transitional doc type with vastly improved markup validity (flash still causing some issues here). Some tables remain, to be slowly phased out when I have time.
13. Flash 9 homepage scroller (flash cs3 didn’t play nice with my old flash 6 swfs).
14. Self installed and configured SSL.
15. New hosting provider (discountasp.net, very impressed so far)

All told the upgrade took about 3 months’ worth of train trips, nights and weekends (around my day job). It’s meant an overall reduction in sleep, but ultimately worth it. Now I have just to force myself to stop tinkering with it.