The Right Tool

My Father has a garage full of tools and years of experience using them for good. I grew up spending many hours in his workshop; some of them voluntary, some of then conscripted. Having moved too far away for quick tool loans, I frequently find myself without the drill, level, saw, or other various tools that I need for some project. Over the last few years I've gradually built up my own small collection of tools.

I appreciate the value of quality tools and my cash supply is limited, and so I must make my purchases carefully. More then a few times in the past few years I've called my Father and said: "I need some bar clamps to fix a drawer. If I'm going to buy a single pair, which size should I get?" This question has been asked about laser levels, bar clamps, crescent wrenches, and other's I've forgotten. His advice has never failed me in choices of brands, sizes, and knowing when getting the cheaper option isn't going to eat me in the long run.

I do appreciate my Father for things other then his tool advice, but in this post, I highlight his excellent guidance in getting the right tool for the job.

I took a moment with my new bar clamps to practice my photography skills. Jess let me borrow the camera for a few minutes, and also gave me feedback on my editing.

Extracting Pictures from Flickr with Python

A few years ago, when Flickr was new, we made kind of a silly decision to only store our images on Flickr. While this did make a transition between computers easier and freed up a little bit of drive space, we decided that we'd like to pull those pictures back onto our own system.

In the past, I've tried several Flickr Downloadr (missing 'e' intended as a pun) programs, and everything choked or did strange things. Last night, right after I crawled into bed, I decided that I knew a better way.

I crawled out of bed, did a bit of Googling, and found an excellent Python Flickr library, and someone that wrote a python script to backup Flickr pictures to Amazon's S3. In 15 minutes or so, I had a solution that would page through our public pictures, check to see if they were already downloaded, and store them in year and month folders.

import flickr
import urllib
import os.path
import os
page = 1
total_photos = found_photos = 0
while True:
   photos = flickr.people_getPublicPhotos('68432331@N00', 100, page)
   if not len(photos):
      break
   for photo in photos:
      total_photos = total_photos + 1
      photoYear = photo.datetaken[0:4]
      photoMonth = photo.datetaken[5:7]
      photoURL = photo.getURL('Original', 'source')
      photoPath = "C:\FlickrPics\%s\%s\%s.jpg" % (photoYear, photoMonth, photo.id)
      if not os.path.exists(photoPath):
         if not os.path.exists("c:\FlickrPics\%s" % photoYear):
            os.mkdir("c:\FlickrPics\%s" % photoYear)
         if not os.path.exists("c:\FlickrPics\%s\%s" % (photoYear, photoMonth)):
            s.mkdir("c:\FlickrPics\%s\%s&" % (photoYear, photoMonth))
         urllib.urlretrieve(photoURL, photoPath)
         found_photos += 1
      page = page + 1
      print "  Moving to page %s" % page
print "Found %s photos, saved %s new photos" % (total_photos, found_photos)</p>

While running the script, I noticed that the obvious slow part of the process was downloading the images. I wanted a way to download them in parallel, and found my solution in an excellent Python thread pool solution.

I added the following method to their code, that will tell me the number of jobs pending in the thread pool:

#new method in the ThreadPool class
def getWaitingTaskCount(self):
   self.__taskLock.acquire()
   count = len(self.__tasks)
   self.__taskLock.release()
   return count

I had to rewrite my code a little to put the download code into a method, and then I let 'er rip. Downloading now happened 3 at a time. It took about 2 hours to download our 5GB collection of approx 3,300 pictures, and the job was done!

I don't bother with authenticated requests (all of our pictures are public), and I don't do any error checking. It is a one off script, after all. :)

I love python. I really do.

import flickr
import urllib
import os.path
import os
import threading
from time import sleep

#code from ThreadPool not shown.
#Copied from linked solution, with addition new method listed above

pool = ThreadPool(3)

def getPicture(data):
   photoURL = data[0]
   photoPath = data[1]
   urllib.urlretrieve(photoURL, photoPath)
   print photoPath

page = 1
total_photos = found_photos = 0

while True:
   photos = flickr.people_getPublicPhotos("68432331@N00", 100, page)
   if not len(photos):
      break
   for photo in photos:
      total_photos = total_photos + 1
      photoYear = photo.datetaken[0:4]
      photoMonth = photo.datetaken[5:7]
      photoURL = photo.getURL('Original', 'source')
      photoPath = "C:\FlickrPics\%s\%s\%s.jpg" % (photoYear, photoMonth, photo.id)
      if not os.path.exists(photoPath):
         if not os.path.exists("c:\FlickrPics\%s" % photoYear):
            os.mkdir("c:\FlickrPics\%s" % photoYear)
         if not os.path.exists("c:\FlickrPics\%s\%s" % (photoYear, photoMonth)):
            s.mkdir("c:\FlickrPics\%s\%s&" % (photoYear, photoMonth))
         # Insert tasks into the queue and let them run
         pool.queueTask(getPicture, (photoURL, photoPath))

         found_photos += 1
   page = page + 1
   #don't get too far ahead of the download threads
   while pool.getWaitingTaskCount() > 10:
      sleep(1)
   print " Moving to page %s" % page
# When all tasks are finished, allow the threads to terminate
pool.joinAll()
print "Found %s photos, saved %s new photos" % (total_photos, found_photos)

What Online Banking Needs

For years, banks forced customers to bend their money habits into the forms and methods that were available at the bank. Customers could choose from a limited selection of accounts, and were forced to learn the bank's rules for managing their money. New technology is providing a way to make banking customer centric. Now we just need banks to catch the drift.

I'm a big del.icio.us user, and I love keeping track of my bookmarks with tags instead of folders. What I would really like from banks is a way to tag my money. When my wife and I save for things, we want to keep track our money so that we can measure progress. Many money market accounts have a $1,000 minimum to get the base interest rate, and we've discovered that it is common to limit the number of accounts that a single customer can have. We create multiple accounts so that we can keep track of money.

So here is what tagging money looks like: In their online interface, I can tag a transaction. I can run reports based on tags. When interest is deposited into the account, allocate a portion of the interest payment to each tag. This can all happen in one transaction (from the bank's point of view), but will just carry a little annotation along with it. I can create a tag for something I'm saving for (a car, for example), and tag deposits with 'car'. Each month, the car tag grows a little from interest, and I can easily view the money that I currently have saved.

If my bank offered this, I wouldn't need so many accounts!

CompactFlash seems to be giving way to SD/SDHC

Canon just announced the XSi, their upgrade to the XTi. Notably, they dropped Compact Flash and replaced it with an SD/SDHC slot. I'm not glued to CF, but it seems to have been the storage format of choice for higher end cameras for quite some time.

My favorite flash media format is in fact SD, and extending the cards to handle large cards with SDHC only makes sense. The addressing scheme on SD cards tops out at 2GB, which is becoming far to small. SDHC's addressing scheme has a theoretical maximum of 2048GB (2TB!), which should be enough for a little while. I'm sure they have some work to cram that much storage into the SD form factor, but I'm glad to see them planning for the future.

When hard drive based video cameras started showing up on the market, I was pretty excited. Having a random access way to record and work with video only makes sense. I was convinced that they were the way to go. Now, I'm more inclined to look forward to an SDHC based video camera.

Canon XTi vs Canon 5D

Two years ago we purchased a Canon Rebel XT, and have been enjoying it ever since. It has introduced my wife Jess to photography in a way she never had been before, and she has become quite good at it. Our initial purchase eventually led to buying studio backdrops and strobe lighting equipment, wireless triggers and other fun toys.

We were taking studio portrait pictures a few weeks ago for our church Christmas party, and the shutter started developing problems. The shutter was slow (or late) on some pictures, resulting in pictures with the bottom portion blocked by the shutter, or completely black pictures. Also mixed into the fun was the infamous Err-99, which required a camera restart.

Frustrating. The XT's shutter is rated for 30k pictures, and we've only taken ~17k. We made it through the night, but it is clear that our XTs days are numbered. We can fix the shutter for something in the neighborhood of $200, but it requires our camera to be sent into the shop for a couple weeks, and that isn't a good plan right before Christmas and the New Year!

We have a few options within our price range. The cheaper option is the XTi, the XT's current incarnation. It's been upgraded to a 10.1 MP sensor and has a few other improvements, but is otherwise the same XT that we've come to love.

The more expensive option is the 5D, the cheapest Canon with a full-frame sensor. It has a 12.8 MP sensor, and it will remove the slightly annoying 1.6 focal length conversation ratio as a result of using a full-frame sensor. It does have the size and heft of a full size camera, which is quite a but more then the smaller XT.

An XTi body is ~ $500, and the 5D body is  ~ $2000. The 5D is a better camera, but is it $1500 better? We've decided not. The problem with Canon's lineup isn't that the 5D isn't good enough, it is that the XTi has become pretty good. The 5D hasn't had an upgrade in a few years, but perhaps Canon will announce such an upgrade soon.

One of the advantages of the 5D is the fact that a 28mm lens actually acts like one, instead of the 45mm it becomes on the XT or XTi. This is a problem for indoor shooting, as you often can't backup enough to get a good picture. We decided to buy the XTi with the 18-55mm kit lens, which will turn into 29-88mm as a result of the lens conversion factor. Since that gives us most of the desired benefit of the 5D, the only difference would be a 2 MP increase in picture resolution.

We will likely fix the XT as well, and keep it as a backup body, but it will be nice to have a working shutter during the holidays!

Eye-Fi Usefulness

Engadget posted a review of the Eye-Fi Secure Digital Wifi card, and summarized it with the following sentence:

Sounds like a great idea in theory, it turns out that the Eye-Fi is, in its current release, an answer seeking a question.

This opinion is based off the fact that you can't use it on any wifi hotspot. The card will only connect to computers that have been pre-configured with software that will enable it to receive the pictures, and only on the same access point as the camera.

What they fail to miss is how useful this card is to anyone who shoots events or works in a studio, and prefers to work without a cord tying them to a machine. By setting the system up to auto-download the images to a computer (even if they are not uploaded to an online service), you can enable some pretty nice workflow options. Adapters for most cameras to enable them to transfer pictures to a computer as they are taken are usually very pricey. Allowing this with any SD capable camera just by purchasing the card is awesome.

I do agree that having the card work on any public wifi hotspot would be cool. I just don't think every product must be usable by any random person to be useful.

Adobe's Research into 3D Photography

I just watched an amazing video demonstrating the power of 3D photography. This seems like one of those mind bending things of today that our kids won't understand not having.

[Found via CNet News]

Canon EF 50mm f/1.8 II Camera Lens

We just purchased a 5mm f/1.8 lens from Amazon, and we love it. My previous lens experience gets cut short at a aperture of 3.5, and I really did not know what I was missing. Performance is great for indoor shots in low light. With one light on in the room, I was able to take a hand held shot with an ISO of 800 and a fast enough shutter to get a sharp picture.

Our RebelXT has a smaller then full frame sensor, which creates a lens focal length multiplier of 1.6. This is a bit inconvenient, as our new 50mm lens performs like an 80mm lens. Buying a wide angle lens with a decent aperture turns out to be quite expensive, so we don't have much.

Someday, we will replace our camera and get a full frame sensor, but in the meantime, we'll have plenty of fun.