hacker / welder / mechanic / carpenter / photographer / musician / writer / teacher / student

Musings of an Earth-bound carbon-based life form.

5 min read

A personal photo gallery V1: S3, Lambda and Rekognition

Published June 16, 2017 #aws #cloud #photography #tech

With the recent addition of new high-level services in AWS’ arsenal I decided that it was time to build a proper photo gallery application. Why not use one of the many other open source gallery applications? Well, for one I don’t have any desire to manage infrastructure at home, and for another I would like to build something that lets me backup my large number of photos at home (closing in on 200GB of photos over the last 11 years) somewhere that they would be safe.

I have a number of requirements for this tool, and I will walk through how it’s built so you can do something similarly interesting!

Requirements:

  • Safe storage that is cost efficient (see pricing calculations)
  • Permit sharing of photos with others (public or privately)
  • Low operational complexity (i.e stack could be deployed by anyone with some tech skill using CloudFormation)
  • Be able to categorize and identify entities in photos
  • Searchable (preferably using English search queries such as “find photos of Natalie from 2007”)
  • Fun!
  • Ability to extend in the future (“this day in…”, “you haven’t seen this image in a while”, etc.)

Technologies

  • S3 - Image storage and image metadata
  • Lambda - image processing driven by S3 events
  • Rekognition - identify features and people in photos, store data in S3
  • DynamoDB - Store the data

Pricing calculations (Compare 1TB of S3 storage with a hard drive, MTBF calculations, ECC RAM, power, server, bit-rot, off-site backups)

Step 1 - Image uploading

The first thing I needed is a place to store the photographs - S3 is a great candidate for this, the cost is low (~$.02/GB per month as of this writing), it has a variety of storage options with reduced cost for entities that are not often retrieved. This means that the full-size, high quality, images can automatically be stored at a reduced price while the smaller, more frequently accessed, thumbnail images would be kept in S3’s active storage tier (though these are much smaller, on the order of a few hundred KB per RAW photo compared to the 3-30MB for each RAW image), saving money.

Initially image uploading will be done using s3cmd or another similar tool that syncs my local data into S3 from my on-site NAS (but it would be trivial to write a tool that monitors local storage / integrates with common photo platforms in the future).

A note on S3 events

I’m a huge fan of the KISS philosophy; the simpler your system, the easier it is to understand and it also requires less work! S3 has the ability to perform a variety of actions when certain events happen (in our case image uploads) including calling out to Lambda.

IMAGE: Local Disk –> S3 -> S3 event -> Lambda -> Process Image

Events can be limited to keys matching various criteria including the extension or key prefix; for our bucket we will be storing our original images with the key prefix (or folder) of originals and the processed thumbnails using thumbnails. This is so that an image such as 2006/10/04/IMG_00001.NEF will be stored as originals/2006/10/04/IMG_00001.NEF and the corresponding thumbnail will be thumbnails/2006/10/04/IMG_00001.NEF.JPG. If we did not do this then we would need two buckets to avoid processing the original and uploading the thumbnail, then processing the thumbnail and uploading its thumbnail, upon which point we would … (see also: recursion)

Code: CloudFormation for S3 bucket

Step 2 - Image processing

S3 events that are emitted to Lambda are sent as JSON that contains one or more record about objects in S3; each record contains a number of data points in them but for our purposes we are only interested in the bucket name and the key of the photo that was uploaded into S3. The reason for this is that the process will be:

  1. For each object in the event, fetch the S3 bucket and key for that object
  2. Consider the image format
  3. If the image is already a lossy format (JPEG, PNG, etc.) then we just resize the image
  4. If the image is a RAW file then pull down our dcraw binary from S3 and extract the JPEG thumbnail
  5. Store the image back in S3 under the thumbnails prefix
  6. Pass the thumbnail’s S3 URI to Rekognition in order to extract interesting features, faces, etc.
  7. Store the labels back in S3 for later searching / processing / querying

Example S3 event JSON

  {
    'Records': [
      {
        's3': {
          'bucket': {
            "name": "photo-bucket",
            "arn": "..."
          },
          "object": {
            "key": "originals/2006/10/04/IMG_00001.NEF"
          }
        }
      }
    ]
  }

Some Python code to process the image


DCRAW_S3_URL = "https://s3.amazonaws.com/shoeboxapp-data/support/dcraw-static"
DCRAW_BINARY_PATH = ""
def handle_lambda(json_input, context):
    rekognition = boto3.client('rekognition')
    s3 = boto3.resource('s3')

    records = json_input['Records']
    thumbnail_output_path = "/tmp/thumbnail.jpg"
    original_image_path = "/tmp/rawfile"

    for record in records:
        s3_event_data = record['s3']
        key = s3_event_data['object']['key']
        bucket = s3_event_data['bucket']

        try:
            print("Downloading %s from %s to %s" % (bucket['name'], key, raw_file))
            s3.Bucket(bucket['name']).download_file(key, original_image_path)
        except botocore.exceptions.ClientError as e:
            if e.response['Error']['Code'] == "404":
                print("The object does not exist.")
                raise

        urllib.request.urlretrieve(DCRAW_S3_URL, dcraw_binary)

        print("Contents of %s" % (rootdir))
        call(["chmod", "0777", dcraw_binary])

        thumbnail = open(thumbnail_file, "w")
        p = Popen([dcraw_binary, "-e", "-c", raw_file], stdout=thumbnail, stderr=PIPE)
        rc = p.returncode
        print("Retval: %s" % (rc))

        print("Contents of /tmp")
        p = Popen(['ls', '-al', "/tmp"], stdout=PIPE, stderr=PIPE)
        output, err = p.communicate()
        print(output)

        thumbnail_key = ("%s.jpg" % (key)).replace("originals", "thumbnails", 1)
        print ("Uploading %s as %s in %s" % (thumbnail_file, thumbnail_key, bucket['name']))
        s3.meta.client.upload_file(thumbnail_file, bucket['name'], thumbnail_key)

        print("Running rekognition...")

        response = rekognition.detect_labels(
            Image={
                'S3Object': {
                    'Bucket': bucket['name'],
                    'Name': thumbnail_key
                    }
            },
            MaxLabels=20
        )

        print("Rekognition response: %s" % (response))

        labels = response['Labels']

        for label in labels:
            print("%s @ %s" % (label['Name'], label['Confidence']))