Image Resizing with Python and ImageMagick (bleh)

Python    2009-12-11

Let me first give a little background on this project, and then I'll break down the script.

I'm working with a legacy image handling system that's more than a little convoluted. Source images for different music artists are uploaded through a multi-tenant CMS to a media server, to a DAV-enabled folder that kicks off a series of scripts and subversion commands. The source images are first committed to one repository, then resized according to specifications in each artist's config file. Then the resized images are committed to a separate folder for publication. That second commit triggers an rsync that sends the new images out to the production image server where they're ultimately served from.

Our goal is to replace the whole process with something simpler. Heh. But for now, we're replacing one piece at a time.

The scripts we're currently using are either shell scripts or written in PHP (legacy system, go figure). We're replacing everything with Python to integrate more cleanly with subversion's hooks.

This piece is the script that grabs the source images, parses the config files, resizes, then does the second commit.

I'm writing to a temp log for each source image that gets processed, embedding the contents of that log in an email that's then sent to an admin list, then destroying the log at the end of the run. The first few methods are about running checks - on arguments, the existence and validity of the source image, etc. The next few create paths and handle the resizing. One thing to note: it was mandated early on that I use ImageMagick instead of PIL. Not my choice, but the powers-that-be insisted (the reasoning being that ImageMagick is already installed on all of our media servers - I made my case for PIL being easy to install, but alas).

#!/usr/bin/python

# usage: ./imageprocess.py source_imagepath revision_number state
# example: ./imageprocess.py admin/uploads/artistname/source/non_secure/images/20091026/mytestimage.jpg 17 A

import os, sys, smtplib, MySQLdb as Database, datetime, tempfile, subprocess
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart

log = "/tmp/artist.processing.log"
now = datetime.datetime.today()
vars = {}

def logThis(text):
    print str(now.strftime("%Y-%m-%d %H:%M")) + " IMAGE PROCESSING: " + text
    logfile = open(log, "a")
    logfile.write(str(now.strftime("%Y-%m-%d %H:%M")) + " IMAGE PROCESSING: " + text + "\n")
    logfile.close()

def sendMail():
    emailto = ['barbara.shaurette@mycompany.com',]
    fromaddr = "barbara.shaurette@mycompany.com"

    COMMASPACE = ', '
    msg = MIMEMultipart()
    msg['Subject'] = "Image processing log"
    msg['From'] = fromaddr
    msg['To'] = COMMASPACE.join(emailto)

    fp = open(log, 'rb')
    a = MIMEText(fp.read())
    fp.close()
    msg.attach(a)

    s = smtplib.SMTP('localhost')
    s.sendmail(fromaddr, emailto, msg.as_string())
    s.quit()

    logThis("CONFIRMATION MAIL SENT")

    # once the confirmation email is sent, blow away the log to make way for the next one
    os.remove(log)

def checkParams():
    """ if there are not at least three arguments, log the error and exit """
    if not sys.argv[3]:
        logThis("INSUFFICIENT ARGS PROVIDED, EXITING")
        sys.exit()
    else:
	vars['img_path'] = sys.argv[1]
	vars['revision'] = sys.argv[2]
	vars['state'] = sys.argv[3]

        vars['img_name'] = vars['img_path'].rpartition("/")[2]
        vars['artist_docroot'] = vars['img_path'].replace("admin/uploads/", "").partition("/")[0]
        vars['admin_uploads_dir'] = 'admin/uploads/'
	vars['root_svn'] = "file:///srv/svn/webmedia/"
	
        vars['source_image_path'] = vars['root_svn'] + vars['img_path'].replace(vars['img_name'], "")
	vars['source_image_fullpath'] = vars['root_svn'] + vars['img_path']
        vars['publish_image_path'] = vars['source_image_path'].replace("source", "pub")
        vars['publish_image_fullpath'] = vars['source_image_fullpath'].replace("source", "pub")

	logThis("ALL ARGS PROVIDED: source image: " + vars['source_image_fullpath'] + ", revision number: " + vars['revision'] + ", state: " + vars['state'])
	return vars

def checkState():
    """ bypass this script if the image is being deleted """
    if vars['state'] == 'D':
	logThis("IMAGE IS SCHEDULED FOR DELETION, EXITING")
	sys.exit()

def checkExists():
    """ if for some reason the image doesn't actually exist, log an error and exit """

    # I started out using subprocess.Popen (you can use communicate() to return a tuple from the stdout data)
    # but I really only need a boolean returned here - the source image is either there or it's not - so I switched to check_call
    list = subprocess.check_call("svn", "ls", vars['source_image_fullpath']])
    results = str(list)

    if results:
	logThis("SOURCE IMAGE FOUND: " + vars['source_image_fullpath'])
    else:
	logThis("SOURCE IMAGE NOT FOUND AT " + vars['source_image_fullpath'] + ", EXITING")
	sys.exit()

def checkImageExtension():
    """ check for valid image types """
    img_parts = vars['img_path'].rsplit('.')
    extension_list = ['jpeg', 'jpg', 'gif', 'png']
    if img_parts[1] in extension_list:
	logThis(img_parts[1] + " IS A VALID IMAGE EXTENSION")
	vars['img_extension'] = img_parts[1]
    else:
	logThis(img_parts[1] + " IS NOT A VALID IMAGE EXTENSION, EXITING")
	sys.exit()
    return vars

def makeTempDir():
    """ create a temp directory for image processing """
    tmpdir = tempfile.mkdtemp()
    if tmpdir:
        vars['tmp_dir'] = tmpdir
        vars['tmp_img'] = vars['tmp_dir'] + '/' + vars['img_name']
        logThis("created dir: " + tmpdir)
    else:
	logThis("COULD NOT CREATE THE TMP DIRECTORY " + tmpdir + ", EXITING")
	sys.exit()
    return vars

def makeSizes():
    """ locate the appropriate ini per artist, per upload type, make sure it's where it's expected to be """

    todaysdate = now.strftime("%Y%m%d")

    vars['ini_file'] = "image_upload.ini"
    if vars['publish_image_path'].find(todaysdate + "/news") == 1:
        vars['ini_file'] = "news_upload.ini"
    if vars['publish_image_path'].find(todaysdate + "/discography") == 1:
	vars['ini_file'] = "discography_upload.ini"
    if vars['publish_image_path'].find(todaysdate + "/user/images/letters") == 1:
	vars['ini_file'] = "letters_upload.ini"

    logThis("using " + vars['ini_file'])

    vars['ini_path'] = vars['source_image_path'].rsplit("/", 4)[0] + "/" + vars['ini_file']

    list = subprocess.check_call(["svn", "ls", vars['ini_path']])
    results = str(list)

    if results:
	logThis("CONFIGURATION INI FOUND AT " + vars['ini_path'])

	# read from the ini and get the list of image sizes
	f1 = open(vars['ini_path'], "r")
	text = f1.readlines()
	f1.close()
	widths = []
	heights = []
	for line in text:
	    if line.find(";") < 0:  # ignore the commented lines
		if line.find("width") > -1:
		    widths.append(line.split()[2].replace('"', ''))
                if line.find("height") > -1:
		    heights.append(line.split()[2].replace('"', ''))

	vars['sizes'] = {'size1': {'height': heights[0], 'width': widths[0]}, 'size2': {'height': heights[1], 'width': widths[1]}, 'size3': {'height': heights[2], 'width': widths[2]}}
	logThis("SIZES " + str(vars['sizes']))
	return vars
    else:
	logThis(vars['ini_path'] + " NOT FOUND, EXITING")
	sys.exit()

def makePaths():
    """ make and commit new paths for the final published images """
    path = str(vars['publish_image_path'])
    if not os.path.exists(path):
	logThis("NOT A DIR, CREATING: " + path)
	success = os.mkdir(path, 0777)
	if success:
	    logThis("ADD AND COMMIT " + path)
	    addpath = os.popen("svn add " + path)
	    addpath.close
	    commitpath = os.popen("svn ci " + path + "-m 'adding folder: " + path + "'")
	    commitpath.close
	else:
	    logThis("FAILED TO CREATE " + path)
	    sys.exit()
    else:
	logThis("PATH EXISTS: " + path)
    return vars

def resizeImages():
    """ NEW METHOD: RESIZE THE IMAGES """

    # Copy the image out of the repository to the tmp_dir so we have a local working copy
    export = subprocess.check_call('svn export ' + vars['source_image_fullpath'] + ' ' + vars['tmp_img'], shell=True, stdout=subprocess.PIPE)
    results = str(export)

    if results:
	logThis("EXPORT THE ORIGINAL IMAGE TO A TMP FOLDER FOR RESIZING: " + results)
    else:
	logThis("COULD NOT EXPORT ORIGINAL IMAGE FOR RESIZING, EXITING NOW")
	sys.exit()

    for size in vars['sizes']:
        logThis('RESIZING TO HEIGHT: ' + vars['sizes'][size]['height'] + ', WIDTH: ' + vars['sizes'][size]['width'])

	if vars['ini_file'] in ['news_upload.ini', 'discography_upload.ini', 'letters_upload.ini']:
	    new_img_name = vars['sizes'][size]['height'] + '.' + vars['img_extension']
	else:
	    new_img_name = vars['img_name'].replace('.', '_'+vars['sizes'][size]['height']+'.')

        # create a new path and file name based on the dimensions of the new image
	new_path = vars['publish_image_path'] + new_img_name
        # resize the image using an ImageMagick command
	convert = subprocess.check_call(["convert", vars['source_image_fullpath'], "-scale", vars['sizes'][size]['width'] + 'x' + vars['sizes'][size]['height'] + '!',  new_path])
	convert_results = str(convert)

	if convert_results:
	    logThis("NEW IMAGE PATH: " + new_path)
	    # import the new image to the repository - this triggers an rsync, moving each image to production
            export = subprocess.check_call(["svn","import", vars['tmp_dir'] + ' ' + new_path + ' ' + '-m "adding file: ' + vars['publish_image_path'] + '"'])
	    results = str(export)

    # once all the resizing is done, clear out the tmp directory and image
    os.remove(vars['tmp_img'])
    os.removedirs(vars['tmp_dir'])
    return vars

logThis("SCRIPT STARTING")

checkParams()
checkState()
checkExists()
checkImageExtension()

makeTempDir()
makeSizes()
makePaths()
resizeImages()

logThis("ENDING SUCCESSFULLY")
sendMail()