Simon Wadsworth

Master of Science | Computer Scientist

Updating METADATA on Amazon S3 objects
Published Wed 15 June 2011 by Simon Wadsworth in Internet

So I host the static content from my blog on the Amazon S3 Simple Storage Service. This allows me to remove some of the load of my server for static content. However this means that over time I need to pay money for the S3 hosting, and if I have a lot of requests this could end up costly. So how do I get around this. Well by setting the Content-Control META tag onto the objects in S3, I can ensure that the static content is cached by the remote user for however long I want. In this case I have set it for 7 days. However updating all the files in S3 would take a long time to do manually, so I use this Python code to update the objects in my S3 bucket.

I had to modify it to support encoding as I use gzip encoding on some of the static content to reduce the amount of data needing to be transferred.

from boto.s3.connection import S3Connection

connection = S3Connection('API_KEY', 'API_SECRET')

buckets = connection.get_all_buckets()

for bucket in buckets:
    for key in bucket.list():
        print('%s' % key)
        encoding = None
        if key.name.endswith('.jpg'):
            contentType = 'image/jpeg'
        elif key.name.endswith('.gif'):
            contentType = 'image/gif'
        elif key.name.endswith('.png'):
            contentType = 'image/png'
        elif key.name.endswith('.css.gzip'):
            encoding = 'gzip'
            contentType = 'text/css'
        elif key.name.endswith('.js.gzip'):
            contentType = 'application/x-javascript'
            encoding = 'gzip'
        elif key.name.endswith('.css'):
            contentType = 'text/css'
        elif key.name.endswith('.js'):
            contentType = 'application/x-javascript'
        else:
            continue
        if encoding is not None:
            key.metadata.update({
                'Content-Type': contentType,
                'Cache-Control': 'max-age=604800',
                'Content-Encoding': encoding
            })
        else:
            key.metadata.update({
                'Content-Type': contentType,
                'Cache-Control': 'max-age=604800'
            })
            key.copy(
                key.bucket.name,
                key.name,
                key.metadata,
            )
            key.set_acl('public-read')

Comments


CONTACT

Email
contact at simonwadsworth dot geek dot nz

Address
Hamilton, Waikato
New Zealand

Phone
+64 27 403 4531
02825516452@2talk.co.nz

SOCIAL LINKS

Generated with Pelican

Created by BLACKTIE.CO