Add resizing to the Image Service with PIL

My previous article explained how to create an Image Service using API Gateway and Lambda Proxy integration. But we realized it had a big limitation with this solution related to the response size. There is a 10MB limit on API Gateway and now we can’t serve images bigger than that. Lambda also has a limit for the return size of 6MB, so the maximum response size is even reduced to 6MB.

So, how can we fix this? We couldn’t directly serve from S3 because we have a custom logic to map the requested URL into the path of the file. The next option is to resize the image before serving. So, why am I writing an article about it? Because, there is a catch!

The code change

Well, the code change is quite straightforward. We use PIL: the thirdparty library used for image manipulation in Python. We just need to add pillow into the requirements file, do the code change, and boom! Right?

Nope! But let’s first see the code change we have to do.

import os
from PIL import Image
from io import BytesIO

BASE_WIDTH = int(os.environ.get("MAX_IMAGE_WIDTH", 2000))


def _resize(binary_image):
    image = Image.open(BytesIO(binary_image))

    # Return the original image if it's already smaller
    if image.width < BASE_WIDTH:
        return binary_image

    # Resize the image maintaining the aspect ratio
    w_percent = (BASE_WIDTH / float(image.width))
    h_size = int((float(image.height) * float(w_percent)))
    resized_image = image.resize((BASE_WIDTH, h_size), Image.NEAREST)
    image.close()

    # Write the image into a new bytes object
    img_byte_arr = BytesIO()
    resized_image.save(img_byte_arr, format='JPEG')
    resized_image.close()

    return img_byte_arr.getvalue()

Let me explain. This function will take an image as a binary object and resize it if the width is higher than our predefined max. This block of code works locally without an issue! But this probably won’t work on a Lambda function. But why?

PIL binaries

Python Imaging Library (PIL) is a Python library that adds a lot of functionality for image manipulation. The original PIL was discontinued and now it’s called Pillow. It uses C++ behind the scene to perform all the hard work. So, we must have these C++ library files in the Lambda package we upload, for it to work properly. And these libraries are taken from your computer and get bundled into the package. That is the problem!

I have a Mac for my development and it bundles the MacOS specific libraries for the package. But Lambda functions run on a Linux-based environment, and these libraries don’t work there.

When you upload these to the Lambda package, it will give all kinds of errors and it took me some time to understand what was going on.

How to fix this?

We need to upload the correct libraries which are compatible with the Lambda environment along with the Lambda package, or we can create a Lambda layer with those libraries. It doesn’t matter how you do this, both approaches will work without an issue. I will explain the first option: packaging with the Lambda itself.

So, how can we extract the Lambda compatible libraries? There can be other ways, but I took the Docker image approach, which is explained below.

1. Create a Docker container with Lambda dependencies.
2. Install pillow on it.
3. Extract the lib files from it to your computer.

FROM public.ecr.aws/lambda/python:3.8

RUN pip install pillow

CMD [ "app.handler" ]

Just add a simple Python file with a handler function for the sake of completion.

Then build the docker image with:
docker build -t lambda_image .

Then run the Docker image with:
docker run lambda_image

Note down the container ID of the running image, or use docker container ls to obtain the container details.

Then run the below command (in a new terminal) to copy the files from the container into your computer.

docker cp <container_id>:/var/lang/lib/python3.8/site-packages/ ~/test/

Note: The path should reflect the correct Python version you defined in the Dockerfile.

Then all the dependencies of the Lambda function will be copied into the ~/test/ folder and you can do whatever you want with them.

How to package?

There are many ways you can inject these libraries into your Lambda function. I took the easiest way: copying them into the build folder before packaging the code.

I created a zip file of the required libraries (it’s only Pillow for the moment), and placed it inside a libs folder. Then I extract it as the last step of the dependency installation.

build:
   @mkdir -p _build
   @pip install --upgrade pip
   @pip install -r src/api/requirements.txt -t _build
   @echo "Unzipping libs"
   @unzip -o libs/PIL.zip -d _build/  # <-- Unzips the libs

These files will be packaged when I call cloudformation package on the Lambda function. You can also create a Lambda Layer and upload these into that, but that’s up to you to do.

Tada! Now your Lambda function uses proper binaries/libraries of PIL and your Image Service doesn’t crash for bigger images.

Summary

If you thought that a Python code is cross platform, it’s not quite the case, especially when it has underlying compiled libraries. Then we have to package the correct/compatible libraries to our Lambda function (or any other service).

In this article, I explained how I integrated Image Resizing using PIL and how I managed to get it working properly on Lambda functions.

Do you think there is a better way? Drop a comment and let me/everyone know!

Cheers!