Updating Example Code for Accessing AWS EFS from Lambda

I’m always interested in learning new things about serverless and AWS, so in one of the modules I was teaching about Elastic File System (EFS), I thought it’d be kind of cool to go through the process of mounting a file share that is accessible from Lambda. I found an AWS article about this from June, 2020, “New – A Shared File System for Your Lambda Functions“, and went about seeing if it still works as-is. Spoiler alert: it doesn’t work, but fixing it was pretty easy.

Here is a very high-level diagram about what I was aiming for:

I would suggest reading the post over for a couple of pages to understand use cases and overall what you need to do, then come back here to get the correct code for implementing the solution.

Go with the Flow

Following the steps in the post, you should

1. Create the File System in EFS, using the default VPC. For testing, I created it just in one AZ, and I turned off backups (you get to this by choosing Customize when creating the file system). I chose the VPC’s default security group, which allows all traffic to items that are in the security group.
2. Create the Access Point. The old console UI apparently allowed one to create an Access Point during the creation of the File System, however, now you have to create the Access Point after you create the file system. Make sure the “Root directory path” is set to “/message“, then use the same value (1001) as the POSIX User ID and Group ID, as well the Root directory creation permissions Owner user ID and Owner group ID. Also set the POSIX permissions to 750.
3. Build the Lambda Function. Next step is to build the Lambda function implementing a MessageWall API to add, read, or delete text messages. Messages are stored in a file on EFS so that all concurrent execution environments of that Lambda function see the same content. I created a function called MessageWall in Python 3.9.
   • Now we have to add permissions, so go to the Configuration tab and choose Permissions. As the instructions say, open the Execution role and add policies for AWSLambdaVPCAccessExecutionRole and AmazonElasticFileSystemClientReadWriteAccess. Of course in a production environment, you can restrict access to a specific VPC and EFS access point.
   • Next is to connect the Lambda to the VPC in which the EFS mount target(s) exist. Again in the Configuration tab, go to VPC and choose the VPC and subnet(s) in which you have your mount targets. Make sure to set the security group to the same one you did for the file system (in my case, the default VPC security group).
   • Still under Configuration, go to File systems and add the file system you created, as well as specify the access point, and under Local mount path, use /mnt/msg.
   • Add the code. The code from the post does not work as is, specifically, (a) the event coming in has a single property under requestContext, ‘httpMethod‘ instead of split over ‘http‘ and ‘request‘, and (b) the message being returned should include a json payload with a status code, not simply the message body.

[
import os
import fcntl
import json

MSG_FILE_PATH = '/mnt/msg/content'


def get_messages():
    try:
        with open(MSG_FILE_PATH, 'r') as msg_file:
            fcntl.flock(msg_file, fcntl.LOCK_SH)
            messages = msg_file.read()
            fcntl.flock(msg_file, fcntl.LOCK_UN)
    except:
        messages = 'No message yet.'
    return messages


def add_message(new_message):
    with open(MSG_FILE_PATH, 'a') as msg_file:
        fcntl.flock(msg_file, fcntl.LOCK_EX)
        msg_file.write(new_message + "\n")
        fcntl.flock(msg_file, fcntl.LOCK_UN)


def delete_messages():
    try:
        os.remove(MSG_FILE_PATH)
    except:
        pass


def lambda_handler(event, context):
    method = event['requestContext']['httpMethod']
    if method == 'GET':
        messages = get_messages()
    elif method == 'POST':
        new_message = event['body']
        add_message(new_message)
        messages = get_messages()
    elif method == 'DELETE':
        delete_messages()
        messages = 'Messages deleted.'
    else:
        messages = 'Method unsupported.'
    return {
        'statusCode': 200,
        'body': json.dumps(messages)
    }
]

4. Add the API Gateway trigger as the instructions present, choosing the HTTP API.
5. In VS Code, using a terminal with bash (not PowerShell), I am able to execute the curl commands.

[
$ curl https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/default/MessageWall
No message yet.
kayej@DESKTOP-K0HIF0Q MINGW64 ~
$ curl -X POST -H "Content-Type: text/plain" -d 'Hello from EFS!' https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/default/MessageWall
Hello from EFS!

kayej@DESKTOP-K0HIF0Q MINGW64 ~
$ curl -X POST -H "Content-Type: text/plain" -d 'Hello again :)' https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/default/MessageWall
Hello from EFS!
Hello again :)

kayej@DESKTOP-K0HIF0Q MINGW64 ~
$ curl -X DELETE https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/default/MessageWall
Messages deleted.
kayej@DESKTOP-K0HIF0Q MINGW64 ~
$ curl https://xxxxxxxxx.execute-api.us-east-1.amazonaws.com/default/MessageWall
No message yet.
]

You can continue reading the post and following along, but I thought it’d be fun to connect an EC2 instance and mount the file share so I can examine and even change the file from either side (EC2 or Lambda). Here are the steps I did to make an EC2 instance and access it with Cloud9:

1. Go to AWS Cloud9 and create a new instance (AWS Linux 2, I used a t2.micro) in the VPC and a subnet with the file share. In my case, I only put the instance in 1 subnet so the choice was easy.
2. Once the instance is created, go to the EC2 page and modify the security group of the instance to include the VPC’s default security group — I need this so that the instance can interact with the file share.
3. On the EFS page, select my file system and then click the button “Attach”. It will give me instructions for mounting the file share in the instance (via DNS), particularly installing the EFS mount helper and creating the directory to mount:
   • sudo yum install amazon-efs-utils
   • sudo mkdir efs
   • sudo mount -t efs -o tls fs-0706axxxxxxxxxxx:/ efs

Now in Cloud9 I have my efs folder with “message” inside it. I left the directory permissions such that I have to sudo into efs/message, but now I can see the content I post from Lambda, and vice versa. Voila!