AWS Lambda is a service that executes code in response to triggers and data received from other AWS services. It is our preferred solution for any kind of background processing in a project. Key benefits over past approaches include:
- Automated Administration - All of the infrastructure for running your code is handled so you don't need to worry about EC2 instances, operating systems, or maintenance. Also, logging and monitoring is provided by CloudWatch.
- Fault Tolerance - Capacity is distributed across multiple Availability Zones per region, protecting against machine and datacenter failures.
- Automatic Scaling - Code is executed only when needed and scales to handle the rate of incoming requests without configuration.
- Cost - Billing happens in increments of 100 milliseconds for only the requests that are served. Applications will no longer incur monthly EC2 charges for idle time in a worker process.
Historically background processing on projects have been triggered in two ways: by schedule (cron) or by message publication (SQS). AWS Lambda has a suitable replacement for both of these scenarios.
Any time you think you need a cron job, you can use a Lambda that is invoked by a CloudWatch Events Schedule.
Instead of publishing messages to an SQS queue, send them to SNS. Lambdas that subscribe to SNS Topics will receive the payload of messages that are published, from there the world is yours.
Once a Lambda has been initially configured using the AWS Console, you can push updates during the CI process using Fabric. The following
fabfile.py snippet demonstrates creating a ZIP file for Lambda code in a local path and updating an existing function by name using boto3:
import io import os from zipfile import ZipFile from boto3.session import Session from fabric.api import env, task AWS_ACCESS_KEY_ID = env.AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY = env.AWS_SECRET_ACCESS_KEY AWS_REGION = env.get('AWS_REGION', 'us-east-1') session = Session( aws_access_key_id=AWS_ACCESS_KEY_ID, aws_secret_access_key=AWS_SECRET_ACCESS_KEY, region_name=AWS_REGION) aws_lambda = session.client('lambda') def files_to_zip(path): for root, dirs, files in os.walk(path): for f in files: full_path = os.path.join(root, f) archive_name = full_path[len(path) + len(os.sep):] yield full_path, archive_name def make_zip_file_bytes(path): buf = io.BytesIO() with ZipFile(buf, 'w') as z: for full_path, archive_name in files_to_zip(path=path): z.write(full_path, archive_name) return buf.getvalue() @task def deploy_my_cool_lambda(): lambda_name = 'my-cool-lambda' lambda_dir = os.path.join('./path/to', lambda_name) aws_lambda.update_function_code( FunctionName=lambda_name, ZipFile=make_zip_file_bytes(path=lambda_dir))
- Lambda Function Handler (Python) - Info about the signature of a function handler and the parameters it receives when invoked.
- The Context Object (Python) - Info about how to get runtime info from the Lambda service in your function handler.
- Logging (Python) - Info about logging in your function handler.
- Exceptions (Python) - Info about how Lambda serializes any exceptions raised by the function handler.
Creating a Deployment Package (Python) - See the Advanced scenario section for info on including resources in your Lambda that you would normally