Search for a command to run...
Invoking One Lambda Function from Another in Python: A Quick Guide to Asynchronous Model Training
Yusuf Adeyemo is a DevOps Engineer from Nigeria. He loves helping startups deliver better software and provide more control over their environment and software development process with the help of modern tools and automation.
No comments yet. Be the first to comment.
We’ve all been there. You spend hours perfectly tuning your OpenWrt configuration, setting up your captive portals, and hardening your firewall. Then, six months later, the system hangs. Why? Because
This article started from a tweet. Someone on Twitter said they "lowkey want to understand the technology behind Google Authenticator" and I dropped a quick reply - explaining that it's basically TOTP: your device and the server share a secret key, b...
Have you ever sat there waiting for your CodeBuild project to rebuild your entire Docker image... again? Even though you only changed a single line of code? Yeah, me too. And it's frustrating. Today, I'm going to show you how I reduced our Docker bui...
After you complete this article, you will have a solid understanding of: Why deploying Next.js SSR on Cloudflare is different from traditional hosting The real difference between Cloudflare Pages with next-on-pages and OpenNext adapter How the fre...
After you complete this article, you will have a solid understanding of: What L1, L2, and L3 constructs actually are and when to use each Why AWS created three different abstraction levels (and the hidden benefits) How to avoid the most common CDK...
Imagine being tasked with the challenge of training machine learning (ML) models in a serverless environment at work. Your boss expects a fast, efficient, and scalable solution to this problem. A key requirement is for the primary Lambda function to return a response immediately after triggering the model training process, without waiting for the training to complete. Sounds like a tall order, doesn't it? Fortunately, AWS Lambda has got your back. In this article, we'll explore how Lambda invocation can be utilized to solve this issue by asynchronously calling another Lambda function using Python. We'll also demonstrate how the child function can receive data from the parent function.
AWS Lambda is a serverless compute service that allows you to run your code without provisioning or managing servers. Lambda invocation is a powerful feature that enables one Lambda function to call another, allowing you to offload tasks, process events in parallel, and set up complex workflows. In our case, invoking another Lambda function to train ML models enables the primary Lambda function to return a response immediately, ensuring that the main process doesn't get blocked waiting for the training job to finish.
To invoke a Lambda function from another, you'll first need the AWS SDK for Python, Boto3. Make sure you have it installed by running:
pip install boto3
Next, you need to set up the necessary permissions to allow the parent Lambda function to call the child Lambda function. In the AWS Management Console, navigate to the IAM role assigned to the parent Lambda function and attach the AWSLambdaRole policy.
Now let's create a simple parent Lambda function that will invoke the child Lambda function. The parent function will use Boto3's invoke method to make an asynchronous call to the child function.
import boto3
import json
def lambda_handler(event, context):
lambda_client = boto3.client('lambda')
# Replace 'child-function-name' with the actual name of your child Lambda function
response = lambda_client.invoke(
FunctionName='child-function-name',
InvocationType='Event', # Asynchronous invocation
Payload=json.dumps(event)
)
return {
'statusCode': 200,
'body': 'Model training job started.'
}
Here's a sample child Lambda function that trains an ML model using the data passed from the parent function:
import json
def lambda_handler(event, context):
# Load data from the event object sent by the parent Lambda function
training_data = json.loads(event['body'])
# Train your ML model here using the training_data
# ...
return {
'statusCode': 200,
'body': 'Model training completed.'
}
In this example, the parent Lambda function sends the input event as a payload to the child function, which then extracts the data and uses it for training the ML model.
As demonstrated in this article, using AWS Lambda invocation to asynchronously call another Lambda function is a powerful technique that can solve complex use cases such as initiating ML model training without waiting for the process to complete. By leveraging this approach, you can achieve faster response times, better scalability, and an efficient serverless architecture for your applications. The Python examples provided serve as a solid foundation to build upon for your specific use case, allowing you to focus on the core logic of your application rather than infrastructure management.