What is AWS Lambda?#
AWS lambda is a popular serverless computing service that allows you to write code in any language and run it on AWS. In this case "serverless" means that rather than having to manage a server or set of servers, you can instead run your code on-demand on highly-available machines in the cloud.
Lambda manages your "serverless" infrastructure for you including:
- Server maintenance
- Automatic scaling
- Capacity provisioning
- and more
AWS Lambda functions are a "black box"#
However, the tradeoff that happens as a result of using AWS Lambda is that because AWS handles so much of the infrastructure and management for you, it ends up being somewhat of a "black box" with regards to:
- Cost: You have little insight into why your Lambda function is costs so much or what functions are responsible
- Performance: you often run into hard-to-debug latency or memory issues when running your Lambda function
- Reliability: You have little insight into why your Lambda function is failing as much as its failing
Depending on availability of resources, these issues can be balloon over time until they become an expensive foundation which is hard to analyze and fix post-facto once much of your infrastructure relies on these functions.
Continuous Profiling for Lambda: A window into the serverless "black box" problem#
Continuous Profiling is a method of analyzing the performance of an application giving you a breakdown of which lines of code are consuming the most CPU or memory resources and how much of each resource is being consumed. Since, by definition, a Lambda function is a collection of many lines of code which consume resources (and incur costs) on demand, it makes sense that profiling is the perfect tool to use to understand how you can optimize your Lambda functions and allocate resources to them.
While you can already use our various language-specific integrations to profile your Lambda functions, with the naive approach, adding Pyroscope will add extra overhead to the critical path due to how the Lambda Execution Lifecycle works:
However, we've introduced a more optimal solution which gives you insight into the Lambda "black box" without adding extra overhead to the critical path of your Lambda Function: our Pyroscope AWS Lambda Extension.
Pyroscope Lambda extension adds profiling support without impacting critical path performance#
This solution makes use of the extension to delegate profiling-related tasks to an asynchronous path which allows for the critical path to continue to run while the profiling related activities are being performed in the background. You can then use the Pyroscope UI to dive deeper into the various profiles and make the necessary changes to optimize your Lambda function!
How to add Pyroscope's Lambda Extension to your Lambda Function#
Pyroscope's lambda extension works with our various agents and documentation on how to integrate with those can be found in the integrations section of our documentation. Once you've added the agent to your code, there are just two steps needed to get up and running with profiling lambda extension:
Step 1: Add a new Layer in "function settings"#
Add a new layer using the latest "layer name" from our releases page.
Step 2: Add environment variables to configure where to send the profiling data#
You can send data to either either pyroscope cloud or any running pyroscope server. This is configured via environment variables like this:
Lambda Function profile#
Here's an interactive flamegraph output of what you will end up with after you add the extension to your Lambda Function:
While this flamegraph is exported for the purposes of this blog post in the Pyroscope UI you have additional tools for analyzing profiles such as:
- Labels: to view function cpu performance or memory over time using FlameQL
- Time controls: to select and filter for particular time periods of interest
- Diff view: Compare two profiles and see the differences between them
- And many more!