[Python] How to ensure compatibility between my local dev environment and AWS Lambda runtime...

I am currently having an issue with Python packages on AWS Lambda.

I have defined a Lambda layer like this:

my_layer = _lambda.LayerVersion(
self, "MyLayer",
code=_lambda.Code.from_asset("layer_code_directory"),
compatible_runtimes=[_lambda.Runtime.PYTHON_3_12],
description="Lambda Layer for common dependancies"
)


The layer is used for common dependencies that most of my Lambdas share. One of them is the cryptography package. The pip install all the packages in the requirements.txt into "layer_code_directory". The CDK (afaik) packages those into a zip and then uploads in to the lambda layer.

However, when I try to run my Lambda, I get an error, that some files from cryptography are missing:

Runtime.ImportModuleError: Unable to import module 'MyAuthorizer': /opt/python/cryptography/hazmat/bindings/_rust.abi3.so: cannot open shared object file: No such file or directory

I have debugged this by checking whether the package is available to the Lambda and what paths those files have that are missing. I crosschecked them with the error message and all the files are available in the correct places.

From some Google research I now assume, that the version of cryptography, I install locally via pip is not compatible with the lambda runtime, and afaik, the pip installs different files depending on what OS you are on.

On my first tries, I was on Alpine (Official Python docker container), and I have now moved to a container based on the AWS Python, Lambda, and Image. I assumed, when I bundle packages installed in this environment, the packages would be compatible with AWS Lambda. But I still got the same error message.

How to solve this issue or what else there could be?

I assume, that deploying Lambda code and packages via AWS CDK is a very common practice, but using Google, AWS, Docs, and LLMs.

I was unable to find the solution or a working example.

Continue reading...