Deploy serverless punch cards with AWS Lambda Container
How to deploy a Fortran container in serverless mode using AWS Lambda
Following what I said in my previous post (available in Italian here), with this new one I will show you something that, after the announcement of AWS Container Lambda, it’s now possible to implement and that it was much more difficult in the past. The purpose of this post is to demonstrate how you can deploy a punch cards in serverless mode using the new AWS Lambda service.
Starting from the fantastic talk by SparkFabrik (Paolo Mainardi), available here in italian, I forked the repository which uses a similar approach to deploy a punch card on Google Cloud Run. At the end of this post we will also be able to evaluate the differences between the two projects in order to identify the different approaches of Google and AWS in terms of serverless containers. The idea of using punch cards (and Fortran), as reported by Paolo in his talk, stems from the fact that it can be considered as the “mother of all legacy”.
Let’s assume we want to bring two complex legacy functions written in Fortran language to the Cloud and we want to preserve the original business logic of the code.
In our case the two functions that we want to migrate in Cloud will be “sum” and “triangle-area”. Considering that the implementation of a Lambda Runtime API in Fortran is not available (for obvious reasons!), we must definitely write a wrapper that will help us to invoke our function with the correct parameters. Among the implementations available I chose to use Python that allows to make it writing very few lines of code.
From Punch Cards to Fortran code
Imagine having real punched cards and having to start with the digital transformation of this. The first thing to do is to transform these punched cards into code. Following what Paolo proposes in his talk, the easiest way to do it is to use this site (https://www.masswerk.at/cardreader/). By uploading a Fortran punch card, the code is returned in output, making it downloadable.
Integration
The Python Wrapper, needed for integration with “AWS Lambda Runtime Client”, is also used to read the request parameters and return the response in JSON. The purpose of this project is to avoid modifying the original Fortran code
According to code snippet, the Python Wrapper simply takes care of selecting the function to use according to the "operation" parameter. Once the correct function is selected, the parameters are retrieved, a subprocess is invoked to execute the Fortran code and the result is retrieved from the STDOUT. The final step is to build the response JSON based on the outcome of the invocation. In case of an unsupported operation, an HTTP status code 400 is returned and in case of unexcepted error an HTTP status 500 is returned after invocation.
The Dockerfile used for this project is almost the same that you can find on AWS Blog post here. I just add the installation of libgfortran for runtime and gfortran for build process.
The installation of aws-lambda-runtime-interface-emulator (line 50–51), as reported on AWS Blog post, is not mandatory because it’s needed only for local test purpose.
Build & Deploy
The CI/CD pipeline of this project is implemented with the GitHub Actions. It isn’t the purpose of this post to illustrate the features of Github Actions so I will only highlight what is useful for this project. After push on master branch of project, a build and deploy is triggered. When the build is triggered, the first step performed is the build of docker container. After this build, the image is pushed in a private ECR and a CloudFormation stack is executed for the deploy. The stack use “AWS Serverless Application Model” for Lambda Function:
With the SAM shown above, we are able to deploy a Lambda Function with Api Gateway Trigger with just few lines in a YAML file. Unfortunatelly, at the time of this writing, it is not available a pre-existent GitHub Actions for AWS Lambda Containers deploy.
Running
After the deployment, we finally have the endpoint to test serverless punch cards. By making a POST request to API Gateway Endpoint, we can verify that the Fortran code is executed and the result is returned to the caller.
curl -X POST -H 'Content-Type: application/json' https://<ENDPOINT>.execute-api.<REGION>.amazonaws.com/<stage>/ --data "{\"operation\":\"sum\",\"a\":\"2\", \"b\":\"3\", \"c\":\"4\"}"{"result": "9"}
If necessary parameters are missing, an error is returned. Error handling is not implemented optimally but this was not the purpose of this post.
curl -X POST -H 'Content-Type: application/json' https://<ENDPOINT>.execute-api.<REGION>.amazonaws.com/<stage>/ --data "{\"operation\":\"wrong_operation\",\"a\":\"2\", \"b\":\"3\", \"c\":\"4\"}"{"message": "Unsupported Operation"}
Regarding the invocation time, in case of cold start the average response is returned after about 2250 ms. In case of an hot invocation, the times are much better and the average response is returned after about 500 ms. These times obviously depend on a series of factors including the size of the image, the integration mode of the Lambda, the latency between the user and the chosen region, etc.
Conclusion
Looking at implementation done by Paolo, the main difference is related to request handling.
In case of Google Cloud Run, the container need to have HTTP Capabilities. Paolo use Nginx with CGI and wrap the invocation of Fortran function with two scripts. For AWS, the implementation can leverage on AWS Lambda Runtime Client. With it, the Python Wrapper does not need to take care of handling HTTP requests. In terms of reuse, the solution proposed by Google Cloud Run is the best since the container is self-consistent and also works outside the Google ecosystem. In terms of ease of integration, the AWS solution could be better since the programmer has to deal only with the implementation of the business logic by setting up the handler method. In the AWS solution it is not necessary to perform tuning of web servers or other parameters that are not part of the implementation as it relies on the client that is developed by AWS.
However, it is true that from a Cloud migration perspective, often of the services that already exist have capabilities of managing HTTP requests. For example, the migration of a Spring Boot service to Google Cloud Run does not require any effort while on AWS a rework is certainly necessary.
The answer to which service is best therefore depends on the use case. Like anything in the IT landscape, there is no better technology than any other.
As last consideration I would like to focus on the ease of deployment: with Google Cloud Run the creation of a lambda container is feasible with a single command. With AWS, it is necessary to use a SAM with CloudFormation as the CLI deployment would be very complex. As mentioned before, however, AWS Lambda Container can be attached to any existing trigger for "traditional" Lambda. For this reason it was chosen, losing the simplicity of deployment for the simplest setups, to create a SAM for CloudFormation that allows you to configure all the necessary settings (for example the trigger that will execute the lambda).
All code used for this project is available on my GitHub. You can find it in this repository:
