Local and Automated Testing with AWS Lambda ( Node.js )

In one of our new projects in the Viacom technology team, we decide to kick the tires with AWS Lambda. We needed some basic translation / proxying of existing APIs, the ability to prototype the API by just spitting out JSON files from S3, and maybe storing a few things in a key/value database. We have many different ways to solve for this in various teams, but Lambda offers a quick way of prototyping without the need to spin up containers or virtual machines. It's server-less, I.E. Amazon handles all of this for you.

Lambda allows for a few different languages. We opted for Node.js. While our time-to-market with this new API was fast, we also wanted a good way to ensure that as as we refactor and add new endpoints we do not cause harm. In other words, T.D.D. -- or at least development with some regression tests ( D.W.S.R.T. ). The initial documentation from AWS doesnt really cover how one might go about this. The examples push code into their servers and use aws tools or a hosted development environment to test your lambda functions.

After a bit of research on some competing open-source solutions, we landed on lambda-local. It has a convenient CLI ( command line interface ) and also can be executed in Node for automated testing in your favorite testing framework. For our tests, I opted to use tape. since it felt to be pretty no-nonsense.

Now, lets jump into our testing solution:

Pre-Requisites:

Install node and aws CLI

  • node
  • aws
  • aws login and cli setup:
    • default region to ~/.aws/config
    • credentials to ~/.aws/credentials
    • or just run aws configure once you have the aws cli installed
  • working lambda functions
  • understanding of lambda, aws API gateway, and Node / NPM

A typical project package.json:

{
  "name": "my-lambda",
  "version": "1.0.0",
  "description": "lambda project for automated testing",
  "main": "index.js",
  "scripts": {
    "test": "node ./tests/integration_tests.js | tap-spec"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "async": "^2.0.0-rc.4",
    "lodash": "^4.12.0",
    "request": "^2.72.0",
    "tape": "^4.6.3",
    "tap-spec": "^4.1.1",
    "lambda-local": "1.3.0",
    "winston": "^2.3.1"
    }
}

Testing lambdas locally with the CLI:

Install local lambda ( should be able to run from the path ):

#sudo npm install -g lambda-local

From one of your lambda folders, run the lambda-local cli, passing in the function js, handler, and an event sample:

#lambda-local -l index.js -h handler -e event-samples/test-data.js

An event sample is basically the input paramaters you would send from the api call. Here is an example one event-samples/test-data.js:

module.exports = {  
    deviceId: "5432543543534"
};

This is the equivilent of calling http://mylambdaapi.com/api/methodname?deviceId=5432543543534, but using local-lambda, you can call it from your local machine. It gives you a nice log of the calls and any output from the function. Here is an example output:

info: -----  
info: lambda-local successfully complete.  
C02MNAPNFD57:seamless hillm$ lambda-local -l index.js -h localHandler -e event-samples/spike.js  
info: Logs  
info: ------  
info: START RequestId: 8fba8f39-3191-68b8-016e-a1e9e5f45efe  
{ url: 'http://someotherapi.mydomain.com/api/video/14f30',
  headers: { 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/601.6.17 (KHTML, like Gecko) Version/9.1.1 Safari/601.6.17' } }
info: END  
info: Message  
info: ------  
info: {  
    "package": {
        "video": {
            "item": [
                {
                    "rendition": {
                        "type": "application/x-mpegURL",
                        "method": "hls",
                        "duration": "1270",
                        "cdn": "akamai"
                    },
                    "origination_date": "02-14-2017 20:33:00"
                }
            ]
        },
        "version": "1.7.1.1"
    }
}
info: -----  
info: lambda-local successfully complete.  

Automated ( integration ) testing with the library:

NPM install tape, tap-spec, lambda-local, and winston in your package dependancies:

"dependencies": {
...
    "tape": "^4.6.3",
    "tap-spec": "^4.1.1",
    "lambda-local": "1.3.0",
    "winston": "^2.3.1"
    }

Now define how you want to run tests in package.json. I have a folder called tests in which ill create a file integration_tests.js:

"scripts": {
    "test": "node ./tests/integration_tests.js | tap-spec"
  },

A typical test file /test/integration_tests.js:

var test = require('tape');  
var lambdaLocal = require('lambda-local');  
var winston = require('winston');

var lambdasPath = '../lambdas/'

function testLocalLambda(func, event, cb) {  
    var lambdaFunc = require(func);
    var lambdaEvent = require(event);
    winston.level = 'none'; //'error' //'debug', 'info'
    lambdaLocal.setLogger(winston);
    lambdaLocal.execute({
        event: lambdaEvent,
        lambdaFunc: lambdaFunc,
        lambdaHandler: 'handler',
        callbackWaitsForEmptyEventLoop: false,
        timeoutMs: 5000,
        mute: true,
        callback: cb
    });
}

test('myfirst_test', function (t) {

    testLocalLambda(lambdasPath + 'alambdafunc/index.js', lambdasPath + 'alambdafunc/event-samples/event.js', 
        function (_err, _data) {
            err = _err;
            json = _data;
            //console.log("JSON" + JSON.stringify(json))
            t.equal(err, null, 'no errors');
            t.notEqual(json.package, null, 'has package object');
            t.notEqual(json.package.video, null, 'has package/video object');
            t.equal(json.package.video.item.length, 1, 'items length 1');
            const video_item = json.package.video.item[0];
            t.notEqual(video_item.rendition, null, 'has video item rendition object');
            t.notEqual(video_item.rendition.type, null, 'has video item rendition type object');
            t.equal(video_item.rendition.type, 'application/x-mpegURL', 'video item rendition type mpeg');
            t.end()
        });
});

Whats going on here:

1) A testLocalLambda function to do the following:
- load up the lambda function file and event - set logging level by attaching winston to localLambda - set options for localLambda execution

2) A test that calls testLocalLambda sending it the path to a specific function and event data and then doing test assertions on the json output or errors.

Run your tests:

npm test  

It should look something like this:

> discover-lambda@1.0.0 test /Users/hillm/CocoaApps/Zenith-API
> node ./tests/integration_tests.js | tap-spec

  myfirst_test

    ✔ no errors
    ✔ has package object
    ✔ items length 1
    ✔ has video item rendition object
    ✔ has video item rendition type object
    ✔ has package object
    ✔ video item rendition type

  total:     7
  passing:   7
  duration:  0.9s

In fact, tap-spec even uses color-ansi and looks great in a color terminal:

Now all we have to do is connect this up to a build script so we run the tests before deploying to AWS, but we will save that for another post.

Mike Hill

Read more posts by this author.