How to include node_modules when using typescript aws-cdk?

[zoonman]

❓ How to include node_modules using typescript aws-cdk?

My lambda is supposed to rely on the whole bunch of different modules, part of which is private.
How should I tackle this problem with conformity with aws-cdk?

Environment

• CDK CLI Version: 1.6.0 (build 3a0cde0)
• OS: all
• Language: TypeScript

Other information

Could you provide example when used some external node module, axios or joi or something else?

[JoshM1994]

I have prepared a short example of how to include a set of node_modules below. The basic answer is that you need your node_modules folder to be in the same folder as specified in new lambda.AssetCode('this_folder')

1. Assume you are starting a fresh project with cdk init sample-app --language=typescript
2. Create a new top-level folder called src where you will be storing your Lambda functions
3. Run npm init inside src to generate a package.json
4. npm install your_module inside src --> you should now have a node_modules folder inside src
5. Perform all the usual steps to create your lambda and specify the code key as new lambda.AssetCode('src')
6. Run cdk synth from the root of your project
7. Look at cdk.out and you will see a folder starting asset.somethingHere --> this asset folder should contain your lambda handler and the node_modules required by it
8. Deploy and you're good to go

The final tree structure should look something like this:

├── bin
│   └── lambda.ts
├── cdk.json
├── cdk.out
│   ├── LambdaStack.template.json
│   ├── asset.lotsOfLetters
│   │   ├── basic.js
│   │   ├── node_modules # Node modules found here
│   │   ├── package-lock.json
│   │   └── package.json
│   ├── cdk.out
│   ├── manifest.json
│   └── tree.json
├── lib
│   └── lambda-stack.ts
├── package-lock.json
├── package.json
├── src
│   ├── basic.js
│   ├── node_modules # this is inside the src, (in addition to the root one required to build)
│   ├── package-lock.json
│   └── package.json

For reference, my lambda-stack.ts file looks like:

import lambda = require('@aws-cdk/aws-lambda')
import cdk = require('@aws-cdk/core');

export class LambdaStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const myLambda = new lambda.Function(this, 'iHaveNodeModules', {
      code: new lambda.AssetCode('src'),
      handler: 'basic.handler',
      runtime: lambda.Runtime.NODEJS_10_X
    })
  }
}

My basic.js example lambda (yes, the left-pad usage is meant to be ironic)

const leftPad = require('left-pad');

const handler = async () => ({
    statusCode: 200,
    body: leftPad(1, 3, '0')
});

module.exports = {
    handler
};

After deploying, see the screenshot of what the Lambda console looks like with running a test event:

[zoonman]

Josh, it looks like a complete hack & absolutely dirty solution.
As part as regular development process goes - infrastructure is an utilitary part of the project and normally has to be considered as dev dependency. Nobody wants to spend more than 5% of their project time on it and write endless boilerplate code to trivially deploy API.

Project structure supposed to look like this:

infra/cdk/index.ts
src/application-lamdbas-code.ts
src/application-lamdbas-code.spec.ts
node_modules/*
package.json
.env

I expect cdk to be able to create all the stuff inside aws by npm run deploy.
To be honest I would prefer npm run deploy to run tests, assert that code was committed and properly tagged so generated artifacts will be versioned and deployed to a correct matching stage.

I advise you to get out of cavern and look at something like serverless framework.

[JoshM1994]

I advise you to get out of cavern and look at something like serverless framework.

Yes, I agree that it is hacky and dirty but I do not believe CDK was designed to be an entire build system for every single part of AWS' enormous stack. Serverless Framework is certainly better in its ability to package node_modules for you but you lose a lot of flexibility in doing so.

It's really no extra work to have your npm run deploy do a very quick packaging step for you. Something super basic like

zip -r lambdas.zip src/* node_modules/*

and then replace the code in my example above with

const myLambda = new lambda.Function(this, 'iHaveNodeModules', {
  code: lambda.Code.fromAsset('lambdas.zip'),
  handler: 'src.basic.handler',
  runtime: lambda.Runtime.NODEJS_10_X
})

(Obviously make sure your CI only runs npm install --only=production to avoid packaging all your devDependencies as well!)

P.S. We actually switched away from Serverless Framework in favour of CDK because of CDK's much greater range of product offerings (Elastic Beanstalk) and intuitive IAM role creation (something we found to be lacking in Serverless because of its multi-vendor approach).

[NGL321]

@zoonman Sorry that no one got back to you sooner. @JoshM1994 is correct however, the CDK is not designed to be the end-all-be-all build system for everything in AWS. The AWS ecosystem is far too complex to have such specific deployment structure.

Josh's solution mirrors exactly what I would have encouraged to deploy lambdas with dependencies. It is not the most optimal solution, but it is how you would deploy a lambda without the CDK, and therefore what it is mirroring. Each step can be very easily automated within npm run deploy.

All that said, this is a fairly specific structure, and I'm not sure it makes sense to create an example for it in this repo. If you would like to request adding a specific lambda feature, I recommend you create an issue on the main CDK repo rather than the one for code samples

[SomayaB]

Closing this issue since a solution has been provided. Feel free to reopen.