Implementing Unit Testing

When developing any dynamic functionality it is best practice to implement testing. This will ensure anytime the code is deployed it is in a working state and will provide a good user experience to the visitors/users of your service.

Pre-requisites

This article will be based on the blog described in Hexo, AWS and Serverless Framework, Securing A Test Environment Using AWS WAF and Securing S3 Bucket from Direct Access; however, the basic concepts should be applicable to any NodeJS functionality or any static site.

Install a test framework for Javascript

Although it's possible to write a custom test framework, it's much easier to use an existing solution. This example will use Jest. Start by installing Jest with NPM.

Install Jest
1
npm i jest --save-dev

Create a test

The next step is to create a test. As the example project only has one JavaScript function these instructions will only implement a test for it. There is no reason many more tests can't be created. To make it easy to find tests the new test will be created in tests/functions/urlRewrite.test.js.

URL Rewrite test suite
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
// Load the file to test
const urlRewrite = require('../../functions/urlRewrite');
// Load some data that can be reused for other lambda@Edge functions
const lambdaAtEdgeFixture = require('../fixtures/lambdaAtEdge');

test(
'url-rewrite handler appends index.html to root request',
() => {
const expectedResponse = { uri: lambdaAtEdgeFixture.event.root_object.Records[0].cf.request.uri + 'index.html' };
expect(
urlRewrite.handler(
lambdaAtEdgeFixture.event.root_object,
lambdaAtEdgeFixture.context.webClient,
lambdaAtEdgeFixture.callback
)
).toEqual(expect.objectContaining(expectedResponse));
}
);

test(
'url-rewrite handler appends index.html to directory requests',
() => {
const expectedResponse = { uri: lambdaAtEdgeFixture.event.subdirectory.Records[0].cf.request.uri + 'index.html' };
expect(
urlRewrite.handler(
lambdaAtEdgeFixture.event.subdirectory,
lambdaAtEdgeFixture.context.webClient,
lambdaAtEdgeFixture.callback
)
).toEqual(expect.objectContaining(expectedResponse));
}
);

test(
'url-rewrite handler does not append index.html to root-directory requests',
() => {
const expectedResponse = { uri: lambdaAtEdgeFixture.event.file.Records[0].cf.request.uri };
expect(
urlRewrite.handler(
lambdaAtEdgeFixture.event.file,
lambdaAtEdgeFixture.context.webClient,
lambdaAtEdgeFixture.callback
)
).toEqual(expect.objectContaining(expectedResponse));
}
);

test(
'url-rewrite handler does not append index.html to non-directory requests',
() => {
const expectedResponse = { uri: lambdaAtEdgeFixture.event.file_in_subdirectory.Records[0].cf.request.uri };
expect(
urlRewrite.handler(
lambdaAtEdgeFixture.event.file_in_subdirectory,
lambdaAtEdgeFixture.context.webClient,
lambdaAtEdgeFixture.callback
)
).toEqual(expect.objectContaining(expectedResponse));
}
);

The 4th line of the test includes a file that doesn't exist yet. This file contains some sample values that can be reused if we need to test other Lambda@Edge functions. Create the file at tests/fixtures/lambdaAtEdge.js (this is a large file so the content is at the bottom of this post).

Test that the test suite works.

Run Jest
1
jest

Now that the javascript is tested, it's worth testing the links on the static site to make sure they are valid. For this broken-link-checker-local can do all the hard work, so install it.

Install broken-link-checker-local
1
npm i broken-link-checker-local --save-dev

As long as the site has been deployed previously (it has been if you're following all the blog posts) it's possible to test for broken links.

Test the deployment container for broken links
1
blcl --filter-level 3 --get --recursive public

Add NPM scripts to test more easily

Remembering the right commands, especially when managing multiple sites, can be confusing and easy to forget. To avoid this, it's possible to define some scripts in the pacakge.json file. The commands to add are as follows:

  • clean: a simple wrapper for hexo clean
  • build: a wrapper for hexo generate
  • buildclean: a wrapper for both clean and build
  • linkcheck: run the link checking script
  • jest: run the jest tests
  • test: run an end-to-end test

Add the scripts to package.json after devDependencies (at line 25)
1
2
3
4
5
6
7
8
"scripts": {
"clean": "hexo clean",
"build": "hexo generate",
"cleanbuild": "npm run clean && npm run build",
"linkcheck": "blcl --filter-level 3 --get --recursive public",
"jest": "jest",
"test": "npm run cleanbuild && npm run jest && npm run linkcheck"
}

Ignore static site files when running Jest

Because the static site can contain files that Jest will assume are tests a configuration needs to be added to package.json to ignore these files.

Configure Jest to ignore static site files in package.json (at line 33)
1
2
3
4
5
6
7
"jest": {
"testPathIgnorePatterns": [
"/node_modules/",
"/source/code/",
"/public/"
]
}

Run the full test suite

Running a full test is now as simple as using NPM to run the test command.

Run the full test suite
1
npm run test

Based on the posts undertaken in the examples the following links will show as broken in a number of files:

  • atom.xml
  • favicon.png

The Final Configuration Files

[test/functions/urlRewrite.test.js.js] []view raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
// Load the file to test
const urlRewriteTest = require('../../functions/urlRewrite');
// Load some data that can be reused for other lambda@Edge functions
const lambdaAtEdgeFixture = require('../fixtures/lambdaAtEdge');

test(
'url-rewrite handler appends index.html to root request',
() => {
const expectedResponse = { uri: lambdaAtEdgeFixture.event.root_object.Records[0].cf.request.uri + 'index.html' };
expect(
urlRewriteTest.handler(
lambdaAtEdgeFixture.event.root_object,
lambdaAtEdgeFixture.context.webClient,
lambdaAtEdgeFixture.callback
)
).toEqual(expect.objectContaining(expectedResponse));
}
);

test(
'url-rewrite handler appends index.html to directory requests',
() => {
const expectedResponse = { uri: lambdaAtEdgeFixture.event.subdirectory.Records[0].cf.request.uri + 'index.html' };
expect(
urlRewriteTest.handler(
lambdaAtEdgeFixture.event.subdirectory,
lambdaAtEdgeFixture.context.webClient,
lambdaAtEdgeFixture.callback
)
).toEqual(expect.objectContaining(expectedResponse));
}
);

test(
'url-rewrite handler does not append index.html to root-directory requests',
() => {
const expectedResponse = { uri: lambdaAtEdgeFixture.event.file.Records[0].cf.request.uri };
expect(
urlRewriteTest.handler(
lambdaAtEdgeFixture.event.file,
lambdaAtEdgeFixture.context.webClient,
lambdaAtEdgeFixture.callback
)
).toEqual(expect.objectContaining(expectedResponse));
}
);

test(
'url-rewrite handler does not append index.html to non-directory requests',
() => {
const expectedResponse = { uri: lambdaAtEdgeFixture.event.file_in_subdirectory.Records[0].cf.request.uri };
expect(
urlRewriteTest.handler(
lambdaAtEdgeFixture.event.file_in_subdirectory,
lambdaAtEdgeFixture.context.webClient,
lambdaAtEdgeFixture.callback
)
).toEqual(expect.objectContaining(expectedResponse));
}
);

[package.json] []view raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
{
"name": "hexo-site",
"version": "0.0.0",
"private": true,
"hexo": {
"version": "3.8.0"
},
"devDependencies": {
"@silvermine/serverless-plugin-cloudfront-lambda-edge": "^2.1.1",
"broken-link-checker-local": "^0.2.0",
"hexo": "^3.7.0",
"hexo-generator-archive": "^0.1.5",
"hexo-generator-category": "^0.1.3",
"hexo-generator-index": "^0.2.1",
"hexo-generator-tag": "^0.2.0",
"hexo-renderer-ejs": "^0.3.1",
"hexo-renderer-marked": "^0.3.2",
"hexo-renderer-stylus": "^0.3.3",
"hexo-server": "^0.3.1",
"jest": "^24.0.0",
"serverless-lambda-version": "^0.1.2",
"serverless-plugin-scripts": "^1.0.2",
"serverless-s3-deploy": "^0.8.0"
},
"scripts": {
"clean": "hexo clean",
"build": "hexo generate",
"cleanbuild": "npm run clean && npm run build",
"linkcheck": "blcl --filter-level 3 --get --recursive --exclude http://2019-01-28-securing-s3.demo.alphageek.com.au.s3-website-us-east-1.amazonaws.com --exclude http://localhost:4000 --exclude http://dev./%3Cyour_domain%3E/ public",
"jest": "jest",
"test": "npm run cleanbuild && npm run jest && npm run linkcheck"
},
"jest": {
"testPathIgnorePatterns": [
"/node_modules/",
"/source/code/",
"/public/"
]
}
}

[serverless.yml] []view raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
# The name of your project
service: <project>

# Plugins for additional Serverless functionality
plugins:
- serverless-s3-deploy
- serverless-plugin-scripts
- '@silvermine/serverless-plugin-cloudfront-lambda-edge'

# Configuration for AWS
provider:
name: aws
runtime: nodejs8.10
profile: serverless
# Some future functionality requires us to use us-east-1 at this time
region: us-east-1

# This enables us to use the default stage definition, but override it from the command line
stage: ${opt:stage, self:provider.stage}
# This enables us to prepend the stage name for non-production environments
domain:
fulldomain:
prod: ${self:custom.domain.domain}
other: ${self:custom.stage}.${self:custom.domain.domain}
# This value has been customised so I can maintain multiple demonstration sites
domain: ${self:custom.postname}.${self:custom.domain.zonename}
domainname: ${self:custom.domain.fulldomain.${self:custom.stage}, self:custom.domain.fulldomain.other}
# DNS Zone name (this is only required so I can maintain multiple demonstration sites)
zonename: alphageek.com.au
cacheControlMaxAgeHTMLByStage:
# HTML Cache time for production environment
prod: 3600
# HTML Cache time for other environments
other: 0
cacheControlMaxAgeHTML: ${self:custom.domain.cacheControlMaxAgeHTMLByStage.${self:custom.stage}, self:custom.domain.cacheControlMaxAgeHTMLByStage.other}
sslCertificateARN: arn:aws:acm:us-east-1:165657443288:certificate/61d202ea-12f2-4282-b602-9c3b83183c7a
assets:
targets:
# Configuration for HTML files (overriding the default cache control age)
- bucket:
Ref: WebsiteS3Bucket
acl: private
files:
- source: ./public/
headers:
CacheControl: max-age=${self:custom.domain.cacheControlMaxAgeHTML}
empty: true
globs:
- '**/*.html'
# Configuration for all assets
- bucket:
Ref: WebsiteS3Bucket
acl: private
files:
- source: ./public/
empty: true
globs:
- '**/*.js'
- '**/*.css'
- '**/*.jpg'
- '**/*.png'
- '**/*.gif'
scripts:
hooks:
# Run these commands when creating the deployment artifacts
package:createDeploymentArtifacts: >
hexo clean &&
hexo generate
# Run these commands after infrastructure changes have been completed
deploy:finalize: >
sls s3deploy -s ${self:custom.stage}
# AWS Region to S3 website hostname mapping
s3DNSName:
us-east-2: s3-website.us-east-2.amazonaws.com
us-east-1: s3-website-us-east-1.amazonaws.com
us-west-1: s3-website-us-west-1.amazonaws.com
us-west-2: s3-website-us-west-2.amazonaws.com
ap-south-1: s3-website.ap-south-1.amazonaws.com
ap-northeast-3: s3-website.ap-northeast-3.amazonaws.com
ap-northeast-2: s3-website.ap-northeast-2.amazonaws.com
ap-southeast-1: s3-website-ap-southeast-1.amazonaws.com
ap-southeast-2: s3-website-ap-southeast-2.amazonaws.com
ap-northeast-1: s3-website-ap-northeast-1.amazonaws.com
ca-central-1: s3-website.ca-central-1.amazonaws.com
eu-central-1: s3-website.eu-central-1.amazonaws.com
eu-west-1: s3-website-eu-west-1.amazonaws.com
eu-west-2: s3-website.eu-west-2.amazonaws.com
eu-west-3: s3-website.eu-west-3.amazonaws.com
eu-north-1: s3-website.eu-north-1.amazonaws.com
sa-east-1: s3-website-sa-east-1.amazonaws.com
# Determine what resources file to include based on the current stage
customConfigFile: ${self:custom.customConfigFiles.${self:custom.stage}, self:custom.customConfigFiles.other}
customConfigFiles:
prod: prod
other: other

# Define the Lambda functions for the site
functions:
# This function will be deployed to Lambda@Edge and rewrite URLs to include index.html
urlrewrite:
name: ${self:service}-${self:custom.stage}-cf-url-rewriter
handler: functions/urlRewrite.handler
memorySize: 128
timeout: 1
lambdaAtEdge:
distribution: WebsiteCloudFrontDistribution
eventType: origin-request

# Define the resources we will need to host the site
resources:
# Include the resources file
- ${file(config/resources.yml)}
# Include the outputs file
- ${file(config/outputs.yml)}
# Include a custom configuration file based on the environment
- ${file(config/resources/environment/${self:custom.customConfigFile}.yml)}

[config/resources.yml] []view raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
Resources:
# Set-up an S3 bucket to store the site
WebsiteS3Bucket:
Type: AWS::S3::Bucket
Properties:
AccessControl: BucketOwnerFullControl
BucketName: ${self:custom.domain.domainname}
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
# Set-up a policy on the bucket so it can be used as a website
WebsiteBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
PolicyDocument:
Id:
Fn::Join:
- ""
- - ${self:service.name}
- BucketPolicy
Statement:
- Sid: CloudFrontForGetBucketObjects
Effect: Allow
Principal:
CanonicalUser:
Fn::GetAtt:
- CloudFrontIdentity
- S3CanonicalUserId
Action: 's3:GetObject'
Resource:
Fn::Join:
- ''
-
- 'arn:aws:s3:::'
- Ref: WebsiteS3Bucket
- /*
Bucket:
Ref: WebsiteS3Bucket
# Configure CloudFront to get all content from S3
WebsiteCloudFrontDistribution:
Type: 'AWS::CloudFront::Distribution'
Properties:
DistributionConfig:
WebACLId:
Ref: CustomAuthorizationHeaderRestriction
Aliases:
- ${self:custom.domain.domainname}
- www.${self:custom.domain.domainname}
CustomErrorResponses:
- ErrorCode: '404'
ResponsePagePath: "/error.html"
ResponseCode: '200'
ErrorCachingMinTTL: '30'
DefaultCacheBehavior:
Compress: true
ForwardedValues:
QueryString: false
Cookies:
Forward: all
SmoothStreaming: false
TargetOriginId: defaultOrigin
ViewerProtocolPolicy: redirect-to-https
DefaultRootObject: index.html
Enabled: true
Origins:
- DomainName:
Fn::GetAtt:
- WebsiteS3Bucket
- DomainName
Id: defaultOrigin
S3OriginConfig:
OriginAccessIdentity:
Fn::Join:
- "/"
- - origin-access-identity
- cloudfront
- Ref: CloudFrontIdentity
PriceClass: PriceClass_All
ViewerCertificate:
AcmCertificateArn: ${self:custom.domain.sslCertificateARN}
SslSupportMethod: sni-only
# DNS Record for the domain
WebsiteDNSRecord:
Type: "AWS::Route53::RecordSet"
Properties:
AliasTarget:
DNSName:
Fn::GetAtt:
- WebsiteCloudFrontDistribution
- DomainName
HostedZoneId: Z2FDTNDATAQYW2
HostedZoneName: ${self:custom.domain.domain}.
Name: ${self:custom.domain.domainname}
Type: 'A'
# DNS Record for www.domain
WebsiteWWWDNSRecord:
Type: "AWS::Route53::RecordSet"
Properties:
AliasTarget:
DNSName:
Fn::GetAtt:
- WebsiteCloudFrontDistribution
- DomainName
HostedZoneId: Z2FDTNDATAQYW2
HostedZoneName: ${self:custom.domain.domain}.
Name: www.${self:custom.domain.domainname}
Type: 'A'
# Predicate to match the authorization header
CustomAuthorizationHeader:
Type: AWS::WAF::ByteMatchSet
Properties:
ByteMatchTuples:
-
FieldToMatch:
Type: HEADER
Data: Authorization
TargetString:
Fn::Join:
- " "
- - Custom
- "<Password>"
TextTransformation: NONE
PositionalConstraint: EXACTLY
Name:
Fn::Join:
- "_"
- - ${self:custom.domain.domainname}
- Authorization
- Header
CustomAuthorizationHeaderRule:
Type: AWS::WAF::Rule
Properties:
Name:
Fn::Join:
- "_"
- - ${self:custom.domain.domainname}
- Authorization
- Header
- Rule
MetricName:
Fn::Join:
- ""
- - ${self:custom.stage}
- ${self:service.name}
- Authorization
- Header
- Rule
Predicates:
-
DataId:
Ref: CustomAuthorizationHeader
Negated: false
Type: ByteMatch

[config/outputs.yml] []view raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Outputs:
WebsiteURL:
Value:
Fn::GetAtt:
- WebsiteS3Bucket
- WebsiteURL
Description: URL for my website hosted on S3
S3BucketSecureURL:
Value:
Fn::Join:
- ''
-
- 'https://'
- Fn::GetAtt:
- WebsiteS3Bucket
- DomainName
Description: Secure URL of S3 bucket to hold website content

[config/prod.yml] []view raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
Resources:
# Allow the custom authorisation header in the production environment
CustomAuthorizationHeaderRestriction:
Type: AWS::WAF::WebACL
Properties:
DefaultAction:
Type: ALLOW
Name:
Fn::Join:
- "_"
- - ${self:custom.domain.domainname}
- Authorization
- Header
- Restriction
MetricName:
Fn::Join:
- ""
- - ${self:custom.stage}
- ${self:service.name}
- Authorization
- Header
- Restriction
Rules:
-
Action:
Type: ALLOW
Priority: 1
RuleId:
Ref: CustomAuthorizationHeaderRule

[config/other.yml] []view raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Resources:
# Require the custom authorisation header with the correct password in non-production environment
CustomAuthorizationHeaderRestriction:
Type: AWS::WAF::WebACL
Properties:
DefaultAction:
Type: BLOCK
Name:
Fn::Join:
- "_"
- - ${self:custom.domain.domainname}
- Authorization
- Header
- Restriction
MetricName:
Fn::Join:
- ""
- - ${self:custom.stage}
- ${self:service.name}
- Authorization
- Header
- Restriction
Rules:
-
Action: ALLOW
Priority: 1
RuleId:
Ref: CustomAuthorizationHeaderRule

Example Site