NodeJS Course

by Douglas Collioni

Content table

Overview

Intro

What's Node.js?

  • Node.js is an open source server framework
  • Node.js is free
  • Node.js runs on various platforms (Windows, Linux, Unix, Mac OS X, etc.)
  • Node.js uses JavaScript on the server

Why Node.js?

  • Node.js uses asynchronous programming

Scenario: server must open a file and return the content to the client

PHP, Java, C# execution:

  1. Sends the task to the computer's file system
  2. Waits while the file system opens and reads the file
  3. Returns the content to the client
  4. Ready to handle the next request

Node.js execution:

  1. Sends the task to the computer's file system
  2. Ready to handle the next request
  3. When the file system has opened and read the file, the server returns the content to the client

What can Node.js do?

  • Generate dynamic page content
  • Create, open, read, write, delete, and close files on the server
  • Collect form data
  • Add, delete, modify data in your database

What is a Node.js file?

  • Node.js files contain tasks that will be executed on certain events
  • A typical event is someone trying to access a port on the server
  • Node.js files must be initiated on the server before having any effect
  • Node.js files have the extension ".js"

Installation

The official Node.js website has installation instructions for Node.js:

nodejs.org

Running

Node.js files must be initiated on a Command Line Interface program such as:

  • Command Prompt (cmd)
  • Git Bash
  • PowerShell
  • Terminal

Hello world

To get started, create a folder for your code:

mkdir ~/dev/nodejs

Navigate to the folder:

cd ~/dev/nodejs

Initiate Visual Studio Code in this folder:

code .

Create a new file named "hello.js":

touch hello.js

Write the following "hello world" JavaScript code in it:

console.log('Hello world!')

Run the code using node:

node hello.js

"Hello world!" should be printed on the console.

Running Server

Create a new file named "server.js" with the following code:

const http = require('http')
http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/plain' })
  res.end('Hello world!')
}).listen(8080)

Run the server using node:

node server.js

This command is going to start a server on the localhost, port 8080.

Access the server on localhost:8080 and check the "Hello world!" text is displayed.

Don't worry about the code syntax in this example for now. We will go through it in detail later on in the course.

Modules

What is a Module in Node.js?

  • Modules can be considered the same as JavaScript libraries
  • A set of functions you want to include in your application

Built-in Modules

Node.js has a set of built-in modules which you can use without any further installation.

Look at the Built-in Modules Reference for a complete list of modules.


Include Modules

To include a module in your application, use the require() function with the name of the module:

const http = require('http')

Now the application has access to the HTTP module:

http.createServer((req, res) => { // ...

Create your own modules

Custom modules can be created and easily included into the application.

Create a file named "myDateTime.js" that returns the current date and time:

exports.myDateTime = () => {
  return Date()
}

Now you can include and use the module in any of your Node.js files using require():

const dt = require('./myDateTime')
console.log('myDateTime:', dt.myDateTime())

Notice that we use ./ to locate the module, that means that the module is located in the same folder as the Node.js file.


Try it yourself

Create a module in a new file named "myName.js" containing a function that returns your name as a string. After that, include this module in your application code and print your name using the console.

HTTP

Node.js has a built-in module called HTTP, which allows Node.js to transfer data over the Hyper Text Transfer Protocol (HTTP).

const http = require('http')

http.createServer((req, res) => { // create a server object
  res.write('Hello World!')       // write a response to the client
  res.end()                       // end the response
}).listen(8080)                   // the server object listens on port 8080

If the response from the HTTP server is supposed to be displayed as HTML, you should include an HTTP header with the correct content type:

const http = require('http')

http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/html' })
  res.write('Hello World!')
  res.end()
}).listen(8080)

The function passed into the http.createServer() has a req argument that represents the request from the client.

This object has a property called url which holds the part of the URL that comes after the domain name:

const http = require('http')

http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/html' })
  res.write(req.url)
  res.end()
}).listen(8080)

There are built-in modules to easily split the URL query string into readable parts, such as the URL module:

const http = require('http')
const url = require('url')

http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/html' })
  const q = url.parse(req.url, true).query
  const txt = q.year + " " + q.month
  res.end(txt)
}).listen(8080)

Run the code above using node, then access your running server passing year and month as query strings:

localhost:8080?year=2022&month=october

The year and month should be displayed on the page.


Try it yourself

Modify the previous server to also display your name using the "myName" module created earlier.

Additionaly, modify the server to also read the day from the query string and display it with the year and month.

FS (File System)

The fs module allows you to work with the file system on your computer using node:

const fs = require('fs')

Common uses for the File System module are:

  • Read files
  • Create files
  • Update files
  • Delete files
  • Rename files

The fs.readFile() method is used to read files from your computer:

const http = require('http')
const fs = require('fs')

http.createServer((req, res) => {
  fs.readFile('demo.html', (err, data) => {
    res.writeHead(200, { 'Content-Type': 'text/html' })
    res.write(data)
    res.end()
  })
}).listen(8080)

In the example above, a file named "demo.html" is being read from the local disk and written as a HTML response to the client.


Try it yourself

Create a HTTP server to read the pathname from the URL and read a file from the disk with that name. Then return the file contents as HTML to the client. Example:

NPM

Intro

What is NPM?

  • NPM stands for Node Package Manager, it is a tool used to install packages on your Node application
  • npmjs.com hosts thousands of free packages to download and use
  • NPM tool is installed on your computer when you install Node.js

For more information, visit npm Docs.


What is a package?

A package in Node.js contains all the files you need for a module.

Modules are JavaScript libraries you can add to your project. Examples:

  • https
  • fs
  • url
  • express
  • mongodb
  • mocha

Download a Package

Downloading a package using NPM is very easy. Open the command line interface and tell NPM to download the package:

npm install upper-case

This command is going to download and install the "upper-case" package.

NPM creates a folder named "node_modules" where the package will be installed. All packages installed in the future will be placed in this folder.

Use a Downloaded Package

Once the package is installed, it is ready to use.

Include the "upper-case" package the same way you would include any other module:

const upperCase = require('upper-case')

const text = upperCase('hello world!')
console.log(text)

In the example above, the upperCase function is transforming all letters in the string to upper case.

When running this code, the console should display "HELLO WORLD!"

Package.json

It is a good practice to initialise your application using NPM. You can do that using:

npm init

You can accept the default values of the wizard.

This command will generate a "package.json" file in your application folder.

The "package.json" file holds some important information about your application such as name, version, dependencies, scripts, etc.


Saving dependencies

As soon as your application is initialised and contains a "package.json" file, you can save the installed dependencies:

npm install upper-case --save

Modules installed using the option --save will be saved in your "package.json" file in the "dependencies" section:

"dependencies": {
  "upper-case": "^2.0.2"
}

It is very important to keep the dependencies saved so other developers can clone your repository and have the same set of modules installed using npm install.

The "node_modules" folder in your application should be git ignored. Developers cloning your repo should be able to start woking with it by simply installing the dependencies declared in the "package.json" file.


Updating and removing dependencies

Dependencies can also be updated and removed using NPM:

npm update upper-case --save
npm remove upper-case --save

Scripts

In the package.json file you can declare scripts that can be executed for your project. Example:

"scripts": {
  "test": "echo \"Error: no test specified\" && exit 1",
  "start": "node index.js"
}

Scripts can be run using NPM:

npm run start

We will explore more examples of scripts later on in this course.

Express

Intro

Express is a fast and minimalist web framework for Node.js.

Express can be installed using NPM:

npm install express --save

Express offers a number of HTTP utility methods and middlewares, making the creation of a robust API quick and easy.

For more information, visit expressjs.com.


Hello world

Create a folder for your Express demo application:

mkdir express-demo

Go into the folder:

cd express-demo

Initialise NPM:

npm init

Install Express:

npm install express --save

Create an entry file:

touch index.js

Update the "index.js" file with the following code:

const express = require('express')
const app = express()
  
app.get('/', (req, res) => res.send('Hello world!'))

app.listen(3000, () => console.log('app listening on port 3000!'))

Run the server:

node index.js

This example is creating an Express server which runs on localhost:3000/ and renders a "Hello world!" text on the root ("/") endpoint.

Express abstracts the built-in Node.js http module, so that it is much easier to map routes and responses to the API clients.

App Generator

Express offers an application generator tool which quickly creates an app skeleton.

For more information, visit Express application generator.

Basic Routing

Routing refers to defining how an application responds to a client request to a particular endpoint, which is a URI (or path) and a specific HTTP request method such as:

  • GET
  • POST
  • PUT
  • DELETE, etc

Each route can have one or more handler functions, which are executed when the route is matched.

The syntax of a route definition using Express looks like the following:

app.METHOD('PATH', HANDLER)

Examples:

app.get('/', (req, res) => {
  res.send('Hello World!')
})

app.post('/', (req, res) => {
  res.send('Got a POST request')
})

app.put('/user', (req, res) => {
  res.send('Got a PUT request at /user')
})

app.delete('/user', (req, res) => {
  res.send('Got a DELETE request at /user')
})

Static Files

To serve static files such as images, CSS files, and JavaScript files, use the express.static built-in middleware function from Express.

For example, the following code serves images, CSS files, and JavaScript files from a directory named "public":

app.use(express.static('public'))

Now, you could access static files that are in the public directory:

Note that if the file does not exist, you will receive a 404 (Not Found) message from the server.

Advanced Routing

All routes

There is a special routing method, app.all(), used to load middleware functions at a path for all HTTP request methods.

app.all('/secret', (req, res, next) => {
  console.log('Accessing the secret section ...')
  next() // pass control to the next handler
})

In the example above, all HTTP methods reaching the "/secret" endpoint would be handled by that method.

Route parameters

Route parameters are named URL segments that are used to capture the values specified at their position in the URL.

The captured values are populated in the req.params object, with the name of the route parameter specified in the path as their respective keys.

app.get('/users/:userId/books/:bookId', (req, res) => {
  res.send(req.params)
})

The route above can be accessed at localhost:3000/users/101/books/202 and it will respond the params object containing the userId and the bookId properties.

Multiple handlers

More than one callback function can handle a route (make sure you specify the next object). For example:

app.get('/example/b', (req, res, next) => {
  console.log('response will be sent by the next function')
  next()
}, (req, res) => {
  res.send('Hello from B!')
})

Access the route above at localhost:3000/example/b to check the response and the console logs.

Response methods

Express provides different methods to respond to a request:

res.download() Prompts a file to be downloaded
res.end() Ends the response process
res.json() Sends a JSON response
res.jsonp() Sends a JSON response with JSONP support
res.redirect() Redirects the request to another route
res.render() Renders a view template
res.send() Sends a response of various types
res.sendFile() Sends a file as an octet stream
res.sendStatus() Sets the response status code and send its string as body

Chainable handlers

Route handlers can be chained for a route path using the app.route function:

app.route('/book')
  .get((req, res) => { res.send('Get a book') })
  .post((req, res) => { res.send('Add a book') })
  .put((req, res) => { res.send('Update a book') })
  .delete((req, res) => { res.send('Delete a book') })

Express router

The Express router can be used to create modular, mountable route handlers:

const express = require('express')
const app = express()
const router = express.Router()
  
router.get('/', (req, res) => {
  res.send('Birds home page')
})
  
router.get('/about', (req, res) => {
  res.send('About birds')
})

app.use('/birds', router)

Try it out:

Middlewares

Middleware functions are functions that have access to the request object (req), the response object (res), and the next function in the application's request-response cycle.

Middleware functions can perform the following tasks:

  • Execute any code
  • Make changes to the request and the response objects
  • End the request-response cycle
  • Call the next middleware in the stack
const requestTime = (req, res, next) => {
  req.time = new Date()
  next()
}
  
app.use(requestTime)
  
app.get('/', (req, res) => {
  const { time } = req
  res.send(`Hello, World! ${time}`)
})

In the example above, the requestTime function is a middleware which is being added to all routes by the app. When any route is called, the request object is being updated with a property time receiving the current timestamp. The time of the request is then accessible to any route handlers that follow next.

Body Parser

Body Parser is a useful middleware module to parse the incoming request bodies before the route handlers. The parsed body is available under the req.body property.

The Body Parser module can be insalled using NPM:

npm install body-parser --save

Example of use:

const bodyParser = require('body-parser')
app.use(bodyParser.json())

app.post('/login', (req, res) => {
  const { user, password } = req.body
  
  if (user !== 'root' || password !== '1234') {
    res.sendStatus(403)
  } else {
    res.sendStatus(200)
  }
})

The code example above is adding a generic JSON parser as a top-level middleware, which will parse the bodies of all incoming requests.

Try it yourself

Create a HTTP server using Express with a middleware to validate two parameters sent via querystring: user and password.

The access to the "/admin" route must be blocked in case the credentials are different than "admin/1234"

MongoDB

Intro

The official MongoDB NodeJS driver provides both callback-based and Promise-based interaction with MongoDB.

Given that you have created your own project using npm init we install the mongodb driver and it's dependencies by executing:

npm install mongodb --save

Connection

The following is an example of how to open a connection with a MongoDB database:

const { MongoClient } = require('mongodb')
const url = 'mongodb://localhost:27017'
const dbName = 'mydb'

MongoClient.connect(url, (err, client) => {
  console.log('Connected successfully to server')
  const db = client.db(dbName) // use the db object to perform operations with the database
  client.close()
})

Usage

Insert a document

The following is an example of how to insert a document in a MongoDB collection:

const collection = db.collection('users')

const { result } = await collection.insertOne({
  name: 'John Doe',
  email: 'john.doe@gmail.com'
})

console.log(result)

Insert many documents

The following is an example of how to insert multiple documents in a MongoDB collection:

const collection = db.collection('countries')

const { result } = await collection.insertMany([
  { name: 'Brazil', code: 'BR' },
  { name: 'Italy', code: 'IT' }
])

console.log(result)

Find documents

The following is an example of how to find all documents from a MongoDB collection:

const collection = db.collection('users')

const users = await collection.find({}).toArray()
  
console.log(users)

Find documents with a query

The following is an example of how to find documents from a MongoDB collection with a filter:

const collection = db.collection('countries')

const query = { code: 'BR' }
const countries = await collection.find(query).toArray()

console.log(countries)

Update a document

The following is an example of how to update a document in a MongoDB collection:

const collection = db.collection('countries')

const query = { code: 'IT' }
  
const { result } = await collection.updateOne(query, {
  $set: {
    name: 'Itália'
  }
})

console.log(result)

Delete a document

The following is an example of how to delete a document from a MongoDB collection:

const collection = db.collection('users')

const query = { email: 'john.doe@gmail.com' }
const { result } = await collection.deleteOne(query)
  
console.log(result)

Map MongoDB collections

The following is an example of how to map the collections of a MongoDB database and return them as asynchronous functions to be used somewhere else:

const { MongoClient } = require('mongodb')
const url = 'mongodb://localhost:27017'
const dbName = 'mydb'
  
const db = async () => (await MongoClient.connect(url)).db(dbName)
  
const users = async () => (await db()).collection('users')
const countries = async () => (await db()).collection('countries')
  
module.exports = { users, countries }

The collections can then be used in different modules by importing the one you need:

const { users } = require('./mongo')

const run = async () => {
  const collection = await users()
  const data = await collection.find({}).toArray()
  console.log(data)
}
  
run()

Mongoose

Intro

Mongoose is an elegant MongoDB object modeling for NodeJS.

Mongoose provides a straight-forward, schema-based solution to model your application data. It includes built-in type casting, validation, query building, business logic hooks and more, out of the box.

Install it using NPM:

npm install mongoose --save

There is no need to install the mongodb package separately because the mongoose package already includes it as a dependency.

Connection

The following is an example of how to open a connection using Mongoose:

const mongoose = require('mongoose')
const url = 'mongodb://localhost:27017/mydb'
const connection = mongoose.connect(url)

Schema and model

The following is an example of how to define a schema and a model:

const countrySchema = mongoose.Schema({
  name: String,
  code: String
})
  
const Country = mongoose.model('Country', countrySchema)

const france = new Country({ name: 'France', code: 'FR' })

Insert a document

The following is an example of how to insert a document using the defined model:

const france = new Country({ name: 'France', code: 'FR' })
const result = await france.save()
console.log(result)

Find documents

The following is an example of how to find documents using the defined model:

const countries = await Country.find({})
console.log(countries)
                  
const brazil = await Country.findOne({ code: 'BR' })
console.log(brazil)

Update a document

The following is an example of how to update a document using the defined model:

const brazil = await Country.findOne({ code: 'BR' })
brazil.name = 'Brasil'
await brazil.save()

Delete a document

The following is an example of how to delete a document using the defined model:

const italy = await Country.findOne({ code: 'IT' })
const result = await italy.remove()
console.log(result)

Schema validators

const breakfastSchema = mongoose.Schema({
  eggs: {
    type: Number, min: [6, 'Too few eggs'], max: 12
  },
  bacon: {
    type: Number, required: [true, 'Why no bacon?']
  },
  drink: {
    type: String, enum: ['Coffee', 'Tea'],
    required: function() { return this.bacon > 3; }
  }
})

const Breakfast = mongoose.model('Breakfast', breakfastSchema)

const bf = new Breakfast({ eggs: 4, bacon: 2 })

try {
  await bf.save()
} catch (err) {
  console.log(err)
}

console.log(bf)

Try it yourself

Create a HTTP server using Express with routes to insert, update, find and delete countries from a MongoDB database.

Test all routes using Postman.

Try complementing this exercise creating HTML pages to access these routes and make the CRUD operations with countries.

Unit Tests

Intro

Mocha is a feature-rich JavaScript test framework running on NodeJS and in the browser, making asynchronous testing simple and fun.

Mocha tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases.

NPM can be used to install the Mocha package:

npm install mocha --save-dev

Note we are using the flag --save-dev in this case. That is because Mocha is not necessary for the production app to run, only in development time to run the unit tests.

Assertions

Unit tests are written based on assertions. It is a good practice to describe the object and the function being tested. The assertions will then be contained in the description. For example:

const assert = require('assert')

describe('Array', () => {
  describe('indexOf()', () => {
    it('should return -1 when the value is not present', () => {
      const array = [1, 2, 3]
      const actual = array.indexOf(4)
      const expected = -1
      assert.equal(actual, expected)
    })
  })
})

In the example above, we are expecting that the method indexOf of the array object will return -1 when the value is not present in the array. We do this by using the assert.equal function from the NodeJS assert built-in module.

The describe and it are functions imported from the Mocha package which, when installed, register such functions as NodeJS globals, hence there is no need to require them explicitly.

The test above can be executed by running:

./node_modules/mocha/bin/mocha

The result in the console will look something like:

Array
  #indexOf()
    √ should return -1 when the value is not present
1 passing (7ms)

Scripts

It is a good practice that the tests are set up in your package.json scripts, for example:

"scripts": {
  "test": "mocha"
}

To be able to call mocha like in the script above, you have to install it globally:

npm install mocha -g

This way you can run the same tests using:

npm run test

Database Testing

Sometimes the tests need some preset data to run. Sometimes there is also a need for deleting test data after running the unit tests. This can be achieved by using before and after functions from Mocha:

before(async () => {
  await db.clear()
  await db.save([tobi, loki, jane])
})
  
describe('#findUsers()', () => {
  it('responds with matching records', async () => {
    const users = await db.find({ type: 'User' })
    users.should.have.length(3)
  })
})

In the example above, before the tests run, 3 users are created in the database. Then the test is listing the users from the database and checking there is 3 items found.

Ideally, there would be an after function to do the clean up of the test data as well.


Try it yourself

Create a test to assert that the function that finds countries in the database always returns the correct number of items.