Beginning Node and Express

Ryan Dahl's little experiment is still going strong. 💪

Node isn’t a programming language or framework; instead, it’s a JavaScript runtime or an environment that allows us to execute JavaScript code outside of the browser.

A “runtime” converts code written in a high-level, human-readable, programming language and compiles it down to code the computer can execute.

Companies including Netflix, Uber, Trello, PayPal, LinkedIn, eBay, NASA, and Medium have adopted Node, and most professional developers will have encountered Node tools.

Node was initially developed by Ryan Dahl. He took the V8 JavaScript engine from Google’s Chrome browser, added some APIs, wrapped it in an event loop, and launched it as an open-source product on Linux and macOS in 2009. The Windows edition arrived in 2011. The first (non-beta) release of Node.js arrived in 2015.

Uses for Node

Though Node was created with the goal of building web servers and web applications in JavaScript, it can also be used for creating command-line applications or desktop applications.

Node can be combined with any number of robust frameworks like Express.js for creating effective web application back-ends.

Node Execution Model

Unlike the traditional web-serving technique, where each request creates a new thread cramping up the system RAM, Node operates on a single thread. This enables it to support thousands of concurrent connection handling event loops.

A thread is an allocated bundle of computer resources used to run a series of instructions in a task. Usually, the tasks handled by threads are simple and fast. For this reason, the Node.js event loop needs only one thread to act as the manager of all other tasks.

Node is also event-driven, which means that everything that happens in Node is in reaction to an event.

For example, when a new request comes in (one kind of event) the server will start processing it. If it then encounters a blocking I/O operation (reading/writing to files, etc.) instead of waiting for this to complete, it will register a callback before continuing to process the next event. When the I/O operation has finished (another kind of event), the server will execute the callback and continue working on the original request.

Under the hood, Node uses the Libuv library to implement this asynchronous (that is, non-blocking) behavior.

Node is designed to pass larger tasks to the host computer, and the computer may create new threads and processes to operate those tasks beyond the single thread that it uses to manage all of it’s tasks.

The event loop handles a series of tasks, always working on one task at a time and using the computer’s processing power to offload some larger tasks while the event loop shortens the list of tasks.

The Node event loop cycles forever in a loop, listening for JavaScript events triggered by the server to notify of some new task or another task’s completion.

Node Downsides?

Node is very fast compared to other dynamic languages thanks to the V8 JIT compiler. However, if you are looking for a platform and language that can squeeze the most performance out of your computing resources, Node is not the best.

CPU-bound workloads can typically benefit from using a lower-level language like C, C++, Go, Java, or Rust. If you have a specialized task that is particularly sensitive to performance, consider using a lower-level language.

Other lesser-known functional languages such as Haskell and Elixir are currently used in tech startups where performance is of the highest priority.

Since Node applications run on a single processing thread there can be some added complexity for building scalable applications:

• If your app fails, it fails for everyone and won’t restart unless you have appropriate monitoring in place.
• Node.js still runs on a single CPU core even when that CPU can have 15 more at its disposal. Solutions such as clustering, caching using node-cache or redis, PM2, and Docker containers can help.
• Node.js web servers are not efficient at serving static files such as images, stylesheets, and client-side JavaScript. Production sites often use a front-end NGINX web server to serve static files or direct the request to the Node.js application when appropriate.

What is the Back-end?

In order to deliver the front-end of a website or web application to a user, a lot needs to happen behind the scenes on the back-end.

• The front-end of a website or app consists of HTML, CSS, JavaScript, and static assets sent to a client, like a web browser or mobile device.
• A web server is a process running on a computer somewhere that listens for incoming requests for information over the internet and sends back responses.
• Storing, accessing, and manipulating data is a large part of a web application’s back-end. Data is stored in databases which can be relational databases or NoSQL databases.
• The server-side of a web application, sometimes called the application server, handles important tasks such as authorization and authentication.
• The back-end of a web application often has a web API which is a way of interacting with an application’s data through HTTP requests and responses.

What is an API?

When a user navigates to a specific item for sale on an e-commerce site, the price listed for that item is stored in a database, and when they purchase it, the database will need to be updated with the correct inventory for that item type. In fact, much of what the back-end entails is reading, updating, or deleting information stored in a database.

In order to have consistent ways of interacting with data, a back-end will often include a web API.

API stands for Application Program Interface and can mean a lot of different things, but a web API is a collection of predefined ways of, or rules for, interacting with a web application’s data, often through an HTTP request-response cycle.

Unlike the HTTP requests a client makes when a user navigates to a website’s URL, this type of request indicates how it would like to interact with a web application’s data (create new data, read existing data, update existing data, or delete existing data), and it receives some data back as a response. This is known as a CRUD application.

Some web APIs are open to the public. Instagram, for example, has an API that other developers can use to access some of the data Instagram stores. Others are only used by the web application internally.

HTTP: The Basics

The World Wide Web is a web of documents known as web pages. Each web page consists of objects, which could be text, graphics, links to other web pages, or scripts. Each object has a ‘URL,’ or Uniform Resource Locator,

HTTP or HyperText Transfer Protocol is the protocol at the core of the web. It’s a client-server protocol that specifies how Web clients request Web pages from Web servers and how Web servers send them.

Servers and clients are processes on devices that communicate with each other through messages. These messages are based on the rules defined by HTTP.

The Anatomy of a URL

A URL, or Universal Resource Locator, is to locate files that exist on servers.

https://www.espn.com/nfl/scoreboard/index.html?week=1

URLs consist of the following parts:

• Protocol in use (https)
• The hostname of the server (www.espn.com)
• The location of the file (nfl/scoreboard/index.html)
• Arguments to the file (week=1)

Structure of a Request

There are four main elements in a request.

• Operation: HTTP methods verbs (e.g. GET, POST, PUT and DELETE).
• Endpoint: This is your REST API endpoint.
• Parameters/body: This is some data that you might send in the request.
• Headers: This is a special part of a REST API request that might contain things like an API key or some authentication data.

HTTP Methods

HTTP methods tell the server what to do. There are a lot of HTTP methods, but the most common ones: GET, POST, PUT, and DELETE.

• GET is the most common method. It requests data.
• POST is used to send data to a server to create or update a resource.
• PUT uploads an enclosed entity under a supplied URI. In other words, it puts data in a specific location.
• DELETE deletes an object at a given URL.

Ice Cream Shop 🍦

What would a REST API request look like? You could have a GET operation because you want to get ice cream flavors, and the endpoint could be /api/flavors.

icecream.com/api/flavors

The api in the endpoint signifies the API portion of the endpoint.

flavors is the actual resource. This signifies that we are working with the flavors resource in this REST API.

Scenario 1: GET
We want to display what is currently in stock. What would our REST API request look like?

You have a GET as the operation because you want to get those flavors.

The endpoint is /api/flavors. In response, you will get an array of those flavor resources. We see strawberries and mint chocolate in stock.

Scenario 2: PUT
We want to update or replace that mint chocolate with just chocolate. What would our request look like?

It is a PUT operation, which updates or replaces a flavor.

The endpoint is /api/flavors/1 to indicate the ID of 1 you want to replace. In the parameter body, you specify the flavor of chocolate, which is the new flavor that has replaced mint chocolate. In response, we see the ID of 1 is replaced with the flavor of chocolate.

Scenario 3: POST
We want to create a new flavor. How do we do that with the REST API?

It is a POST operation, which will create a new flavor.

The endpoint is once again /api/flavors, and we included the flavor that we wanted to create (restful raspberry). In response, we see a new ID of 2 was created, and the flavor is restful raspberry.

Structure of a Response

The structure of a response is in the form of JSON data, but a REST API can also return XML YAML, etc.

A typical response message has three parts: an initial status line, some header lines, and an entity body.

REST API

REST, or REpresentational State Transfer, is an architectural style for providing standards between computer systems on the web, making it easier for systems to communicate with each other. REST-compliant systems, often called RESTful systems, are characterized by how they are stateless and separate the concerns of client and server.

The first thing that you should really know about REST APIs is that they are all about communication. You may have heard the term restful. So, a restful web service is a service that uses REST APIs to communicate.

Additionally, because of being stateless, you don’t have to worry about what data is in which state or keep track of that across client and server. It’s truly stateless.

Separating the user interface concerns from the data storage concerns, we improve the flexibility of the interface across platforms and improve scalability by simplifying the server components. Additionally, the separation allows each component the ability to evolve independently.

Statelessness means that every HTTP request happens in complete isolation. When the client makes an HTTP request, it includes all information necessary for the server to fulfill the request. The server never relies on information from previous requests from the client.

API Versioning

API versioning is the practice of transparently managing changes to your API.

Versioning is effective communication around changes to your API, so consumers know what to expect from it. You are delivering data to the public in some fashion and you need to communicate when you change the way that data is delivered.

With APIs, something as simple as changing a property name from productId to productID can break things for consumers.

What constitutes a “breaking change” in an API endpoint? Any change to your API contract that forces the consumer to also make a change.

There are several methods for managing the version of your API. URI path versioning is the most common.

http://www.example.com/api/v1/products
http://api.example.com/v1/products

This strategy involves putting the version number in the path of the URI, and is often done with the prefix “v”. More often than not, API designers use it to refer to their application version rather than the endpoint version.

URI path versioning implies orchestrated releases of application versions that will require one of two approaches: maintaining one version while developing a new one or forcing consumers to wait for new resources until the new version is released. It also means you’d need to carry over any non-changed endpoints from version to version.

Other approaches:

Query params:

This type of versioning adds a query param to the request that indicates the version.

http://www.example.com/api/products?version=1

Header:

Accept: version=1.0

Installing and Running Node

The simplest way to install Node.js is to go to the download link at https://nodejs.org/en/download and follow the instructions and prompts to download the installer for the latest version of Node.js.

The installer also carries the Node.js package manager (npm) within it. It means you don’t need to install the npm separately.

Even-numbered Node.js versions — such as 16, 18, and 20 — focus on stability and security with long-term support (LTS). Updates are provided for at least two years, so I recommend them for live production servers. You should install an identical version on your development machine.

Odd-numbered versions — such as 15, 17, and 19 — are under active development and may have experimental features. They’re fine for development if you’re learning, experimenting, or upgrading frequently.

Create Your First Node Project

Run npm init to initialize a new Node.js project. npm will prompt you for values, but you can hit Enter to accept the defaults (just creates a default package.json file in your root directory).

The package.json file provides a single place to configure your application. It contains the name, the version, the main entry/starting script, useful application scripts, configuration data, and module dependencies.

Most Node.js projects use semantic versioning, with three MAJOR.MINOR.PATCH numbers such as 1.2.33. When a change occurs, you increment the appropriate number and zero those that follow:

• MAJOR for major updates with incompatible API changes
• MINOR for new functionality that doesn’t affect backward compatibility
• PATCH for bug fixes

Switch to ES6 Modules

If you want to have your Node application use ES6 modules like an application built for the browser you can add “type”: “module”, to package.json

{
  "name": "node",
  "version": "1.0.0",
  "description": "Example Node app",
  "type": "module",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
  },
  "author": "",
  "license": "ISC"
}

ES6 modules are identical to those used in web browsers. Node.js uses CommonJS by default, but ES6 support arrived in version 13. ES6 modules will become predominant over time.

CommonJS:

const package = require('module-name');

ES6 Modules:

import package from 'module-name';

Node Server Application

A first quick example of a native Node.js application:

import * as http from 'http';
import {StatusCodes} from 'http-status-codes';
const port = process.env.PORT || 3000;
const app = http.createServer((request, response) => {
    console.log("Received an incoming request!");
    response.writeHead(StatusCodes.OK, { "Content-Type": "text/html"
    });
    let responseMessage = "<h1>Hello, World</h1>";
    response.write(responseMessage);
    response.end();
    console.log(`Sent a response : ${responseMessage}`);
});
app.listen(port, () => {
   console.log(`The server has started and is listening on port
                number:${port}`)
});

Modules in Node.js are libraries of code that come packaged to offer specific functionality to your application. Here, the http module helps you communicate across the web by using HTTP.

For extra information on HTTP status codes, we can use a package called
http-status-codes.

Status Codes

Each RESTful API operation is a combination of an HTTP request against a URL and an appropriate HTTP method.

When executed, each operation will return a status code, indicating whether the invocation has been successful or not. Successful invocation is indicated by an HTTP 2xx status code, while operations that are not executed correctly indicate this with erroneous status code — 4xx if the error is at the client side, or 5xx when the server fails to process a valid request.

100–199: Informational response; unlikely will return this range
200–299: Success; the request was received, understood and processed
300–399: Redirection; a resource at a different url was substituted
400–499: Client error; problem in how client submitted request
500–599: Server error: request accepted, but server error prevented fullfillment

The createServer function generates a new instance of http.Server, a built-in Node.js class with tools for evaluating HTTP communication. With this newly created server instance, your app is prepared to receive HTTP requests and send HTTP responses.

The argument in createServer is a callback function that’s invoked whenever some event occurs within the server. When the server is running and your application’s root URL (home page) is accessed, for example, an HTTP request event triggers this callback and allows you to run some custom code. In this case, the server returns a simple HTML response.

You log that a request was received from the client and use the response parameter in the callback function to send content back to the user, from whom you first received a request.

The first line uses a writeHead method to define some basic properties of the response’s HTTP header. HTTP headers contain information fields describing the content being transferred in a request or response. Header fields may contain dates, tokens, information about the origins of the request and response, and data describing the type of connection.

The response object is used by Node.js and carried throughout the application as a way to pass information about the current client transaction from function to function. Some methods on the response object allow you to add data to or remove data from the object; writeHead and write are two such functions.

Process.env is a global variable injected by Node.js at runtime for your application to use, and it shows the state of the system environment your app is in when it starts. When we write the code for our node application, we can never know where our app can be deployed.

Type node in the Node REPL (Node’s interactive shell) and type process.envto see the current environment variables on your computer. Since there is no port the hard-coded value will be used.

To make starting this app a little easier, edit package.json and change the “scripts” object to this:

"scripts": {
    "start": "nodemon index.js"
  },

If you don’t have nodemon (short for node monitor) installed, you can install it globally with npm install nodemon -g. If you’d rather use node directly, use “node index.js” as your “start” script (but you’ll need to stop and restart your app every time you want to test a change).

nodemon is a tool that automatically restarts the node application when file changes in the directory are detected.

Start the application with npm start and browse to http://localhost:3000.

Node and Callbacks

Part of what makes Node.js so fast and efficient is its use of callbacks. Callbacks aren’t new to JavaScript, but they’re overwhelmingly used throughout Node.js.

A callback is an anonymous function (a function without a name) that’s set up to be invoked as soon as another function completes. The benefit of using callbacks is that you don’t have to wait for the original function to complete processing before running other code.

In the web server example above, incoming requests from the client are received on a rolling basis and thereupon pass the request and response as JavaScript objects to a call-back function, as shown in the following figure:

Using Express

As we’ve seen, Node.js comes with a number of built-in modules, one of which is http which allows you to build an HTTP server that responds to HTTP requests from browsers. In short, the http module lets you build websites with Node.

Although you can build full web servers with nothing but Node’s built-in http module, most codebases will use a framework. The API exposed by the http module is pretty minimal and doesn’t do a lot of heavy lifting for you.

Express is a Node web server framework that promotes itself as “fast, unopinionated, and minimalist”. Express can be thought of as an abstraction layer on top of Node’s built-in HTTP server.

Express Server

Create a new project folder and install Express using npm:

npm install express

After installing express you will see 2 new files installed:

• a new package-lock.json file for npm internal use, which lists all the installed modules.
• a new node_modules folder, which contains the Express module and all submodules code (around 2MB of files)

A module such as Express is required for your application to run. It’s a dependency.

You can also install development dependencies, which typically are build tools or testing libraries that are only required on your development PC.

Create the Express Entry Script

You can now write code that uses Express to create a web application. Create a new index.js file in the root directory with the following code:

// Express application
import express from 'express';
// configuration
const
  cfg = {
    port: process.env.PORT || 3000
  };
// Express initiation
const app = express();
// home page route
app.get('/', (req, res) => {
  res.status(200).send('<h1>Hello World!</h1>');
});
// start server
app.listen(cfg.port, () => {
  console.log(`Example app listening at http://localhost:${ cfg.port }`);
});

Start the application with npm start and browse to http://localhost:3000.

The script imports the express module and creates an instance named app.

What is Routing?

Once the Express server is listening, it can respond to any and all requests. But how does it know what to do with these requests? To tell our server how to deal with any given request, we register a series of routes.

Routes determine how to handle requests based on path and HTTP verb.

Routing determines which functions Express executes when it receives a request for a specific URL, such as / or /another/path/.

A routing function is passed these two objects:

• An Express HTTP Request object (req), which contains details about the browser’s request.
• An Express HTTP Response object (res), which provides methods used to return a response to the browser. It sends “Hello, World!” text.

Try adding another routing function below the / handler to handle HTTP GET requests to /hello/:

// another route
app.get('/hello/', (req, res) => {
  res.status(200).send('Hello again!');
});

Entering a different URL path in the browser — such as http://localhost:3000/abc — returns Cannot GET /abc.

We will add a 404 route handler to give a customized message to our visitors instead of this generic error.

Routing is central to Express, and the framework provides options for parsing and responding to different URLs.

Query Parameters

A query parameter are optional key-value pairs that we can include after a quotation mark at the end of a URL to filter the resources that are being requested from the server.

The query string portion of a URL is the part of the URL after the question mark ?

The URL below has a catroute with a query string of name=fluffy. The query parameter key is name and the value is fluffy. 🐈

localhost:3000/cats?name=fluffy

Each key=value pair is called a query parameter. If your query string has multiple query parameters, they're separated by &.

For example, the below string has 2 query parameters, name and age.

?name=fuffy&age=1

Express automatically parses query parameters for you and stores them on the request object as req.query.

app.get('/cats', (req, res) => {
    const name = req.query.name;
    const age = req.query.age;
    res.send(`<h2>My cat ${name} is ${age} years old</h2>`);
});

If a query parameter appears multiple times in the query string, Express will group the values into an array.

app.get('/colors', (req, res) => {
   res.json(req.query);
});

Perilous parsing of query strings

You’ll always want to make sure that your users are giving you the data you expect, and if they aren’t, you’ll need to do something about it. One simple option is to provide a default case, so if they don’t give anything, assume the query is empty.

app.get("/search", (req, res) => {
   let search = req.query.q || "";
   let terms = search.split("+");
   // ... do something with the terms
});

This fixes one important bug: if you’re expecting a query string that isn’t there, you won’t have undefined variables.

But there’s another important gotcha with Express’s parsing of query strings: they can also be of the wrong type (but still be defined)!

If a user visits /search?q=abc, then req.query.q will be a string. It’ll still be a string if they visit /search?q=abc&name=douglas. But if they specify the q variable twice, like this:

/search?q=abc&q=xyz

then req.query.q will be the array [“abc”, “xyz”]. Now, if you try to call .replace on it, it’ll fail again because that method isn’t defined on arrays.

You could just write a function that will check if a value is an array. If it isn’t already an array, it wraps it in an array. If the argument is an array, it returns the argument because it is already an array.

const arrayWrap = (value) => {
   let isArray = Array.isArray(value);
   if (isArray) return value;
   return [value].flat();
}

And then since your query string will be an array, you can tweak your code:

app.get("/search", (req, res) => {
   let search = arrayWrap(req.query.q || "");
   let terms = search[0].split("+");
   // ... do something with the terms
});

Route Parameters

A route parameter are the parts of the URL that will change depending on the data that we want to display. They are essentially variables derived from named sections of the URL.

Express captures the value in the named section and stores it in the req.params property as an object.

profileIdis a route parameter. Express will capture whatever string comes after /profile/ in the URL and store it in req.params.profileId.

app.get('/profile/:profileId', (req, res) => {
   res.json(req.params);
});

You can define multiple route parameters in a URL.

In the example below, the Express route is /profile/:profileId/user/:userId, so req.params.profileId will contain the substring after /profile/ and before /user/, and req.params.userId will contain everything after /user/.

app.get('/profile/:profileId/user/:userId', (req, res) => {
  res.json(req.params);
});

We can put a query and route parameter together as well:

const greeting = (req, res) => {
   const greeting = req.params.greeting;
   const name = req.query.name;
   
   const content = greeting && name ? `${greeting}, ${name}!` :
                                      `${greeting}!`;
   res.send(content);
};
app.get('/speak/:greeting', greeting);

Serve Static files

Most web applications contain static files that return the same response to all users. These could include images, favicons, CSS stylesheets, client-side JavaScript, pre-rendered HTML pages, or any other asset.

Express allows you to define a single directory that contains static assets and returns any file that matches the URL path.

Create a directory named static in your project folder and add a file named page.html with the following content:

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Static page</title>
<meta name="viewport" content="width=device-width,initial-scale=1" />
<link rel="stylesheet" href="styles/styles.css" />
</head>
<body>
<h1>This is a static page</h1>
<img src = "assets/cat.jpg" alt="cat" />
</body>
</html>

Edit index.js file and add the following code after the final app.get() route:

// serve static assets
app.use(express.static( 'static' ));

Navigate to localhost:3000/page.html:

Define Working Directories

A hard-coded static directory is used above. That’s fine for Express, but what if another module needed to locate the same directory to read or write a file?

We can define a fully qualified reference to all working directories in the cfg configuration object. The URL of the current module is available in import.meta.urlso it can be parsed to a file path using the standard Node library.

import { fileURLToPath } from 'url';
import { dirname, sep } from 'path';
const
  __dirname = dirname(fileURLToPath( import.meta.url )) + sep,
  cfg = {
    port: process.env.PORT || 3000,
    dir: {
      root:   __dirname,
      static: __dirname + 'static' + sep
    }
  };
...
// serve static assets
app.use(express.static( cfg.dir.static ));

The url module provides a fileURLtoPath() function, which converts a file:// URL to a fully qualified file path.

The path module provides a dirname() function to extract the directory from a path and a sep constant with the platform-specific path separator (/ on POSIX, \ on Windows).

Middleware Functions

Conceptually, middleware is a way to encapsulate functionality — specifically, functionality that operates on an HTTP request to your application. Practically, middleware is simply a function that takes three arguments: a request object, a response object, and a next() function.

Middleware functions typically:

• run code on every request
• manipulate or change the request and response objects
• terminate a response — perhaps if the user isn’t logged in
• call the next middleware function

All middleware functions receive three arguments:

• req: the HTTP Request object.
• res: the HTTP Response object.
• next: a callback that passes control to the next middleware function.

In an Express app, you insert middleware into the pipeline by calling app.use()

In a middleware function, you either call next() so the next middleware function in the pipeline will execute or send a response to the client (e.g. res.send, res.json, res.render, etc.); if neither is done, the client will hang and eventually time out.

// middleware examples:
app.use((req, res, next) => {
  console.log(`processing request for ${req.url}....`)
  next()
})
app.use((req, res, next) => {
  console.log('terminating request')
  res.send('thanks for playing!')
  // we do NOT call next() here...this terminates the request
})
app.use((req, res, next) => {
  console.log(`whoops, i'll never get called!`)
})

Ways of using next()

• next()
• return next()

Using next() standalone will execute the code after the current middleware function is finished. Using return next() will immediately jump out of the callback and the code below return next() will be unreachable.

app.get('/next', function (req,res,next) {
   console.log('hi there ');
   next();
   console.log('you are still here'); // will be executed
});

The code snippet above will display both console messages while the code below will only display the first console message.

app.get('/return-next', function (req,res,next) {
   console.log('hi there');
   return next();
   console.log('you are still here'); // never executed
});

Common Middleware

While there are thousands of middleware projects on npm, there is a handful that is common and fundamental, and at least some of these will be found in every non-trivial Express project. Some of this middleware was so common that it was actually bundled with Express, but it has long since been moved into individual packages.

• body-parser — provides parsing for HTTP request bodies. Provides middleware for parsing both URL-encoded and JSON-encoded bodies, as well as others.
• compression — improves web application performance, you should compress assets before they’re returned to the browser over the network.
• cookie-parser — provides cookie support.
• serve-favicon — serves the favicon (the icon that appears in the title bar of your browser). This is not strictly necessary; you can simply put a favicon.ico in the root of your static directory, but this middleware can improve performance.
• morgan — provides automated logging support.
• helmet — provides secure HTTP headers.
• static — provides support for serving static (public) files. You can link in this middleware multiple times, specifying different directories.

Disable Express Identification

By default, Express sets the following HTTP response header:

X-Powered-By: Express

It doesn’t do any harm, but you can disable it withapp.disable()

// Express initiation
const app = express();
// do not identify Express
app.disable('x-powered-by');

It will save a few bytes on every HTTP request, and will also give malicious hackers less information about your Node stack.

Handling Errors

When you make a request to an endpoint with no route to handle that request, you see an unfriendly Cannot GET / error in your browser.

Add the following code as the last middleware function to gracefully handle errors when a route can’t be found:

app.use((req, res, next) => {
   console.log('route not handled')
   res.status(404).send('<h1>404 - not found</h1>')
});

In production applications you should give the user more information to help get where they want to go.

Error Handling Middleware

When your app is in error mode, Express will skip over all other middleware until the first error-handling middleware in the stack.

To enter error mode, simply call next with an argument.

If you pass an error to next() and you do not handle it in a custom error handler, it will be handled by the Express built-in error handler; the error will be written to the client with the stack trace.

app.get('/', (req, res, next) => {
   next(new Error ("Something bad happened!"))
});

Express has a special way of creating an error handler. It is exactly the same as other middleware, except for one feature: it has an extra parameter of type error.

Express looks for middleware with four parameters instead of three and with the first parameter as the passed error.

app.use((err, req, res, next) => { 
   console.error(err); // send to server termainal
   res.send(err);  // send to user's browser
});

This middleware should be the last in your pipeline.

We can create a custom error handler to give us flexibility on how our application handles errors centrally in an application.

This custom error handler would normally put placed a separate file.

import  { StatusCodes }  from 'http-status-codes';
const NODE_ENVIRONMENT = process.env.NODE_ENV || "development";
const errorHandler = (err, req, res, next) => {
   const errorMessage = getErrorMessage(err);
   logErrorMessage(errorMessage);
   /**
    * If response headers have already been sent,
    * delegate to the default Express error handler.
    */
   if (res.headersSent) {
      return next(err);
   }
   const errorResponse = {
      statusCode: err.status || StatusCodes.INTERNAL_SERVER_ERROR,
      body: errorMessage
   };
   /*** 
    * Error object should never be sent in a response when
    * your application is running in production.
    */
   if (NODE_ENVIRONMENT !== "production") {
      errorResponse.body = errorMessage;
   }
   res.status(errorResponse.statusCode);
   res.json({ 
              status: err.status || errorResponse.statusCode, 
              message: errorResponse.body 
           });
   next();
}
const getErrorMessage = (err) => {
   if (err.stack) {
      return err.stack;
   }
   if (typeof err.toString === "function") {
      return err.toString();
   }
}
const logErrorMessage = (err) => {
   console.error(err);
}
export { errorHandler }

We can then import it into our server and use it as a middleware:

import { errorHandler } from './utils/error';
...
app.use(handleError);
app.listen(cfg.port, () => {
   console.log(`App listening at http://localhost:${ cfg.port }`);
});

The error stack gives important information about the server, which do you not want in production. You can replace error.stack with error.message

function getErrorMessage(err) {
if (err.message) {
   return err.message;
}
...
export { errorHandler };

If no errors happen, it’ll be as if the error-handling middleware never existed.

Routing Middleware

Up to this point, the middleware we have used has been attached to the middleware pipeline with app.use(). This is known as application-level middleware, but there is also router-level middleware which is attached to specific routes instead.

Some of the routes that you create will have similar functionality. Instead of having common code in multiple routes making it less efficient, you can remove the duplicate code into router-level middleware.

Router-level middleware can be thought of as helper functions that allow your code to be more DRY (Don’t Repeat Yourself).

import validateName from './utils/validateName';
app.get("/hello/:name",validateName, 
        (req, res, next) => {
  const message = `Hello, ${req.params.name}!`;
  res.send(message);
});

app.get("/goodbye/:name",validateName,
        (req, res, next) => {
  const message = `Goodbye, ${req.params.name}.`;
  res.send(message);
});

The code below will either do one of two things:

1. Trigger next() which will execute the next middleware in the middleware stack.
2. Trigger next({ status: 400, message: "Name is too short"}) which will have Express regard the request as being an error and will skip any remaining non-error handling routing and middleware functions and will trigger any error handler middleware.

export function validateName(req, res, next) {
  const name = req.params.name;
  if (name.length >= 3) {
    next();
  } else {
    next({
          status: 400, // overrides default 500 error
          message: "Name is too short."
    }); // triggers an Express error
  }
};

Chaining Middleware

We know now that middleware is just a function with three params (req, res, next), where next is a function that allows you to chain multiple functions.

const isMorning = () => {...}
app.get("/hello",
  // middleware #1
  (req, res, next) => {
    if (isMorning()) {
      res.send("morning");
    } else {
      next(); // moves to next middleware
    }
  },
  // middleware #2: called when isMorning() === false
  (req, res, next) => {
    res.send("afternoon");
  }
);

Here we are chaining two middleware functions to handle /hello route. We use next() to pass control from the first middleware to the second.

In real-world scenarios, middlewares are useful for sharing common code among routes.

Here we created a middleware that only calls next() when the user is authenticated. The middleware is shared by two routes. Note that when the user is not authenticated, we don't call next(), which will stop the chain.

const requireUserAuthentication = (req, res, next) => {
  if (req.user == null) {
    res.status("401").send("User is unauthenticated.");
  } else {
    next();
  }
}

app.get("/me/name", RequireUserAuthentication, (req, res, next) => {
  res.send(req.user.name);
});

Or you can just define middleware functions separated by commas:

app.get("/", middleware1, middleware2, (req, res, next) => {
});

The example below chains multiple middleware using return next() as to send a specific response to the user:

app.get('/rgb',
  (req, res, next) => {
    // about a third of the requests will return "red"
    if(Math.random() < 0.33) return next()
    res.send('red')
  },
  (req, res, next) => {
    // half of the remaining 2/3 of requests (so another third)
    // will return "green"
    if(Math.random() < 0.5) return next()
    res.send('green')
  },
  function(req, res){
    // and the last third returns "blue"
    res.send('blue')
  },
)

Passing Data Between Middleware

There may be instances where data needs to be passed from one middleware to the next. This is possible in one of two ways:

• Add properties to the req object

const passOnMessage = (req, res, next) => {
  console.log("Passing on a message!");
  req.passedMessage = "Hello from passOnMessage!";
  next();
};

The passedMessage property was added to the req object so that it could be used in a later middleware function.

• Store properties inside of the res.locals object

app.use((req, res, next) => {
  // storing user and authenticated for other middleware:
  res.locals.user = req.user
  res.locals.authenticated = !req.user.anonymous
  next()
})

This property is useful for exposing request-level information such as the request path name, authenticated user, user settings, and so on to templates rendered within the application.

Using res.locals is a best practice for storing middleware data.

app.route()

Routes can be created in a number of different ways.

Again a route refers to determining how an application responds to a client request to a particular endpoint, which is a URI (or path) and a specific HTTP request method (GET, POST, and so on). Each route can have one or more handler functions, which are executed when the route is matched.

app.METHOD(PATH, HANDLER)

The first way discussed was called route handlers which are middleware attached to one of the Express methods, for example:

// GET method route 
app.get("/customers", function (req, res) { 
    // anonynous function route handler
    ...
});
// POST method route
app.post("/customer", function (req, res) { // route handler
   // anonynous function route handler
   ...
});
// All HTTP request methods
app.all("/secret", function (req, res) {  // route handler
    // anonynous function route handler
    ...
});

Defining all of our route handlers (in app.js) as anonynous functions will be quickly become unmanageable as your app scales.

You can help with this by using app.route() to create chainable route handlers. Because the path is specified at a single location, creating modular routes is helpful, as is reducing redundancy and typos.

This allows the developer to group one endpoint and various HTTP verbs (GET, POST, PUT, DELETE) in one route handler.

app.route('/book')                 // route handler (GET, POST, PUT)
  .get((req, res) => {
      ...
  })
  .post((req, res) => {
      ...
  })
  .put((req, res) => {
      ... 
  })

But using app.route() doesn’t allow you to group similar routes together. That’s where express.Router comes in. 🤓

express.Router

You can use the express.Router class to create modular, mountable route handlers. An Router instance is a complete middleware and routing system; for this reason, it is often referred to as a “mini-app”.

// app.js: 
import express from 'express';
const app = express();
import { countsRouter } from './routes/customer.router.js';
app.use(express.json());
app.use("/customers", customerRouter);
...
export { app }

The router file defines and exports an instance of express.Router. The router file is only responsible for connecting a path (/) with the route handler for that path.

// routes/customer.router.js
const router = require("express").Router();
function middleware1(request, response, next) {  
    ...
    next();
}
function middleware2(request, response, next) {
    ...
    next();
}
router.use("/:customerId/", middleware1, middleware2);
export { router }

By using express.Router you can modularize route “categories” in separate files to help scale your Express app.

Model View Controller

Express.js opens the door to custom modules and code to read, edit, and respond with data within the request-response cycle. To organize this growing code base, you’re going to follow an application architecture known as MVC.

MVC architecture focuses on three main parts of your application’s functionality: models, views, and controllers.

• Views: Rendered displays of data from your application (e.g. EJS files)
• Models: Classes that represent object-oriented data in your application and database.
• Controllers: The glue between views and models. Controllers perform most of the logic when a request is received to determine how the request body data should be processed and how to involve the models and views

To follow the MVC design pattern, you would move your callback functions to separate modules that reflect the purposes of those functions. Callback functions related to user account creation, deletion, or changes, for example.

MVC is an old concept, actually, dating back to the 1970s. It’s experienced a resurgence thanks to its suitability for web development.

Controller

A controller defines and exports the route handler functions.

So far our route handlers have been written as anonymous functions defined in line with calls to Express methods, but we can move these functions to named functions exported from controller file(s).

A controller can do many things, but there is no standard practice on exactly what they should do.

• Validation of data.
• Handles authentication.
• Stores business logic (if it gets too busy move to services).
• Other stuff? 🤷

Basically, the controller's job is to translate incoming requests into outgoing responses. In order to do this, the controller must take the request data and pass it to the services or modellayer.

Express Folder Structure

Since Express was built as a “minimal and flexible framework”, there is no standard way to structure your files.

Sure, there are recommended ways of doing things. But it’s such a flexible platform that you can often pick any way of doing something and it will likely work.

Instead of asking “What is the best way to structure my files and folders?”, a better question to ask is “What places do my different types of logic go?”.

This question should better guide you on how to structure your Express app based on its functionality and separation of concerns.

Note: Be careful passing the req object beyond routes and controllers. You should take any data out of the req object and create custom objects that need to pass to a service or model layer.

A common practice is to create folders based on the data:

Then has a router, controller, and servicefile for each:

This way, each module imports from only inside its own folder which encapsulates the code and makes it easier to modify. 🤓

But I’ve seen others with all the controllers in a controllers folder, etc. Different strokes for different folks.

Want to learn more? Check out this cool course on YouTube by John Smilga.