Skip Links

Alexander Morse

Live Reloading From Scratch in Node.JS

Posted on Sep 21, 2021 by Alexander Morse.

Modern front-end developers have a lot of tools at their disposal, but the idea of "live reloading" is easily one of the most important. The idea behind it is simple — launch a special development server to view your project while it's being developed. Then, when some change is made to the project's source code, the page in our browser automatically refreshes to reflect the change.

It's not too hard to get this feature these days. VS Code has the Live Server extension, and in Node there's the BrowserSync package alongside countless other implementations. For most projects, reaching for one of these tools is usually the right call. But even though the problem is easily solved for us, it's still a good practice to try our hand at a solution ourselves.

So let's follow (very loosely) in the steps of BrowserSync and similar tools, and create our own live-reloading development server in Node.JS.

Overview

Let's start by nailing down exactly what we want to build. There are two main approaches to updating an application in real-time:

  • Live Reloading is the simplest approach. Whenever some source file changes, the entire application restarts. In our case, this means we tell the web page to refresh itself.
  • Hot Reloading (Webpack calls it Hot Module Replacement, or HMR) is more complex. Instead of restarting the entire application for every change, the application dynamically replaces only the parts we edit. For example, if a CSS file changes, the page's style information is updated in real time, with no reloading required.

Between the two, we'll be building a live-reloading development server. This will be fast enough for our purposes, and we could always extend it to allow for hot-reloading later on.

In that case, let's take a moment to figure out what pieces we're going to need.

  • First and foremost, we need an assets server. At a bare minimum, we need to be able to serve the files that live in our project's root directory, like index.html and styles.css.
  • Second, we need to deal with the fact that the HTML files in our source code are dumb. They have no way of knowing that they're in a development server. Therefore, we need to inject some Javascript code into each HTML file before we serve it. This code won't do anything except listen for its cue to refresh the page.
  • Third, we need to give our injected Javascript something to listen for. We'll do this by creating a WebSocket server for the injected code to connect to. This server will accept connections, and then tell those connections when to reload.
  • Finally, we need to set up file watching on the source files, so that changes to these files will trigger a page reload.

And that's it — if we can put all of those pieces together, we should have a perfectly serviceable development server to work with. In fact, even though we'll be using Node, this approach is language-agnostic. You can use any programming language just as long as it can implement each of the four steps.

Let's get started.

Setup

We begin with a simple project directory. Inside of this lives a single src subdirectory that holds all of the files we'd like to develop:

| - live-reload-from-scratch
    | - src
        | - index.html
        | - index.js
        | - styles.css

screenshot of the project in development

If you'd like to follow along with these exact files, I've made the base code available at this GitHub repository, and you can see the code I wrote for this article on its "solution" branch. But there's nothing special about this starting point, so feel free to drop in any assets you'd like.

You'll also need some version of Node.JS and the npm CLI, which is included with Node. We're going to implement as much as we can from scratch, but we will bring in a handful of npm packages to avoid some lengthy digressions.

Once you've got your project directory set up, make sure to initialize it as a Node project:

# Initialize project
npm init -y

Serving Assets

We'll start out by creating a basic web server — there isn't much point to live-reloading a web page if we can't load anything in the first place.

To implement this, we'll use Node's standard http library. The result will be a very bare-bones web server with the following properties:

  • The server rejects every method except for GET.
  • The server responds with a 404 error if the requested resource is not available in src.
  • If the requested resource exists, it is served with the 'Content-Type' and 'Content-Length' headers. (This isn't strictly necessary, but it will make certain browsers shut up about missing MIME types.)
  • If serving an HTML file, that file is injected with a script tag pointing it toward a file called client.js.
  • If serving client.js, this file is loaded and served from a special location outside of the src directory.

That's a lot to take in all at once, and implementing the server will be the lengthiest part of this exercise. But if we take it step-by-step, everything should come together pretty easily.

HTTP Server

Let's begin by creating a new directory sibling to src, so that our server code has somewhere to live. Inside of it, we'll create our server.js file:

mkdir dev-server
touch dev-server/server.js

Inside of server.js, we'll create the simplest web server imaginable: one that doesn't actually do anything.

// server.js

const http = require("http");
const path = require("path")

////////////////////
// Project Constants
////////////////////

// Path to the files we want to serve
const ROOT = path.resolve("./src");

// Port to run the HTTP server on.
const PORT = 8080;

////////////////////
// HTTP Server
////////////////////

const assetsServer = http.createServer(async (request, response) => {});

assetsServer.listen(PORT, () => {
	console.log(`Assets Server is running on port: ${PORT}`);
});

Run this code from the top level of your project with node dev-server/server.js, and you should see that the HTTP server is now listening on port 8080. If you point your browser to localhost:8080...you should see that the request hangs indefinitely, since the function we passed to http.createServer isn't actually doing anything. Let's fix that.

First of all, let's implement the first requirement our server has: reject requests with any method other than GET. By the way: don't worry too much if you're unfamiliar with the server's request and response objects. As you'd expect, these represent the incoming request and outgoing response, respectively. Just try to follow along with the pattern below, since we won't be doing anything more complicated with either of them.

// server.js

////////////////////
// HTTP Server
////////////////////

const assetsServer = http.createServer(async (request, response) => {
	// Special Case: Reject non-GET methods.
  if (request.method !== "GET") {
    const responseBody = `Forbidden Method: ${request.method}`;

    response.writeHead(403, {
      "Content-Type": "plain/text",
      "Content-Length": Buffer.byteLength(responseBody);
    });

    return response.end(responseBody);
  }
});

Let's quickly go over the basic pattern here. When we decide we want to send a response back to the client, we first define responseBody, which contains the content we want to send. Then we call the writeHead method on the response, passing it the status code (in this case, 403 for Forbidden), as well as an object of two header values that describe the content's type and length. Then we end the response, passing in responseBody as the last thing we want to write to it.

Real-world servers tend to be a lot more complex than this, but here we'll follow this pattern every time we want to send something.

Back on topic, we can re-run the server to find that it still doesn't do much. Since browsers default to GET requests, this error-check will never actually be triggered. You can either take my word that it does work, or else test it out yourself. You can use browser extensions like RESTer (for Firefox) or RestMan (for Chrome), or use the command line with something like curl -X POST localhost:8080 to verify that you get the "Forbidden Method: ..." response as intended.

Let's move on to the general case - when receiving a GET request for a resource, we'll try to look up that resource in the ROOT directory (which we've set to src). If it exists, we'll send it to the client no questions asked. If it doesn't, we'll send a 404 response.

Before that, let's go ahead and install our first npm package. 'mime' will make our lives a little easier by figuring out the MIME type of a file based on a file's extension, rather than requiring us to write code for it ourselves. Also, we'll import the fs library to read from the ROOT directory.

Install mime real quick:

npm install --save-dev mime

...and then we can begin to implement the general case. Note the new imports of mime and fs at the top of server.js.

// server.js

const fs = require("fs");
const mime = require("mime");

////////////////////
// HTTP Server
////////////////////

const assetsServer = http.createServer(async (request, response) => {
  // Special Case: Reject non-GET methods.
  if (request.method !== "GET") { /* snip */ }

  
  // General Case: GET request for any resource

  // Parse the request URL to get the resource pathname.
  const url = new URL(request.url, `http://${request.headers.host}`);
  let pathname = url.pathname;

  // If the pathname ends with '/', append 'index.html'.
  if (pathname.endsWith("/")) {
    pathname += "index.html";
  }

  try {
    // Try to read the given resource into a Buffer.
    const resourcePath = path.join(ROOT, pathname);
    const responseBody = await fs.promises.readFile(resourcePath);

    response.writeHead(200, {
      "Content-Type": mime.getType(resourcePath),
      "Content-Length": responseBody.length,
    });

    return response.end(responseBody);
  } catch (e) {
    // Respond to all errors with a 404 response.
    const responseBody = `Cannot GET resource: ${pathname}`;

    response.writeHead(404, {
      "Content-Type": "plain/text",
      "Content-Length": Buffer.byteLength(responseBody),
    });

    return response.end(responseBody);
  }
})

Don't worry — this section of code is as complicated as we're going to get in this project. Let's walk through it. First, we need to parse the URL, in case it's for something weird like /styles.css?foo=1&bar=true. We just want the path name, so we use the URL Interface to parse the entire URL and pick out the path for us. Then, if the path name ends in /, the convention is to serve index.html, so we simply tack the filename onto the end of the path.

Now that we've determined the path name, we'll try to read the file into a Buffer. Since there's no guarantee that the file actually exists, we wrap the operation in a try/catch block, and serve a basic 404 error for the resource if any error occurs. We join the path name to ROOT with path.join to get the absolute path of the file, and then try to read it with await fs.promises.readFile. If that operation works, then we go ahead and write the file contents to response with the usual pattern. Also note that we're setting the Content-Type header using a call to mime.getType(resourcePath), which saves us the trouble of having to distinguish between file types ourselves.

Run the server now, navigate to localhost:8080 in your browser, and you should see that your server is working! Feel free to enjoy your moment of triumph (and perhaps test it with a few different path names) before moving on.

JavaScript Injection

At this point, we have a working server that serves files inside of our src directory. As great as that is, the server is still missing two of our requirements. First, it needs to intercept HTML files and inject a script tag for client.js. Second, it needs to respond to requests to client.js differently than it would for other assets, serving a special file that lives in the dev-server directory.

Let's start by properly serving requests to client.js. First of all, let's create the file:

touch dev-server/client.js

Anything we write in this file will be executed by the browser it's injected into. Eventually, this means connecting to a WebSocket server, but for now let's give it something simple to do:

// client.js

console.log("I'm a WebSocket!");

Next, we'll add a new special case to our server code: if the request is for client.js, we load this file and serve it immediately:

// server.js

////////////////////
// HTTP Server
////////////////////

const assetsServer = http.createServer(async (request, response) => {
  // Special Case: Reject non-GET methods.
  if (request.method !== "GET") { /* snip */ }

  // Special Case: GET '/client.js'
  if (request.url === "/client.js") {
    const responseBody = await fs.promises.readFile("./dev-server/client.js");

    response.writeHead(200, {
      "Content-Length": responseBody.length,
      "Content-Type": "application/javascript",
    });

    return response.end(responseBody);
  }

  // General Case: GET request for any resource
	/* snip */
  }
});

Since we know that client.js exists, we'll allow ourselves to be a little more lax with error-handling. Make a quick request to localhost:8080/client.js to confirm that it's working, and then we can move on.

Next, we somehow need to modify the HTML files in src so that they'll load client.js. We could do this in a few different ways, but we'll keep it simple. Since most HTML files tend to have a <body> tag, there is usually a closing </body> tag. And that's usually a good place to put scripts. So, all we really need to do is replace </body> with a different string...let's say, <script src="client.js"></script></body>.

We can make this happen inside of the code for our general case. If the extension for a resource is .html, we'll convert the file to a string, make the replacement, and then convert it back into a Buffer.

// server.js

////////////////////
// HTTP Server
////////////////////

const assetsServer = http.createServer(async (request, response) => {
  // Special Case: Reject non-GET methods.
  if (request.method !== "GET") { /* snip */ }

  // Special Case: GET '/client.js'
  if (request.url === "/client.js") { /* snip */ }

  // General Case: GET request for any resource

  // Parse the request URL to get the resource pathname.
  /* snip */

  try {
    // Try to read the given resource into a Buffer.
    const resourcePath = path.join(ROOT, pathname);
    let responseBody = await fs.promises.readFile(resourcePath);

    // HTML Files: Inject a <script> tag before </body>
    if (resourcePath.endsWith(".html")) {
      responseBody = responseBody
        .toString()
        .replace("</body>", '<script src="client.js"></script></body>');
      
      responseBody = Buffer.from(responseBody);
    }

    // Serve responseBody as usual...
    
  } catch (e) { /* snip */ }
});

Notice that we had to change the const declaration of responseBody to let, since its value might change. But now, if you run the server and check out the index page, you should notice client.js printing "I'm a WebSocket!" to the console.

And just like that, our server is complete. It can serve arbitrary files from the src directory, and also inject a custom script into HTML files that we'll use to trigger reloads. Our project is almost done at this point, since the last two pieces of the puzzle should come together pretty quickly.

WebSocket Server

Now that we're injecting our client.js script into HTML files, we're finally in a position to implement live-reloading. We'll do this by having client.js connect to a WebSocket server, wait for that server's signal, and then reload the page when it's told. I suppose that means we need to make a WebSocket server, then.

For this, we'll be using the 'ws' package. I'd originally intended for this project to also implement a WS server from scratch, but walking through it would probably double the size of this article. So let's take a shortcut and install the package now:

npm install --save-dev ws

Setting it up will be quite easy. First, import ws at the top of server.js. Then we'll create a new server instance underneath the assets server.

// server.js

const ws = require("ws");

////////////////////
// Project Constants
////////////////////

/* snip */

// Port to run the WebSocket server on.
const WS_PORT = 8081;

////////////////////
// HTTP Server
////////////////////

/* snip */

////////////////////
// WebSocket Server
////////////////////

const reloadServer = new ws.WebSocketServer({
  port: WS_PORT,
});

reloadServer.on("listening", () => {
  console.log(`WebSocket Server is running on port: ${WS_PORT}`);
});

That's all there is to it — it's amazing how succinct code can be when we allow ourselves to use proper libraries. Now, when a WebSocket instance connects to ws://localhost:8081, reloadServer will keep a reference to it inside of a property called clients. Whenever a file changes, we should loop through each client and send it some message telling it to reload.

Since reloadServer implements the EventEmitter interface, we'll define a listener on it for a custom reload event, which we'll emit ourselves in the next section:

// server.js

////////////////////
// WebSocket Server
////////////////////

const reloadServer = new ws.WebSocketServer({
  port: WS_PORT,
});

reloadServer.on("listening", () => {
  console.log(`WebSocket Server is running on port: ${WS_PORT}`);
});

reloadServer.on("reload", () => {
  reloadServer.clients.forEach((client) => {
    client.send("RELOAD");
  });
});

We could have chosen to send data in a more complex format. JSON is usually a popular choice. But since this server will only ever do one thing, a simple string will suffice.

Finally, we need to update the code in client.js. Instead of just logging a message, we'll create an instance of WebSocket , and have it connect to our WebSocket server running at ws://localhost:8081. The socket doesn't need to do much except listen for a message whose data is "RELOAD". When it does, it should reload the page.

// client.js

const socket = new WebSocket("ws://localhost:8081");
socket.addEventListener("message", (event) => {
  if (event.data === "RELOAD") {
    window.location.reload();
  }
});

Here, we make use of the Location interface to actually trigger the reload.

File Watching

We are now this close to a working, live-reloading development server. All that's left is to watch every file in the ROOT directory for changes, and then call reloadServer.emit("reload") whenever that happens.

For this, we'll go with 'chokidar' , which is a file-watching package used by just about every big name in the Node ecosystem. Just take a look at the list of chokidar's dependents — if a tool is watching files for you, it's probably using chokidar under the hood.

Let's install it for ourselves:

npm install --save-dev chokidar

We'll opt for a very straightforward use of chokidar here. Import it at the top of server.js, and then call it's watch function with a glob corresponding to "every file in ROOT". When the instance emits an 'all' event (which occurs whenever something inside the watched directory changes in any way), we call reloadServer.emit('reload').

// server.js

const chokidar = require("chokidar");

////////////////////
// Project Constants
////////////////////
/* snip */

////////////////////
// HTTP Server
////////////////////
/* snip */

////////////////////
// WebSocket Server
////////////////////
/* snip */

////////////////////
// File Watching
////////////////////

chokidar.watch(ROOT + "/**/*.*").on("all", () => {
  reloadServer.emit("reload");
});

Go ahead and run the server now. Then navigate to localhost:8080 in your browser, and open up some file in your project's source directory — perhaps a CSS or HTML file. Edit it however you like, hit save, and watch as the browser page instantly reloads to reflect the changes you've made.

With that, you've successfully created a live-reloading development server. Congratulations!

Wrap Up

When it comes to exploratory articles like this, I'm usually quick to include a warning about using the end result in real-world situations, since our code usually doesn't account for every edge case that can crop up in those situations. But in this case, I think we've built something perfectly sufficient for most basic web-development tasks. Our HTTP server is kind of shaky, but as a local development server, it's kind of allowed to be.

We'll end our exploration here, but there are definitely more improvements we could make. I'll leave these as extra credit for anyone interested:

  • Augment our basic HTTP server with a library like Connect. See how the use of basic middleware streamlines our server logic.
  • Set up hot-reloading for one or more types of asset. This is probably simplest with CSS — our WebSocket server should send a different kind of message to signal client.js to swap out any link tags whenever a CSS file changes.
  • Turn server.js into a fully-fledged CLI. Let it take arguments to specify options like the ROOT directory and server ports.

© 2021 Alexander Morse