How to Implement Search in a Markdown Blog - Next.js Data Caching and API Endpoints

Introduction

Welcome to this tutorial on creating search functionality for a Next.js markdown blog using custom NodeJS scripts and a Next.js API endpoint. API routing in Next.js makes it remarkably easy to create custom API endpoints with little effort. In this tutorial, we'll walk through the steps involved in setting up a cache, implementing a search API with Fuse.js, building a UI to search through posts and finally automating the process with pre-commit hooks.

Prerequisites

For this tutorial you will need access to a terminal with npm installed and ideally an empty git repository to work in (you can omit the pre-commit hooks steps if not).

Getting started

We're going to use the 'Blog Starter Kit' from vercel.com as a starting point. This template already includes some markdown posts which we can use to test our cache and API.

To set up the starter kit, run npx create-next-app --example blog-starter blog-starter-app from an empty directory in the terminal.

Once the installation is complete: using the terminal run npm run dev in the root directory (blog-starter-app) and load up localhost:3000 (or whatever URL is provided in the terminal). It should look something like this.

Creating a posts cache

First, we're going to create a NodeJS script which will take all of the posts from the '_posts' directory and store them in JSON format in a new file. We'll use the JSON object to search through our posts later.

From the root of the project (blog-starter-app) create a new directory named 'cache' containing a file named 'posts.js'

Setting up the Node script

Next, we need to add our new script to our 'package.json' in order to run it.

Add a new script to 'package.json' within the scripts object "cache-posts": "node cache/posts.js"

To run the command and build our posts cache run npm run cache-posts in the terminal from the project root directory.

If the script returns an error make sure your package.json looks like this example and that the cache/posts.js file is in the correct place (it should be like 'blog-starter-app/cache/posts.js').

Check within the cache directory for a new file named 'data/posts.js' - this should contain data from all the posts within the '_posts' directory.

Create a search API

Now we have a cache that we can query; it's time to build our API endpoint. Luckily this couldn't be easier in Next.js! I recommend reading through the official documentation on Next.js API routes for more detail.

Add a new file 'pages/api/search':

Our API will grab the data from our cached posts file and check against the title and description to see if there is any match. If no query is passed to the API, we'll return all posts. If you'd prefer to return nothing if there are no results, you can update the results variable to return an empty array [] instead of posts.

In order to test our API is working as expected, you can test the API response in your browser like so: http://localhost:3000/api/search?query=preview

Add fuzzy search

Our search API is now up and running! We could leave it there or keep building upon the functionality we have created but we're going to implement fuzzy search.

Fuzzy search allows us to display results which don't match our search parameters exactly or are misspelled. This is also known as 'Approximate string matching'. It makes searching much easier for end-users who may not know the exact term they're searching for or how to spell it.

In order to implement fuzzy search we're going to leverage Fuse.js.

Install fuse JS by running npm install --save fuse.js in the root directory.

We'll now need to refactor our API endpoint a little bit in order to utilise Fuse.js:

Make sure to set which fields from your posts you wan't to include in the search.

Now when you run your search you should get results which match even if you misspelled the query: http://localhost:3000/api/search?query=previxw

Display search results

We now have a working API and cache we can query, but we don't have any UI for end-users to be able to utilise this new functionality. We'll continue by creating a new page template which will take a user input and render search results.

Note: Fuse.js returns results in a slightly different format to our initial cache stucture. In the example we've used a map function to restructure our response data. If you've skipped the step using fuzzy search you only need to return res.results without the map function.

Create a new file 'pages/search/index.tsx':

We've utilised the PostPreview component which already exists within the demo project in order to render our search results as post cards.

The template will load all posts by default, if you only want to show results after the user searches then replace defaultPosts with an empty array [].

Navigate to http://localhost:3000/search to check the template is working as expected.

Automate post-Caching with a pre-commit hook

We're almost done! To finish things off we're going to add a pre-commit hook so that our posts are cached every time we make a commit. This saves us the hassle of having to manually run the script every time we make a change to a file. If you're not working with Git you can skip this step. You might also consider caching posts during the build process rather than on commit.

We're going to use Husky to implement our pre-commit hook. First run npm install husky --save-dev to install Husky.

We next need to update our package.json with a new script "prepare": "husky install"

Our scripts should now look like this:

"scripts": {
    "dev": "next",
    "build": "next build",
    "start": "next start",
    "typecheck": "tsc",
    "cache-posts": "node cache/posts.js",
    "prepare": "husky install"
  },

In order to use Husky we now need to run the new script npm run prepare.

Now we'll add our pre-commit hook by running the following command in the root directory:

npx husky add .husky/pre-commit "npm run cache-posts";  git add .husky/pre-commit

Note: We've chained a command to stage the new pre-commit file.

Commit the changes by running git commit -m "Add pre-commit hook for caching posts" from the root directory.

You should see posts being cached during pre-commit. Now every time you commit changes, the posts will be cached automatically.

Next steps

• Now you have search functionality for your blog posts; try adding filters for tags or authors
• Add a check in the search api to load live results in development rather than cached results
• Try paginating the results
• Use debounce to reduce the amount of API requests

Notes

• If you encounter issues installing or running the project, check you're on at least version v18.16.0 of node.

View the completed project on Github