Monorepos and PNPM workspaces

TL;DR; pnpm workspaces provide a great way to manage your JS monorepo and optimize CI with very little effort just using it's built-in command line functions.

Monolith vs Monorepo (And yes, it's blatant copy of Uncle Bob's WFT per min analogy since it applies here too)

Monorepo vs Monolith

A monorepo is a single repository containing multiple distinct projects, with well-defined relationships. Well, They're not Monoliths. Monoliths are many applications in a single repository with shared code scattered across the code base which creates havoc while scaling your team or application in terms of maintenance. Separation of Concerns is the key.

Key difference between Monorepo and Monolith is that Monorepos have well-defined relationships between the modules and shared code. Monorepos give you all the benefits of Monolith like code-colocation, discoverability without all the headaches of Monolith like poor DX, long lead times, flaky tests that's unrelated to your change breaking the CI.

Well, that's the ideal scenario and only works when you have defined your relationships within the Monorepo well.

PNPM - performant NPM (I'll not go into detail about why you should use pnpm but it's a package manager that's better than npm or yarn in terms of speed and security).

It has few amazing tricks up its sleeve apart from being fast. We'll take a look at one of those tricks that can be used with your Monorepo called pnpm workspaces.

Pnpm Workspaces

This is what a folder structure well defined monorepo using pnpm workspaces would look like:

Your pnpm workspace file would look like:

Now this helps you define your dependencies within the repository using the workspace protocol.

This will make your local development much faster as you can locally link your packages without having to build local dependencies in advance and point that dependents to that build in your filesystem.

And during you package release process i.e, pnpm publish will also dynamically replace the dependency versions accordingly. Pretty cool eh?

CI optimization

Now let's talk about optimization of CI to save you time and resources in your Github workflows.

Let's take a naive approach of building and testing hypothetical App 1 and App 2.

Say we change files related to App 1 and in the PR's CI runs a flaky test on App 2 fails. (We've all been there - an unrelated test failure breaking our pull request:/)

pnpm-filter command lets us use filter for only changed packages on the workspace and run a particular command for that package.

So we can replace our pnpm build:app1 with

 > pnpm --filter "{apps/app-1}[origin/main]" build

to only run the build command if there's any change in App 1.

If we do the same for test and build for App 2 respectively then even if there's a flaky test in App 2 it wouldn't interfere with your PR's CI in this case.

Say we go for an approach by using a matrix that runs these tests in parallel. If one of them is flaky then you'll end up with a broken pipeline again.

pnpm also comes with a handy command to get the list of packages that changed in a json format.

  > pnpm --filter "...[origin/main]" list --json --depth=1

We can use jq or node script to get these packages from the given json and set them as inputs for our matrix so that we skip unecessary steps in our build.

Hopefully, I've convinced you to try pnpm filter commands to optimize the CI of your Monorepo.

Having said that, there are other tools that help you manage and optimize your JS Monorepo at scale like NX or Turborepo. These help not only optimize your CI but also your dev experience.

The above tools are complimentary to pnpm workspaces so using pnpm's built-in commands can be the first step to optimize your CI; you can choose to iteratively adopt these 3rd party tools if need arises.

This post was for people who don't have the luxury of adding more 3rd party tools or try use the most out the tools they're handed.

Like a wise man once said,

"Life is like an (p)npm install - you never know what you are going to get."

(I think it was forrest gump ¯\_(ツ)_/¯)

Go check out https://pnpm.io/filtering for more commands that can help you wrangle your monorepo better :)

Thanks for your time!