Running Build Commands Nicely

All Unix systems come with a nice command that runs a program with a modified "niceness" value. This niceness value is used as a scheduling priority by the kernel process scheduler, and the default value for new tasks is 0. When the system is overloaded the kernel will prioritize processes with a negative niceness value, and likewise it will deprioritize processes with a positive niceness value.

You'll see the niceness value for processes on your system in the NI column when you run top or htop. If you run it on a typical system you'll see that nearly all programs have the default niceness of 0, and you may also see a few high priority system processes or kernel threads that are configured with a negative nice value.

The Linux kernel has a fairly advanced process scheduler: since Linux 2.6.23 (released in 2007) the default process scheduler is CFS (the Completely Fair Scheduler), which uses various heuristics to try to automatically detect "interactive" tasks and give them scheduling priority when the system is under high load. The idea is that if you're under high load (say, you're compiling a large C++ project), CFS uses some heuristics that attempt to tell that the Emacs process you're using is "interactive" because it's waiting for keyboard I/O, and therefore CFS will automatically try to make sure the Emacs process gets a higher scheduling priority than the non-interactive compiler processes, even if they're both running with the default niceness of 0.

This is a nifty trick, but it doesn't always work perfectly. For example, I've found that if I'm compiling code big projects Firefox will slow down, especially if I'm doing things like watching videos. I could run my compilation tasks using the nice command (or even renice an existing command) but it's not my job to remember to do that, my computer should do it for me.

Therefore I have use the following code in my bash configs that automatically run non-interactive tasks that I expect to use a lot of CPU to run nicely:

# Check if a command exists.
exists() { command -v "$1" &>/dev/null; }

# Alias a command to run nicely.
nicealias() {
  if exists "$1"; then
    # shellcheck disable=2139,2140
    alias "$1"="nice $1"
  fi
}

# Automatically run these commands nicely.
nicealias bazel
nicealias bzip2
nicealias fedpkg
nicealias gzip
nicealias make
nicealias mock
nicealias rpmbuild
nicealias xz

You can check that this actually works as expected using the type Bash builtin:

# Check that "make" is actually aliased to run nicely.
$ type make
make is aliased to `nice make'

Dealing With Subcommands

The above works well for simple commands, but tools that use "subcommands" are a bit trickier. For example, I begrudgingly use npm from time to time. I want npm subcommands like npm run to run unmodified, with the default niceness. However subcommands like npm install and npm run build download and compile (or transpile) a lot of Javascript and possibly even C++, and that can use up a lot of CPU time.

It's possible to handle this case using a Bash function, but it's a bit tricky. Here's how I do it.

# Force "npm install" and "npm run build" to run nicely
npm() {
  local nicecmd=()
  if [[ "$1" == "install" || ( "$1" == "run" && "$2" == "build*" ) ]]; then
    nicecmd+=(nice)
  fi
  # shellcheck disable=2230
  "${nicecmd[@]}" "$(which npm)" "$@"
}

Note carefully I use the which command to locate the actually npm command on the filesystem (e.g. /usr/bin/npm) when constructing the cmd array. Previously I used the more portable command -v builtin in the implementation of my nicealias function, but that won't work here. This is because command -v will search the shell environment, which can cause the npm function here to call itself recursively without terminating!

To check that this is actually set up correctly, source the file where you defined the function and check what npm means to the shell using the Bash type builtin:

# Type tells us that npm is actually a Bash function
$ type npm
npm is a function
npm ()
{
    local nicecmd=();
    if [[ "$1" == "install" || ( "$1" == "run" && "$2" == "build*" ) ]]; then
        nicecmd+=(nice);
    fi;
    "${nicecmd[@]}" "$(which npm)" "$@"
}

Note that this is not what which will tell you, as which merely searches $PATH without regard to the shell environment it's run in:

# Which just searches for npm in $PATH, and won't notice the function
$ which npm
/usr/bin/npm

You can use this general programming pattern to do more complicated things to automatically wrap or enhance other programs that use subcommands. For example, my previous job used Phabricator, and as a developer I needed to use the arcanist tool (which installs a command that's actually named arc) to do things like submit code reviews and merge code. I don't have access to my old work Bash configs, but I used this same general approach to write an arc wrapper Bash function that automatically modified the behavior of various arc subcommands to suit my needs.

Globbing Is Useful

One more thing: note that the npm bash function I used checked for npm run build like this:

# ...snippet that checks for "npm run build" from the function above
"$1" == "run" && "$2" == "build*"

If you look carefully you'll notice that I used "build*" with the * globbing operator rather than just "build". This is because both of the following are ways to build npm projects:

# Just a regular npm build command
$ npm run build

# This time run a production build
$ npm run build:prod

The globbing form of "build*" makes sure that my function detects and handles both cases.

Using ShellCheck

You may have noticed some strange looking shellcheck disable comments in my code snippets. For instance, here's the nicealias function I listed earlier:

# Same implementation shown earlier.
nicealias() {
  if exists "$1"; then
    # shellcheck disable=2139,2140
    alias "$1"="nice $1"
  fi
}

I make a habit of using ShellCheck on all of the shell code I write. It's available for pretty much every package manager (apt, dnf, brew, etc.) and is extremely useful for checking shell code for common shell programming errors, including quoting issues and various portability problems (like using bashisms in /bin/sh scripts). It is rather pedantic so it may take some time to get used to, but it's a great long term investment as writing correct shell code is no small task.

All of the warnings it produces have excellent documentation in the ShellCheck wiki, and always include the rationale for the warnings. I personally have it set up in Emacs using flycheck to automatically lint code as I write it, but if for some weird reason you don't use Emacs you should be able to get the same behavior in your preferred editor. In communal projects I'd recommend using it as part of your CI system, or as a git hook.

There are certain cases where you need to use various comment directives to take off the training wheels and tell ShellCheck you know what you're doing. I'm using the disable directive here to do just that. I highly recommend that you also use ShellCheck whenever you write shell code, and if you ever need to quiesce it just use the disable directive.