fx is my favorite terminal-based JSON tool. Let's look at how it lets us interactively explore, query, filter, and edit JSON. To wrap things up, we'll compare it to jq, and I'll explain why I prefer fx.

To show off fx, we'll need some JSON. We'll start by fetching JSON of Harry Potter characters.

curl https://hp-api.onrender.com/api/characters > hp.json

If we cat hp.json, we see that it is a single line of JSON (formatted for computers, not for human readability).

Explore interactively

Running fx with the JSON file's name will let you explore it interactively with human-friendly formatting.

fx hp.json

You can navigate using up and down arrows or vim-style j/k navigation. You can search with / and go to previous/next matches with n/N

Note that fx shows the current JSON path in the bottom left as you move around. Press . to enter "dig" mode to fuzzy-search for a path.

Pressing y lets you copy the value, path, or key under the cursor.

q, ctrl-c or esc will exit fx.

fx --help will show you how to collapse/expand nodes and the rest of the shortcuts.

Querying and filtering

fx querying will feel like the JavaScript map/filter/etc. you already know. fx also provides some optional syntactic-sugar.

Here are some example commands with their non-sugary and sugary versions.

# get a list of wand woods
fx hp.json '.map(x => x.wand.wood)'
fx hp.json .[].wand.wood
fx hp.json @.wand.wood

# wand woods without blank wood names
fx hp.json '.map(x => x.wand.wood).filter(x => x)'
fx hp.json .[].wand.wood '.filter(x => x)'
fx hp.json @.wand.wood '.filter(x => x)'

# living blonde wizard names
fx hp.json '.filter(x => x.alive && x.hairColour === "blonde" && x.wizard).map(x => x.name).sort()'
fx hp.json '.filter(x => x.alive && x.hairColour === "blonde" && x.wizard)' @.name sort

Built-ins and custom functions

sort is a built-in function (there's also uniq and more). fx also allows you to define your own functions in ~/.fxrc.js (or a local .fxrc.js).

Using .filter(x => x) to remove blanks is common enough that I've written a custom present function in my ~/.fxrc.js

// reject any falsy/blank values from an array
global.present = (arr) => arr.filter((x) => x);

To use a custom function or built-in function, you add it as an argument to the fx command.

# unique wand woods
fx hp.json @.wand.wood sort uniq present

Because you can run any JavaScript in your custom functions, you can write functions to do things like filter out secrets or sensitive information.

Let's imagine we never want to see Voldemort's name, and we never want to leak our Stripe API key while recording a screencast. We can avoid both disasters by building a confidential function.

// define secrets and their replacements
const secrets = {
  "Lord Voldemort": "He-Who-Must-Not-Be-Named",
  [process.env.STRIPE_API_KEY]: "********",
};

// replace secrets with their replacements
global.confidential = (x) => {
  let stringVersion = JSON.stringify(x);

  Object.keys(secrets).forEach((secret) => {
    stringVersion = stringVersion.replaceAll(secret, secrets[secret]);
  });

  return JSON.parse(stringVersion);
};

Here's some example usages:

# without confidential
fx hp.json '.filter(x => x.house === "Slytherin")' '.[3].name'
Lord Voldemort

# with confidential
fx hp.json '.filter(x => x.house === "Slytherin")' confidential
He-Who-Must-Not-Be-Named

# Set a fake stripe API to test marking those as confidential
export STRIPE_API_KEY="Potter"

# without confidential
fx hp.json .[0].name
Harry Potter

# with confidential
fx hp.json .[0].name confidential
Harry ********

Modifying JSON

You use map to modify JSON.

e.g., here's how we'd replace blank strings for wand.wood with nulls.

fx hp.json '.map(c => { c.wand.wood = c.wand.wood || null; return c })' > modified.json

fx versus jq

jq is a great tool. If you already know it, you'll probably be happy continuing to use it (though you can benefit from the interactive parts of fx).

However, learning jq is daunting because it is a DSL. I've found I can read a jq command reasonably easily but struggle to write one from scratch because I haven't internalized their DSL yet. I could learn jq's DSL, but I haven't made it a priority yet.

While jq is a DSL you have to learn, fx is just using JavaScript with some completely optional syntactic sugar.

Compare the following examples:

jq '[.[] | select(.hairColour == "blonde" and .alive == true and .wizard == true) | .name ] | sort' hp.json
fx hp.json '.filter(x => x.alive && x.hairColour === "blonde" && x.wizard).map(x => x.name).sort()'

Here's that same fx command with the sugar.

fx hp.json '.filter(x => x.alive && x.hairColour === "blonde" && x.wizard).map(x => x.name).sort()'
fx hp.json '.filter(x => x.alive && x.hairColour === "blonde" && x.wizard)' @.name sort

While the syntactic sugar lets you be more succinct, you don't have to learn it to use fx successfully.

---

fx is a power tool with the ergonomics I'm already familiar with. The ability to write custom functions is super compelling. You should give it a try.

I haven't posted here all year, so I figured I could at least aggregate some links to other things I've done this year.

2023 started with me going all-in with some cofounders on Prefab. I'm super enjoying building developer tools. I've been able to go deep into the Language Server Protocol, CLIs, and VS Code/Neovim tooling to help build the best feature flags, live config, and dynamic log levels for developers. You should try it out -- it'd mean a lot to me 🫶

Here are some blog posts I wrote for the Prefab blog.

• Time-traveling Ruby Logger
• Hands-On LSP Tutorial: Building a Custom Auto-Complete
• Hands-On LSP Tutorial: Configuration
• LSP: Writing a Language Server in Bash
• Making Front End Logging Useful
• Gem install sassc is always going to be slow. Your Docker builds don't have to be.

I started a youtube channel. There is only one video so far, but I plan to hit it hard in 2024. Like and subscribe and all that.

I've also started an LSP Newsletter if you'd like to join me on my journey into developer tools.

The problem

Pressing (not holding) ⌘-tab is useful for swapping between apps. If you ⌘-tab from your editor to your browser, you can then ⌘-tab back to your editor. You can jump back and forth between your editor and browser with just a quick ⌘-tab keystroke. It is a great workflow as long as you're using exactly two apps.

Then you get a Slack message and after giving Slack focus, ⌘-tab takes you back to your browser, the next ⌘-tab sends you back to Slack.

Sigh. OK, so we hold ⌘ and push tab until we're back to our editor and then hold ⌘ and push tab until we're back to our browser and now we can hit ⌘-tab to quickly jump between the two again. Order is restored.

This is mostly a minor nuisance except that this can happen dozens of times throughout the day. Little frictions add up and break flow.

The journey

I tried using Spaces a few times before but the animation was honestly slightly nauseating. If you turn on reduce motion then Spaces changes from a dizzying sliding animation to a more subtle fade animation. The fade animation is pretty tolerable and you can drag whatever applications you want to the proper Spaces and jump around with the keyboard. Success!

Except that somehow the reduce motion setting stopped working for me after a reboot -- it is still checked, it just doesn't do anything and I'm back to the abrasive sliding animation.

Gross. I started looking at other options. yabai looks powerful but requires you to disable System Integrity Protection (permanently, not just during installation) and that's a non-starter for me.

The solution

Taking a step back, what am I really trying to accomplish? I want to be able to press a key and jump to an app (or group of apps). Ideally I don't have to wrangle app the apps into the right workspace like I did with Spaces after every reboot.

Fine. I'll write some AppleScript. After lots of Googling and some optimization, I ended up with a script I'm calling Layers.

How does it work? Create a file in your home directory named .layers. Each line of the file should include a comma-separated list of applications in that layer. Running the layers script will generate compiled AppleScripts to focus applications and spit out a command to bind to a keyboard shortcut.

My .layers file looks like this:

Alacritty
Firefox, Google Chrome, Safari
Slack
Messages, Music, Stream Deck
zoom.us

The layers output is

Add keyboard shortcuts for the following:
ls /Users/ship/.layers_scripts/layer_0*_compiled | xargs -L 1 -P 8 osascript
ls /Users/ship/.layers_scripts/layer_1*_compiled | xargs -L 1 -P 8 osascript
ls /Users/ship/.layers_scripts/layer_2*_compiled | xargs -L 1 -P 8 osascript
ls /Users/ship/.layers_scripts/layer_3*_compiled | xargs -L 1 -P 8 osascript
ls /Users/ship/.layers_scripts/layer_4*_compiled | xargs -L 1 -P 8 osascript

I assign each line starting with ls to a shortcut via an Alfred workflow but you can use any tool that lets you map bash commands to keyboard shortcuts. I'm using ctrl-1 through ctrl-5 since those worked well enough for me with Spaces.

The ls ... command finds all the compiled scripts for a layer and passes them to xargs to run in parallel (up to 8 at a time).

The compilation + parallelization felt like overengineering at first, but it took little time to implement and has a real impact on responsiveness.

Pressing ctrl-2 will bring Firefox, Google Chrome, and Safari to the forefront of my screen. It skips any apps that aren't running. I can press ctrl-1 to get back to Alacritty where my editor is running.

This is better for me than ⌘-tab because it is predictable -- the positions of things never change. This is better for me than Spaces because there's no distracting animations.

There's one downside I can think of when compared to Spaces -- you can't have different windows of an Application mapped to different Layers. I didn't use this functionality in Spaces so I don't miss it.

Why "Layers"? Spaces are effectively distinct virtual screens. Layers don't have this same isolation and apps overlap. This overlap allows for composability and use-cases not allowed by space.

I've been using it a few days now and it feels fantastic. Take it for a spin and let me know how it works for you.

I've packaged up the code action from the last post into a neovim plugin: ruby-code-actions.nvim. Since publishing the plugin, I've added new code actions for correcting the current line, selected lines, or entire file with RuboCop.

I wrote it using plenary for unit testing and found the experience lovely. It has good mocking support for vim.api and your custom code. I couldn't out the correct way to mock things like vim.cmd and such but I worked around this by using vim.api equivalents or wrapping calls and mocking the wrappers. I can't say enough nice things about plenary or TJ DeVries' work on telescope and neovim itself.

In the end, I have a spec file I can run via make and the development process feels more like the comfortable TDD I'm used to.

NOTE: since this was published, I've released a plugin with this code action and more: ruby-code-actions.nvim.

Language servers are a great tool for empowering your editor, but your language server might not support every feature you want. So what can you do? Bring your own functionality!

What are my options for custom LSP features?

So far, I've looked into efm-langserver and null-ls.nvim.

efm-langserver is an editor-agnostic language server meant for helping you wire up external commands as diagnostics, code actions, etc. It looks promising but I've had trouble getting it working with neovim. YMMV.

Fortunately, for neovim we have null-ls.nvim. It allows you to "Use Neovim as a language server to inject LSP diagnostics, code actions, and more via Lua."

You might worry that writing custom actions in Lua means you're building something less portable than what you'd get with a script wired up to efm-langserver. Maybe. But you could always throw your Lua code in a shell script and call that from null-ls or from efm-langserver. The benefit of null-ls running Lua directly in neovim is that rather than communicating by shelling out processes, you're keeping everything in-process.

My null-ls setup

Here's my current packer.nvim config for null-ls:

use {
    'jose-elias-alvarez/null-ls.nvim',
    requires = {{'neovim/nvim-lspconfig'}, {'nvim-lua/plenary.nvim'}},
    config = function()
        local null_ls = require("null-ls")
        local sources = {
            null_ls.builtins.code_actions.gitsigns,
            null_ls.builtins.formatting.prettier,
            null_ls.builtins.formatting.mix,
            null_ls.builtins.formatting.rubocop,
            null_ls.builtins.diagnostics.rubocop,
            null_ls.builtins.diagnostics.shellcheck
        }
        require("custom_code_actions")
        null_ls.setup({sources = sources, debug = true})
    end
}

You'll note that null-ls provides some nice builtins for formatting, diagnostics, and code actions. Unfortunately there's no builtins for ruby code actions.

I keep my custom code actions in custom_code_actions.lua and register them with the require("custom_code_actions") line.

Let's write a simple code action to see how it works. We'll write a code action to insert the frozen_string_literal directive in our ruby file if it isn't already present.

How do we define a custom action?

Let's look at the skeleton of a code action:

local null_ls = require("null-ls")

local frozen_string_actions = {
    method = null_ls.methods.CODE_ACTION,
    filetypes = {"ruby"},
    generator = {
        fn = function(context)
          -- ...
        end
    }
}
null_ls.register(frozen_string_actions)

What's going on here?

• We require null_ls to reference.
• We define a table named frozen_string_actions.
• It has a method key to specify that we're defining a code action.
• It has a filetypes array that specifies which file types we want this to apply to.
• It has a generator function that takes a param (here named context) describing the current context of the file in your editor (current selection, etc).
• Finally, we register our code action with null-ls

The generator function

The generator function is invoked when the editor is considering which code actions to present to the user. The function should consider the context of the current file and return a table of relevant actions. It could return nothing if no actions are relevant.

Context params

Let's see what the context argument passed to our generator contains.

We create test.rb with the content

puts "Hello world"

If we change the body of our generator body to be print(vim.inspect(context)), open our test.rb in neovim, and request available code actions (I'm using :Telescope lsp_code_actions), our generator will be invoked, and we'll see the table passed in as context:

{
  bufname = "/private/tmp/out.rb",
  bufnr = 1,
  client_id = 1,
  col = 16,
  content = { 'puts "Hello world"', "" },
  ft = "ruby",
  lsp_method = "textDocument/codeAction",
  method = "NULL_LS_CODE_ACTION",
  range = {
    col = 17,
    end_col = 17,
    end_row = 1,
    row = 1
  },
  row = 1
}

For our needs, the important part is content. If the first line of the file, context.content[1], is something other than the frozen_string_literal directive, we should return an action to allow the user to insert the directive. If it does match the directive, our action isn't applicable so we return nothing.

The implementation

Here's an implementation of the generator function to conditionally insert the directive:

fn = function(context)
    frozen_string_literal_comment = "# frozen_string_literal: true"

    first_line = context.content[1]

    if first_line ~= frozen_string_literal_comment then
        return {
            {
                title = "🥶Add frozen string literal comment",
                action = function()
                    lines = {frozen_string_literal_comment, "", first_line}

                    vim.api
                        .nvim_buf_set_lines(context.bufnr, 0, 1, false, lines)
                end
            }
        }
    end
end

Walking through, we see that:

1. We declare a variable with the directive content
2. We get the first line of the file
3. We check to see if they differ
4. If they do, we return a table with a title that shows up as a prompt for the user and an action that is executed if the user selects our code action.

Here's what the title looks like in :Telescope lsp_code_actions:

After selecting our code action, we see our file has been updated with the directive:

There's not much more to say here that you can't learn with :help (e.g. :help nvim_buf_set_lines).

Here's a gist with the entire implementation.

What's next?

Since we determine in our generator if the code action should apply, we might also want to consider the user's selection. If they've selected multiple lines of text, they're probably not intending to invoke a code action to add the frozen_string_literal directive. We could return nothing if we saw that the context.range.row differs from the context.range.end_row.

In a standalone ruby script, the frozen_string_literal directive can also appear on the second line. We could update our generator to insert the directive on the second line if there's a shebang line on line 1. We could return no action if the directive is on either the first or second line.

Have fun!

Applescript is useful but interpreted scripts are fairly slow to start.

Consider a script that only has the number 1 in it.

$ echo "1" > example.scpt

When run with osascript example.scpt, this will output 1.

A timed run of the script shows it takes roughly 430 milliseconds on my machine.

$ time osascript example.scpt
1

real    0m0.429s
user    0m0.053s
sys     0m0.053s

We can't have a much simpler script than this, so it looks like the price of using interpreted Applescript is a minimum of ~430ms. If you have a script that runs frequently, this might be too slow.

You might not be aware that you can compile AppleScript. Not surprisingly, this compilation improves the startup time.

$ osacompile -x -o compiled-example example.scpt
$ time osascript compiled-example
1

real    0m0.252s
user    0m0.034s
sys     0m0.036s

We're down from over 400ms to 250ms. That's still not ideal to run in a tight loop, but the savings might be compelling depending on your usage.

Without strict enforcement of boundaries, it can be a mystery where changes happen in the state in your JavaScript application (this is a reason tools like Redux are compelling). I was recently investigating a bug in an app where the source of a state change was a mystery. The app keeps track of the count of videos uploaded. Deleting a video could sometimes cause the count to decrement twice when it should only decrement once. I would be able to find this bug quickly if I could get a stack trace every time the state was modified.

This isn't a new problem. You may remember the promise of obj.watch and Object.observe() before they were deprecated.

Let's talk about some current options.

Proxy

Proxy allows you to create a replacement for your object that wraps the original object. You can define your own set "trap" to log the stack trace (or break into a debugger, etc.).

example:

let proxy = new Proxy(objectYouWantToObserve, {
  set: function(obj, prop, value) {
    if (prop === 'thePropertyWeWantToObserve') {
      console.trace(`set: ${prop} -> ${value}`)
    }

    // The default behavior to store the value
    obj[prop] = value;

    // Indicate success
    return true;
  }

  // you can optionally define an additional trap for `get` if you're also
  // interested in tracking reads for the property
});

You would then need to use your proxy instance wherever you would have used objectYouWantToObserve.

The downside of this approach is that, depending on your app design, it isn't always convenient to replace references to the original object.

breakOn

Paul Irish's library break-on-access exposes a breakOn function that will happily invoke the debugger whenever a property is modified (or, optionally, accessed for read).

example:

breakOn(objectYouWantToObserve, "thePropertyWeWantToObserve")

The upside of this approach is that it requires no additional modification to your code. Any changes to thePropertyWeWantToObserve will invoke the debugger.

I keep break-on-access in my Chrome snippets and reach for it first when I need to track down a change in and object's top-level property.

What if I need to observe changes in dynamically nested objects?

If you need to observe properties changing in nested objects, break-on-access won't help you. With some effort, you can use nested Proxys by returning a new Proxy from the Proxy's set trap. I'd also consider something like Observable Slim since it allows observing even deeply-nested changes to an object and its usage is similar to that of Proxy.

I use a custom status line in tmux to show the CI test results of my current branch. This post will talk a little about tmux status lines, hub's ci-status command, and my ci-status-indicator scripts.

Example status line:

Tmux status lines

The tmux status line lets you show information about the current tmux session. You can also use the #() syntax to pull in content from external programs. For example, you might want to show the currently playing track from Spotify.

I recommend The Tao of tmux for a further introduction to the tmux status line (also known as the "status bar").

There are four ways the status line is updated:

1. Automatically every 15 seconds (this frequency can be changed with status-interval in your tmux configuration).
2. When you move between panes or windows.
3. After a command finishes in a tmux-owned shell pane.
4. On-demand when you invoke refresh-client (Note: passing -S to refresh-client will update only the status line).

Tmux appears to have an internal throttle that limits updating the status line to once per second (this throttle is ignored by `refresh-client`).

To test this, I set my status-right in my tmux conf to be set -g status-right '#(date >> /tmp/debug.txt)'. This appends the current time to a temporary file whenever the status line is updated. Moving from one tmux window to another slowly, you can see that you get a new line written to the debug file for each window change. I then ran ruby -e '100.times { system("tmux next-window") }' to cycle through tmux windows 100 times. Only one line was written to the debug file per second.

The status line runs #() commands asynchronously, but because your status line can potentially be updated every second, you should still keep the contents speedy. Avoid content that makes network calls or is cpu intensive. Where network calls or expensive computation are unavoidable, consider throttling or caching.

hub ci-status

hub lets you "do everyday GitHub tasks without leaving the terminal."

Running hub ci-status with no options will return pending, failure, success, etc. to indicate the status of the GitHub checks associated with your branch.

Running hub ci-status -v will show each current check with the associated status.

✔︎       unit-tests              https://url-to-see-unit-test-details/
●       e2e-tests               https://url-to-see-e2e-test-details/

You can see additional options with hub ci-status --help.

ci-status-indicator

Showing the current GitHub checks results in the tmux status line requires a little massaging. The tmux status line doesn't support ANSI colors. If it did, we could reach for hub ci-status -f "%sC%t " which prints out the names of each check color-coded depending on state (failure = red, pending = yellow, success = green).

We could do some clever work to translate the ANSI color output of hub to tmux colors, but, since the number of colors here is limited, I'm going to use a lazy sed command.

hub ci-status -f "%S, %t " | sed 's/failure, /#[fg=red]/g; s/success, /#[fg=green]/g; s/pending, /#[fg=yellow]/g'

That gets us close, but we can do better. The -tests part of the GitHub check name isn't adding any value, so we can remove it. Finally, I'll use a white | to delimit the checks rather than a space.

Since this command is getting long, let's extract it to a script.

#!/usr/bin/env bash

hub ci-status -f "%S, %t|" | \
  gsed -r '
    s/failure, /#[fg=red]/g;
    s/success, /#[fg=green]/g;
    s/pending, /#[fg=yellow]/g;
    s/\|$//;
    s/\|/#[fg=white]|/g;
    s/-tests//g;'

This outputs #[fg=yellow]e2e#[fg=white]|#[fg=green]unit which looks like this:

Much better.

I'm using gsed here rather than OSX's built-in sed. This isn't required, but I find gsed's extended regular expressions more useful than the built-in as these regular expressions grow. brew install gnu-sed and you're good to go.

You'll likely want to customize this script to make the output more succinct as we did by removing -tests. Perhaps replacing codecov with cc or similar could help you conserve real estate.

For customization's sake, since different projects can have different checks, I like to have a copy of this script local to each project. I name it .ci-status-indicator, chmod +x it, and add .ci-status-indicator to my global gitignore. The final tmux configuration line looks like this:

set -g status-right ' #(cd #{pane_current_path} && [ -f ".ci-status-indicator" ] && ./.ci-status-indicator) #[fg=white]%Y-%m-%d %H:%M'

This runs .ci-status-indicator if it exists and also shows the time. You need to use pane_current_path because the default current working directory of the status line process is whatever directory you initially started tmux from.

Caching / Throttling

Remember what I said earlier about keeping the status line speedy and avoiding network calls? hub ci-status is pretty fast, but we can be avoid hammering the GitHub API with a little caching. I've recently been writing about a general purpose cli caching script and that script will work great here.

Since the tmux config line is already a bit long, I put the cache usage in the .ci-status-indicator script itself.

#!/usr/bin/env bash

cache tmux-ci-status --ttl 30 \
  hub ci-status -f "%S, %t|" | \
    gsed -r '
      s/failure, /#[fg=red]/g;
      s/success, /#[fg=green]/g;
      s/pending, /#[fg=yellow]/g;
      s/\|$//;
      s/\|/#[fg=white]|/g;
      s/-tests//g;'

You might also be asking yourself "Won't this get stale when I change branches?" Yes, until the cache TTL expires and the status line updates. Think for a moment about ways to avoid stale content.

Avoiding stale results

We're working against two caches here:

1. tmux's status-interval (15 seconds by default)
2. cache's 30 second TTL (customizable to whatever we want)

We'll need to solve for both.

Git hooks are the best place to start. Issuing tmux refresh-client -S in the post-checkout hook handles the status-interval cache. We could use this same git hook to first purge the cache key on every branch change. That would work fine but is wastefully throwing away our previous information.

We can be a little more clever and use the current git SHA as part of our cache key. This has the advantage of not needing to re-query GitHub if you change between known commits within cache's TTL.

#!/usr/bin/env bash

cache_key="tmux-ci-status-$(git rev-parse HEAD)"

cache $cache_key --ttl 30 --cache-status "*" \
  hub ci-status -f "%S, %t|" | \
    gsed -r '
      s/failure, /#[fg=red]/g;
      s/success, /#[fg=green]/g;
      s/pending, /#[fg=yellow]/g;
      s/\|$//;
      s/\|/#[fg=white]|/g;
      s/-tests//g;'

With a per-SHA cache and a hook-triggered instant status line refresh, we'll never see stale content from another branch/SHA.

Next steps

This works great, and hopefully you've learned several new things along the way.

There's always room for improvement, so here's something to ponder and work on if you wish: Under what circumstances can the GitHub checks results change for a single SHA? Put another way: are there scenarios where you can cache the results indefinitely and avoid pinging hub ci-status again for the SHA?

If the check's result is pending then it makes sense to query again until the check finishes (the pending state will transition to failure or success). If the results are all success then querying hub ci-status again feels wasteful. If the result is failure then you might want to re-check hub ci-status if you manually re-trigger a build (e.g. for flaky tests) but perhaps it is otherwise a final state.

I leave this as an exercise to the reader. I've not yet added this optimization since I've found the cache TTL sufficient.

Shaving that yak is tempting though. It could lead to some fun side paths (e.g. it might be neat to annotate your git history with git notes regarding the CI status of each commit).

There's no shortage of ways to automatically run your tests when files change (e.g. jest has jest --watch). I find these mechanisms useful for maintaining flow. Having Vim open on one screen and a test-runner open on another screen is fantastic.

There's not a custom runner for everything, but you might not need one. I've found that entr often gets the job done. entr lets you "run arbitrary commands when files change."

When I was writing the cli caching script, I kept git ls-files | entr bats test running. Every time I saved my script or the test, the tests would run.

The entr homepage specifies several good use-cases and how to watch for new files, etc. Give it a try.

This is part six in a series about writing a general-purpose script to cache CLI output. In this brief post we'll make our script more useful as a utility to other scripts by adding some new options.

If you want to try your hand at implementing these features, checkout 780d577 locally. I'll omit my implementation here but will link to the appropriate diffs.

--check

We can implement --check to allow checking to see if the cache is fresh per the provided --ttl and --stale-while-revalidate options. The best way to communicate this freshness is with exit status, IMHO. This will allow a shell script to use cache --check as a conditional. We'll return 0 (truthy) if the cache is fresh and 1 (falsy) if it is not.

Example usage:

fresh() {
    cache --check --ttl 1 test-cache-key
}

if fresh; then
    echo "FRESH"
else
    echo "NOT FRESH"
fi

Here's the tests for the feature:

@test "--check returns 0 if the content exists and is inside the TTL/SWR" {
  run ./cache $TEST_KEY echo 1
  [ "$status" -eq 0 ]
  [ "$output" = "1" ]

  run ./cache --stale-while-revalidate 1 --ttl 1 --check $TEST_KEY
  [ "$status" -eq 0 ]
  [ "$output" = "" ]
}

@test "--check returns 1 if the content exists but is outside the TTL/SWR" {
  run ./cache $TEST_KEY echo 1
  [ "$status" -eq 0 ]
  [ "$output" = "1" ]

  wait_for_second_to_pass

  run ./cache --stale-while-revalidate 1 --ttl 0 --check $TEST_KEY
  [ "$status" -eq 1 ]
  [ "$output" = "" ]
}

@test "--check returns 1 if the content is not yet cached" {
  run ./cache --check $TEST_KEY
  [ "$status" -eq 1 ]
  [ "$output" = "" ]
}

Between the feature description and the tests, you can probably guess the implementation. You can check out the --check implementation as you wish.

--purge

Every cache needs a good mechanism for clearing the cached content on-demand. Maybe your cache key wasn't specific enough or outside forces have made the content stale before the TTL naturally expires. Whatever your reason, we'll support cache --purge <cache-key> to remove the content currently cached at the provided key.

Here's the tests for --purge:

@test "--purge exits with 0 if the content is not yet cached" {
  [ ! -f "$CACHE_DIR$TEST_KEY" ]

  run ./cache --purge $TEST_KEY
  [ "$status" -eq 0 ]
  [ "$output" = "" ]

  [ ! -f "$CACHE_DIR$TEST_KEY" ]
}

@test "--purge exits with 0 and removes the file if the content is cached" {
  touch "$CACHE_DIR$TEST_KEY"

  [ -f "$CACHE_DIR$TEST_KEY" ]

  run ./cache --purge $TEST_KEY
  [ "$status" -eq 0 ]
  [ "$output" = "" ]

  [ ! -f "$CACHE_DIR$TEST_KEY" ]
}

Try implementing this feature and then check out my implementation afterwards.

Closing

With a few small tweaks we've made our cache script more useful. This will wrap up the series for now, but I'm sure you'll see the cache script appear as a background character in future posts.