Lūn

Lūn runs linters and formatters. Lūn is so fast that you can use it lint and format on every keypress instead of just in a pre-commit hook or in CI.

How to use it

Lūn requires a configuration file (usually lun.toml). An entry in this file specifies a linter or formatter and which files it operates on. For example:

[[linter]]
cmd = "ruff check"
files = "*.py"

lun init can generate a configuration file for you.

Once you’ve got a configuration, just run lun run.

Why it’s so fast

Lūn is fast primarily because of three features:

1. Caching
2. Batched parallelism
3. Git awareness

Also, it’s written in Rust.

Caching

Whenever you run a linter on a file via Lūn, it saves a record (i.e., a hash) constructed from several components, including the file path, file metadata (modification time, size, permissions, etc.), and the linter command line (see the documentation for a comprehensive list). Before running any tool, Lūn first consults this cache to see if it has a corresponding entry. If so, it skips running the linter again.

Batched parallelism

Lūn runs tools in parallel, but in batches. Tools usually have some nontrivial start-up time. This might involve parsing a configuration file, but even just fork/exec can take a while. Thus, Lūn doesn’t just run one instance of each tool for every changed file, but instead batches them. Given n files that need to be linted and c cores, Lūn creates c size-balanced batches (n/c files per batch if every file is the same size).

For the actual parallelism, Lūn utilizes Rayon, or Ninja if --ninja is passed.

Git awareness

You can configure Lūn to assume that files on specific Git refs (i.e., branches, commits, or tags) are already linted and formatted. For example, if you run your linters in CI, you can safely assume that files in origin/main are in good shape. Lūn will compare files to the known good refs, and only lint and format changed files.

Commands

• lun init: create a new configuration file
• lun run: run formatters and linters
  • --check: run linters, run formatters in “check” mode (i.e., in CI)
  • --format: only run formatters
  • --ninja: use the Ninja backend
  • --staged: only run on staged files (i.e., in a pre-commit hook)
  • --watch: rerun when files are changed
• lun add: add a known tool to the configuration file
• lun clean: delete the cache

See --help for a comprehensive list.

Comparison to other approaches

• Build systems based on file modification times like Make and Ninja can run linters incrementally and in parallel, but will always re-run them when files are changed even if you’ve linted those files before (see the “switched branches” benchmark below).
• lun is a bit like lint-staged, but emphasizes caching and more advanced parallelism. It’s also written in Rust.
• lun is like qlty, but far simpler.

Benchmarks

The following benchmarks compare Lūn against Ninja by running rustfmt on the crates subdirectory of the Ruff project.

In the “clean checkout” scenario, we just run rustfmt against crates/**/*.rs with an empty cache. We run Lūn in three configurations:

• Default: just lun run
• No batching: lun run --no-batch
• Ninja: Uses lun --dry-run --ninja --no-batch to generate a Ninja configuration, then runs ninja. Reported times are just for ninja itself.

The results indicate that Lūn’s batching algorithm significantly speeds up linting.

In the “switched branches” scenario, we lint, checkout an old ref, don’t make any changes, and immediately go back to the current ref. The recorded time is just for the second run.

This sort of scenario is problematic for mtime-based build systems like Make or Ninja, but Lūn handles it easily due to its caching.

You can reproduce these results like so:

git clone https://github.com/astral-sh/ruff
cd ruff
../lun/scripts/bench.sh

Documentation

See the documentation for more information, such as how to install Lūn or build it from source.

Installation

Pre-built binaries

Via the browser

Navigate to the most recent release on the releases page and download the desired artifact(s).

Via curl

You can download binaries with curl like so. Replace X.Y.Z with the most recent version number and TARGET with x86_64-apple-darwin, x86_64-unknown-linux-gnu, or x86_64-unknown-linux-musl, and run:

curl \
  --fail \
  --location \
  --proto '=https' \
  --show-error \
  --silent \
  --tlsv1.2 \
  https://github.com/langston-barrett/lun/releases/download/vX.Y.Z/lun-TARGET.gz | \
  gunzip --to-stdout > lun

From source

To build from source, you’ll need Rust and Cargo. Follow the instructions on the Rust installation page.

Via Cargo

Install the latest, unreleased version with:

cargo install --locked --git https://github.com/langston-barrett/lun.git lun

From a local checkout

See the developer’s guide.

Usage

It’s as easy as:

• lun init
• Add or remove linters in lun.toml
• lun run (or lun run --watch)

As a pre-commit hook

cat <<'EOF' > .git/hooks/pre-commit
#!/usr/bin/env bash
lun run --check --staged
EOF
chmod +x .git/hooks/pre-commit

In GitHub Actions

Lūn provides a GitHub action. To use it, replace SHA by the commit of the most recent release, and use:

- uses: langston-barrett/lun/.github/actions/lun@SHA

Caching

Lūn uses a cache to avoid re-running tools on files that haven’t changed. By default, the cache is stored in .lun/cache in the project root. lun clean clears the cache. Cache entries older than 30 days are automatically deleted.

Keys

There are two kinds of cache entry. They both include the following:

• File path
• File metadata, including size, owner UID and GID, and permissions (mode)
• Tool command line
• Tool working directory, if specified
• Content of the tool configuration file(s), if specified
• Output of the tool’s --version flag (if --careful is used)

mtime entries also include the file modification time. Content entries also include the hash of the file content.

Caching strategy

Essentially, Lūn operates with three “levels” of cache. From fastest to slowest:

• mtime entries (enabled by default, can be disabled with --no-mtime or mtime = false)
• content entries
• Git information (disabled by default, can be enabled with --refs or refs = [...])

For each (file, tool) pair, Lūn queries the caches from fastest to slowest. If any return a hit, the pair is skipped. Also, in the case that a faster cache does not have a hit but a slower one does, Lūn will update the faster caches so that they return a hit next time.

In detail, for each (file, tool) pair, Lūn does the following:

• If mtime is enabled, Lūn first checks if there is an mtime cache entry for the pair. If so, it skips the pair.
• Otherwise, Lūn checks for a content entry. If present, it saves an mtime entry and skips the pair.
• Otherwise, if refs are enabled, Lūn checks to see if the file is unchanged from any of those refs. If so, it saves an content entry and an mtime entry and skips the pair.
• Otherwise, Lūn runs the tool on the file (possibly in a batch with other files).
• If successful, it saves a content entry for the pair. If mtime is enabled, it also saves an mtime entry.

Configuration

The configuration file (lun.toml by default) is written in TOML.

Top-level fields

• careful (boolean, default: false): Include tool version in cache keys for more conservative caching.
• cache_size (integer, optional): Maximum cache size in bytes. Defaults to 1.25 MiB.
• cores (integer, optional): Number of parallel jobs to run. If not specified, uses the number of CPU cores.
• mtime (boolean, default: true): Use file modification times (see Caching).
• ninja (boolean, default: false): Enable or disable Ninja build file generation.
• refs (array of strings, default: []): Git refs to compare against when determining which files to check.
• ignore (array of strings, default: []): Glob pattern(s) matching files that all tools should ignore.
• linter (array of tables): Array of linter configurations, see below.
• formatter (array of tables): Array of formatter configurations, see below.

Warning configuration

• allow (array of strings, default: []): Warning names to allow (suppress).
• warn (array of strings, default: []): Warning names to warn about (print but continue).
• deny (array of strings, default: []): Warning names to deny (print and exit with failure).

[[linter]]

Each linter is defined in a [[linter]] table array.

• name (string, optional): Display name for the linter. If not specified, uses the command.

• cmd (string, required): Command to run for the linter.

• files (array of strings, required): Glob pattern(s) matching files that this linter should process.

• ignore (array of strings, default: []): Glob pattern(s) matching files that this linter should ignore.

• granularity (string, default: "individual"): How files are passed to the linter:

  • "individual": Any number of files per invocation, passed on the command line
  • "batch": All files in one invocation, not passed on the command line

• configs (array of strings, default: []): Paths to configuration files that affect linter behavior. Changes to these files invalidate the cache.

• cd (string, optional): Working directory for the linter.

• fix (string, optional): Command to run to automatically fix issues (see --fix). If not specified, uses cmd.

[[formatter]]

Each formatter is defined in a [[formatter]] table array.

• name (string, optional): Display name for the formatter. If not specified, uses the command.

• cmd (string, required): Command to run for the formatter.

• files (array of strings, required): Glob pattern(s) matching files that this formatter should process.

• ignore (array of strings, default: []): Glob pattern(s) matching files that this formatter should ignore.

• granularity (string, default: "individual"): How files are passed to the formatter:

  • "individual": Any number of files per invocation, passed on the command line
  • "batch": All files in one invocation, not passed on the command line

• configs (array of strings, default: []): Paths to configuration files that affect formatter behavior. Changes to these files invalidate the cache.

• cd (string, optional): Working directory for the formatter.

• check (string, optional): Command to run in check-only mode (no modifications). If not specified, uses cmd.

Warnings

Lūn supports various warnings. Each warning has a name and a default level. The levels are:

• allow: Do not print a warning
• warn: Print a warning, but continue
• deny: Print a warning and exit with failure

Levels can be overridden on the command line with --allow/-A, --warn/-W, or --deny/-D, or in the configuration file in the allow, warn, or deny arrays.

lun warns lists the warnings, and lun warns WARN prints the documentation for WARN.

careful

Warns when careful is not set at CLI or config level.

careful causes lun to include tool versions in its cache keys. This can add significant overhead, and so is disabled by default. However, this means that lun might not re-run a linter after the tool itself has been upgraded.

Default level: allow

In groups:

• pedantic

mtime

Warns when mtime is enabled.

mtime causes lun to skip files that have identical metadata to the last time lun was run. This can significantly speed things up and is enabled by default. However, in certain pathological cases, it can result in not running tools when they should be run (e.g., if the system time was changed).

Default level: allow

In groups:

• pedantic

no-files

Warns when a tool has an empty files array.

A tool with an empty files array will never match any files, making it effectively useless.

Default level: deny

refs

Warns when refs is used on CLI or config file.

Like mtime, refs causes lun to skip certain files. However, it implicitly assumes that the exact same version and configuration of the linter was run on the refs in question. In certain cases, it can result in not running tools when they should be run. refs is not enabled by default.

Default level: allow

In groups:

• all
• pedantic

unknown-tool

An unknown tool name passed to --skip-tool or --only-tool.

Default level: warn

In groups:

• all
• pedantic

unknown-warning

An unknown warning was passed to --allow/-A, --warn/-W, or --deny/-D, or was listed in the configuration file under allow/warn/deny.

Default level: deny

In groups:

• all
• pedantic

unlisted-config

Warns when lun finds a configuration file for a tool that is not listed in lun.toml. Tools might have different output in different configurations, and so it is important that they are re-run if their configuration changes. Tool configuration files listed in lun.toml form part of lun’s cache keys.

Default level: allow

In groups:

• all
• pedantic

cache-full

Warns when the cache is full and entries are being dropped.

Default level: allow

cache-usage

Warns when a single execution uses more than a quarter of the cache size.

Default level: warn

Build

To build and install from source, you’ll need to install Rust and Cargo. Follow the instructions on the Rust installation page. Then, get the source:

git clone https://github.com/langston-barrett/lun
cd lun

Finally, build everything:

cargo build --locked --release

This will put the binaries in target/release.

You can install the binary to ~/.cargo/bin with:

cargo install --locked --path=.

Release

To create a release:

• Create branch with a name starting with release

• Update CHANGELOG.md

• Update the version numbers in Cargo.toml files

  find . -type f -name "Cargo.toml" -print0 | \
    xargs -0 sed -E -i 's/^version = "U.V.W"$/version = "X.Y.Z"/'
  

• Run cargo build --release

• Commit all changes and push the release branch

• Check that CI was successful on the release branch

• Merge the release branch to main

• git checkout main && git pull origin && git tag -a vX.Y.Z -m vX.Y.Z && git push --tags

• Verify that the release artifacts work as intended

• Check that the crates were properly uploaded to crates.io

• Release the pre-release created by CI