Python tooling could be much, much faster

N.B. Ruff now supports over 200 lint rules and is used in major open-source projects like FastAPI, Bokeh, Zulip, and Pydantic. It’s also about ~50% faster than the benchmarks advertised in this blog post. Try it today!

Over the past few years, there’s been a mindset shift in JavaScript ecosystem, best summarized as: “our tools should be extremely fast”.

As projects grew in complexity, and builds slowed down, we saw the emergence of new bundlers, and even new runtimes, with the common theme being “rewrite these tools in more performant languages”.

Our current build tools for the web are 10-100x slower than they could be. The main goal of the esbuild bundler project is to bring about a new era of build tool performance. — esbuild

I sometimes call this (perhaps unfairly) the ‘Rustification’ of the JavaScript toolchain: swc is written in Rust, esbuild is written in Go, Bun is written in Zig, Rome is being written in Rust. The core developer of swc is even working on a new TypeScript type-checker, again written in Rust.

The Python ecosystem could benefit from a similar mindset shift. Python tooling could be much, much faster.

As a proof-of-concept, I’m releasing Ruff, an extremely fast Python linter, written in Rust.

Ruff is ~150x faster than Flake8 on macOS (~25x faster if you hack Flake8 to enable multiprocessing), ~75x faster than pycodestyle, ~50x faster than pyflakes and pylint, and so on.

Even a conservative 25x is the difference between ~real-time feedback (~300-500ms) and sitting around for 12+ seconds. With a 150x speed-up, it’s ~300-500ms vs. 75 seconds. If you edit a single file in CPython and re-run ruff, it’s 60ms total, increasing the speed-up by another order of magnitude.

You can try Ruff today: pip install ruff. If you have a sufficiently large Python codebase, I think you’ll be surprised. I was!

(Ruff is not a drop-in replacement for those other tools. It’s not even production-ready — it’s “rough”. See “What ruff is missing” below.)

I think Ruff is a useful anchor for framing the conversation around developer tooling in the Python ecosystem, where it could be headed, and the tradeoffs we might face.

How Ruff works

Ruff is written in Rust.

It leverages RustPython’s AST parser, and from there, implements its own AST traversal, visitor abstraction, and lint-rule logic. (Much of that logic traces back to existing tools like pycodestyle.) It supports Python 3.10, including the new pattern matching syntax.

Despite being written in Rust, Ruff is pip install-able (facilitated by maturin), just like your other command-line tools. As a user, you shouldn’t even notice that it’s not written in Python.

In building Ruff, I tried to incorporate the features I missed most when moving between JavaScript and Python:

• Ruff supports ESLint-like caching. So, if you’re working in CPython, edit a single file, and re-run Ruff, we’ll only lint that single file, linting the “entire” CPython codebase in ~60ms.
• Ruff supports TypeScript-like file watching. So, running ruff --watch path/to/src will drop you into a persistent linter that re-runs whenever your source code changes.
• Ruff supports pyproject.toml-based configuration, which is increasingly the standard in the Python ecosystem.

The Ruff hypotheses

Ruff is based on two core hypotheses:

1. Python tooling could be rewritten in more performant languages.
2. An integrated toolchain can tap into efficiencies that aren’t available to a disparate set of tools.

I’ve talked about that first hypothesis a lot. I want to spend some time on the second (which is itself inspired by Rome’s philosophy).

Let’s take Flake8 as an example. Flake8 is really a wrapper around other tools, like pyflakes and pycodestyle. When you run Flake8, both pyflakes and pycodestyle are reading every file from disk, tokenizing the code, and traversing the tree (I might be wrong on some of the details, but you get the idea). If you then use autoflake to automatically fix some of your lint violations, you’re running pycodestyle yet again. How many times, in your pre-commit hooks, do you read your source code from disk, parse it, and traverse the parse tree?

An integrated toolchain could read every file exactly once, generate the AST exactly once, and leverage that representation throughout.

This is one of Ruff’s goals: generate all violations in a single pass, and even autofix the low-hanging fruit without a noticeable performance penalty.

What Ruff is missing

Mostly: lint rules.

Though it does a similar amount of work (AST parsing, AST traversal, scope + binding tracking), and thereby should be a fair comparison for benchmarking, I’ve only implemented a small subset of Flake8’s supported checks. Even those checks are missing a few edge-cases.

This is, of course, a big limitation — though I’ve come to believe that with the advent of autoformatting (see: Black, Prettier), stylistic lint rules are becoming less and less relevant, and so I plan to keep style checks to a minimum anyway.

Ruff will crash in some specific cases (#39), though I’ve run it successfully over large, diverse Python codebases.

Unlike Flake8 and others, Ruff doesn’t support plugins. It’s not extensible. It’s also missing some aspirational features, like autofix and incremental computation in watch mode.

Tradeoffs

While Ruff is fast, there are of course benefits to writing Python developer tools in Python. For example:

1. Developers in the Python ecosystem can contribute to those tools without learning a new language.
2. Debugging is straightforward. If Flake8 or Black fail, you’ll get a Python trace, and might find yourself reading Python source code. This won’t be the case with Ruff. (I already feel this pain with Mypy, which is compiled via mypyc — so crashes don’t yield Python traces.)
3. Developers can write plugins and extensions in Python.

The first two are hard to argue with — they’re tradeoffs!

The third has some wiggle room. Bun, for example, is written in Zig, but will support writing plugins in TypeScript. Similarly, given the state of Rust-Python interoperability, it should be possible to support writing Ruff plugins in Python or Rust.

Implications

Ruff is just a linter. But any tool that’s 100, 50, or even 10x faster is worthy of consideration — or, at least, the curiosity to ask “Why?”

The question I keep asking myself is: could we take the Ruff model and apply it to other tooling? You could probably give autoformatters (like Black and isort) the same treatment. But what about type checkers? I’m not sure! Mypy is already compiled with mypyc, and so is much faster than pure Python; and Pyright is written in Node. It’s something I’d like to put to the test.

Ultimately, my goal with Ruff is to get the Python ecosystem to question the status quo. How long should it take to lint, say, a million lines of code? In my opinion: it should be instant.

And if your developer tools were instant, what would that unlock?

Published on August 30, 2022.