12 posts tagged with "typechecking"

We frequently hear from developers who are excited about the new generation of checkers (Ty and Pyrefly) and want to know how they stack up against each other and the existing, established tools (Mypy and Pyright). In this comparison, we'll focus purely on performance (time to run a full check) and talk a little bit about how design choices, architecture, and features impact that latency.

Evaluating a type checker's performance presents a challenge due to many variables, including diverse evaluation metrics and varying results across different operating systems and hardware configurations. Furthermore, unlike the official test suite for typing specification conformance, there is no universally adopted benchmark for performance used by all type checker maintainers.

Nonetheless, in this blog post, we will attempt to compare speed and memory usage when checking several dozen packages from the command line. We use this performance data to catch regressions in Pyrefly changes that impact OSS packages — we previously only measured type checking performance on internal projects with Pyre1.

Before we start, we'd like to caution that these numbers are only a snapshot at the time of publication and will be out of date quickly. Performance numbers can swing wildly from release to release, because the type checkers are under active development.

At Pyrefly, we've always believed that type coverage is one of the most important indicators of code quality. Over the past year, we've worked closely with teams across large Python codebases here at Meta - improving performance, tightening soundness, and making type checking a seamless part of everyday development.

But one question kept coming up: What would it take to reach 100% type coverage?

Today, we're excited to share a breakthrough.

Pyrefly is a next-generation Python type checker and language server, designed to be extremely fast and featuring advanced refactoring and type inference capabilities. This isn’t the Pyrefly team’s first time building a type checker for Python: Pyrefly is a successor to Pyre, the previous type checker our team developed.

A lot of Pyrefly’s design comes directly from our experience with Pyre. Some things worked well at scale, while other things were harder to live with day-to-day. After running a type checker on massive Python codebases for a long time, we got a clearer sense of which trade-offs actually mattered to users.

This post is a write-up of a few lessons from Pyre that influenced how we approached Pyrefly.

When you write typed Python, you expect your type checker to follow the rules of the language. But how closely do today's type checkers actually follow the Python typing specification?

In this post, we look at what typing spec conformance means, how different type checkers compare, and what the conformance numbers don't tell you.

At time of writing, pandas is one of the most widely used Python libraries. It is downloaded about half-a-billion times per month from PyPI, is supported by nearly all Python data science packages, and is generally required learning in data science curriculums. Despite modern alternatives existing, pandas' impact cannot be minimised or understated.

In order to improve the developer experience for pandas' users across the ecosystem, we at Quansight Labs (with support from the Pyrefly team at Meta) decided to focus on improving pandas' typing. Why? Because better type hints mean:

• More accurate and useful auto-completions from VSCode / PyCharm / NeoVIM / Positron / other IDEs.
• More robust pipelines, as some categories of bugs can be caught without even needing to execute your code.

By supporting the pandas community, pandas' public API is now type-complete (as measured by Pyright), up from 47% when we started the effort last year. We'll tell the story of how it happened - but first, we need to talk more about type completeness, and how we measure it.

Empty containers like [] and {} are everywhere in Python. It's super common to see functions start by creating an empty container, filling it up, and then returning the result.

Take this, for example:

def my_func(ys: dict[str, int]):
  x = {}
  for k, v in ys.items():
    if some_condition(k):
      x.setdefault("group0", []).append((k, v))
    else:
      x.setdefault("group1", []).append((k, v))
  return x

This seemingly innocent coding pattern poses an interesting challenge for Python type checkers. Normally, when a type checker sees x = y without a type hint, it can just look at y to figure out x's type. The problem is, when y is an empty container (like x = {} above), the checker knows it's a list or a dict, but has no clue what's going inside.

The big question is: How is the type checker supposed to analyze the rest of the function without knowing x's type?

If we view a type of an expression or variable as the set of possible values it can resolve to, narrowing is the process of applying constraints on those values. For example, if you have a variable x whose contents you don’t know, an if isinstance(x, int) check will narrow the type of x to int inside the body of the if-statement.

Since Python is a duck-typed language, programs often narrow types by checking a structural property of something rather than just its class name. For a type checker, understanding a wide variety of narrowing patterns is essential for making it as easy as possible for users to type check their code and reduce the amount of changes made purely to “satisfy the type checker”.

In this blog post, we’ll go over some cool forms of narrowing that Pyrefly supports, which allows it to understand common code patterns in Python.

We’re excited to share that experimental support for Pydantic is now available in Pyrefly! This integration aims to provide a seamless, out-of-the-box experience, allowing you to statically validate your Pydantic code as you type, rather than solely at runtime. No plugins or manual configuration required!

Today we’re thrilled to announce that we have reached Beta status for Pyrefly, our open-source, high-performance tool for Python code navigation and type checking! We first announced Pyrefly back in April 2025, and thanks to incredible community feedback and a ton of work from our team, we’ve hit a significant milestone.

Because NumPy is one of the most downloaded packages in the Python ecosystem, any incremental improvement can have a large impact on the data science ecosystem. In particular, improvements related to static typing can improve developer experience and help downstream libraries write safer code. We'll tell the story about how we (Quansight Labs, with support from Meta's Pyrefly team) helped bring its type-completeness score to nearly 90% from an initial 33%.