Let's talk about uv, a possible future replacement of pip

A new contender in the Python packaging ecosystem

Photo by Ivan Aleksic on Unsplash

Charlie Marsh, the creator of Ruff, a fast Python linter, shocked again the Python ecosystem with his new open-source uv. Currently, it (almost) replaces some well-known tools in the Python ecosystem, virtualenv, pip and pip-tools. We will explore this tool in this article.

Installation

This library can be installed standalone.

# on Linux and macOS
$ curl -LsSf https://astral.sh/uv/install.sh | sh

# on Windows
$ irm https://astral.sh/uv/install.ps1 | iex

At the moment of writing, I have issues with the Windows command and I don’t know why because the path is correctly set and the executable is in the right folder 😅. Anyway, if you have issues like me, you can use pipx to install it.

$ pipx install uv

Usage

Create virtual environments

The first step when working with a Python project is to create a virtual environment. It helps us isolate dependencies from project to project. With, uv, not only the experience is similar to virtualenv, but it is much faster according to the introduction blog post. Let’s say we have a scraping project, and our root folder is scraper. Inside we can run this command to create the virtual environment.

# on Linux / macOS
$ uv venv -p $(pyenv which python) venv

# on Windows
$  uv venv -p (pyenv which python) venv

Here I use pyenv to handle the different Python interpreters I have. pyenv which python gives me the path to the current active Python interpreter. You can type the path by hand if you want, something like uv venv -p /path/to/python venv will work as well.

Pyenv: Your Linux / Unix tool to manage different Python versions

The last argument venv is the name of the folder where the environment will be created. I decided to call it venv but you can give whatever name you like. Activating the virtual environment works as usual.

# on Linux / macOS
$ source venv/bin/activate

# on Windows
$ venv\Scripts\activate.ps1

Install packages

Now that we have the virtual environment. If you recall at the beginning, I said that uv can replace pip-tools. Usually, the workflow with this tool is to create a requirements.in file where we place our main dependencies and other specific files for test, documentation, etc… This is what we will do, let’s create a requirements.in file with the following content:

httpx
parsel

httpx is an HTTP client library that is more maintained than requests nowadays with even more features like asynchronous support.

parsel is a library to extract data from HTML / XML documents.

You will also create a requirements-test.in file to define our test dependencies with the following content.

pytest
pytest-cov
respx

pytest is the the-facto library for testing in Python nowadays.

pytest-cov is a pytest plugin to measure code coverage.

respx is a companion library to httpx to test HTTP requests.

Now to install dependencies from these two files, we will use the following command

$ uv pip install -r requirements.in -r requirements-test.in

Resolved 23 packages in 160ms
Downloaded 23 packages in 185ms
Installed 23 packages in 49ms
 + anyio==4.3.0
 + certifi==2024.2.2
 + colorama==0.4.6
 + coverage==7.4.1
 + cssselect==1.2.0
 + exceptiongroup==1.2.0
 + h11==0.14.0
 + httpcore==1.0.3
 + httpx==0.26.0
 + idna==3.6
 + iniconfig==2.0.0
 + jmespath==1.0.1
 + lxml==5.1.0
 + packaging==23.2
 + parsel==1.8.1
 + pluggy==1.4.0
 + pytest==8.0.1
 + pytest-cov==4.1.0
 + respx==0.20.2
 + sniffio==1.3.0
 + tomli==2.0.1
 + typing-extensions==4.9.0
 + w3lib==2.1.2

I don’t know for you, but I find it incredibly fast! 😆

Manage dependencies

Now that we have our dependencies installed, it is often a good idea to keep all the dependencies (direct and indirect) in dedicated files to reproduce them exactly from one computer to another. For that purpose, we can use the compile command as follows:

$ uv pip compile requirements.in -o requirements.txt
$ uv pip compile requirements-test.in -o requirements-test.txt

This will create two files containing the direct dependencies written in the .in files plus the dependencies linked to them.

Now, if another user wants to replicate the same dependencies in his environment, he can use the sync command as follows:

$ uv pip sync requirements.txt requirements-test.txt

Note that you need to have created and activated the virtual environment, if not uv will stop dead in his tracks. I really like the fact that it does not try to do any operation if it is not used in a virtual environment. It is a common trap that is avoided.

Also, if you are working on an application you want to deploy, there is no need to install the requirements-test.txt dependencies.

Other features

Since he re-writes from scratch the pip command, Charlie took this opportunity to add additional features.

Alternative resolution strategies

pip tries to find the latest version when resolving packages by default. With uv, we have the opportunity to choose the lowest compatible version. For example, if we have the following requirements.in file.

flask>=2.0.2

With the normal strategy, when installing and compiling the dependencies, we will have the following requirements.txt file.

# This file was autogenerated by uv via the following command:
#    uv pip compile requirements.in
blinker==1.7.0
    # via flask
click==8.1.7
    # via flask
flask==3.0.0
itsdangerous==2.1.2
    # via flask
jinja2==3.1.2
    # via flask
markupsafe==2.1.3
    # via
    #   jinja2
    #   werkzeug
werkzeug==3.0.1
    # via flask

But if we apply the lowest resolution strategy with the command uv pip compile --resolution=lowest requirements.in, the requirements.txt would instead look like this.

# This file was autogenerated by uv via the following command:
#    uv pip compile requirements.in --resolution=lowest
click==7.1.2
    # via flask
flask==2.0.0
itsdangerous==2.0.0
    # via flask
jinja2==3.0.0
    # via flask
markupsafe==2.0.0
    # via jinja2
werkzeug==2.0.0
    # via flask

We can see that we are stuck with Flask 2.X series and the related dependencies also have the lowest possible compatible version.

Note that you can use the —resolution option on uv pip install too.

Dependency overrides

In cases where you know that a dependency which is a direct dependency of another one, can’t be upgraded because the other dependency constrained its version, but you are sure that it will still be compatible, uv provides an escape hatch to override the desired dependency. All you need to do is provide the option --override overrides.txt where overrides.txt contains the dependencies you are sure you want to bypass constraints.

Resolution against arbitrary Python versions

While pip and pip-tools always resolve against the currently installed Python version (generating, e.g., a Python 3.12-compatible resolution when running under Python 3.12), uv accepts a --python-version parameter, enabling you to generate, e.g., Python 3.7-compatible resolutions even when running under newer versions.

To be clear, if you are running uv on Python 3.11, but want to resolve for Python 3.10, you can run uv pip compile --python-version=3.10 requirements.in to produce a Python 3.10-compatible resolution.

The cargo for Python

If you read the introduction blog post for uv, you will notice that the author insists on the idea of reproducing the cargo experience for the Python community. The goal is to handle all aspects of Python installation, package management, and development lifecycle in the easiest way possible for Python developers. For that purpose, he joined forces with Armin Ronacher, the creator of Flask, a well-known Python web framework. They concluded that Charlie and his team would take over Rye, an experimental project by Armin, to handle a Python project. It is an outline of the cargo experience we are looking for. It handles Python installation, virtual environments, package installation, distribution, etc… In short, it handles the whole development lifecycle of a Python project.

The ultimate goal, from what I understood, is the merge the features of rye in uv. I’m really excited to see how things will move, but the future of Python development looks promising. Maybe one day, we will not struggle to find the best packages or ways to install Python or its packages. 😁

Note that, not everyone is happy (like always) about the start of this project, like the Hatch creator, another project in the realm of Python packaging who also wants to create a cargo experience. You can see more about the discussions in this thread.

I think there are some valid concerns, but overall, finally, we have a company with resources to tackle the Python packaging experience, something unfortunately that lacked the Python Packaging Authority for years, this is why their tools didn’t evolve a lot and created the opportunity for a fragmented ecosystem with lots of projects trying to fix the packaging world, just to name a few: pipenv, flit, poetry and pdm. These are good projects, and if you follow me for a long time, you know that I’m a huge fan of poetry.

But I think we need a standard way to handle a Python project, to help junior developers start with confidence their Python development journey, and I think, Charlie and his company can be the solution we all want, so we need to encourage him instead of hurling sterile criticism at him.

This is all for this article, hope you enjoy reading it. Take care of yourself and see you soon. 😁