Ruff on The Cloud Optimist

uv: The Best Thing to Happen to Python - and Why You Should Use It

Mon, 26 May 2025 07:30:00 +0200

Introduction

Python is surely my favorite programming language. Its simplicity allows you to develop scripts, backends, or even websites in no time at all, without racking your brains.

But there has always been one thing I disliked about Python: its packaging tools. For a long time, I stayed away from all these tools like Poetry, because frankly, I didn’t understand a thing!

And then, I made a discovery that not only showed me that packaging Python code is a very simple task, but also completely changed all my habits.

I’m talking about uv.

Created by the same talented team at Astral (who had already delighted us with their tool ruff), uv is presented as an “extremely fast” Python package installer and resolver. This tool is what you might call a game-changer: once you’ve tried it, there’s no going back!

So, what’s so special about uv? Why all the excitement? That’s what we’re going to break down together. Hold on tight, you might just fall in love!

This guide is based on my experience with uv, and the examples were tested with version 0.7.3. Depending on your configuration, especially behind certain corporate proxies, you might need to add the --native-tls option to some uv commands if you encounter SSL connection issues.

uv, what is it exactly?

In a nutshell, uv is a command-line tool that aims to replace pip, pip-tools, venv, pyenv, and even parts of virtualenv and pipx, while being much, much faster. Yes, all of that!

Like many trendy new tools, it’s written in Rust. Sorry, I should say, it’s written in ✨ Rust ✨. This largely explains its lightning-fast performance.

Preamble

Before we dive headfirst into uv and how it works, I’d first like to explain how uv handles dependencies, virtual environments, and Python version installation. This was rather confusing for me the first time I encountered it, so I think it’s important to mention it upfront.

Dependency Management

uv bases its entire configuration on your pyproject.toml file. If you were used to using a requirements.txt file, for example, be aware that uv will completely ignore it.

Since I’m nice, here’s a gist of a pyproject.toml template that I use every time I start a new Python project.

pyproject.toml

[project]
name = "project-name"
version = "0.1.0"
description = "Project description"
readme = "README.md"
requires-python = ">=3.12"

[[tool.uv.index]]
name = "your-index"
url = "[https://my-artifactory.corp/api/pypi/my-virtual-repo](https://my-artifactory.corp/api/pypi/my-virtual-repo)"
publish-url = "[https://my-artifactory.corp/api/pypi/my-local-repo](https://my-artifactory.corp/api/pypi/my-local-repo)"
default = true

[tool.ruff]
# Exclude common directories that are typically not part of the source code or are generated by tools.
exclude = [
    ".bzr",
    ".cache",
    ".direnv",
    ".eggs",
    ".git",
    ".git-rewrite",
    ".hg",
    ".mypy_cache",
    ".nox",
    ".pants.d",
    ".pytype",
    ".ruff_cache",
    ".svn",
    ".tox",
    ".venv",
    "__pypackages__",
    "_build",
    "buck-out",
    "build",
    "dist",
    "node_modules",
    "venv",
]
 
# Set the maximum line length to 127 characters.
line-length = 127
 
# Define the number of spaces used for indentation, aligning with Black's style.
indent-width = 4
 
# The minimum Python version to target, e.g., when considering automatic code upgrades,
# like rewriting type annotations
target-version = "py312"
 
[tool.ruff.lint]
# Enable Pyflakes (F) and a subset of the pycodestyle (E) codes by default.
# pycodestyle warnings (W)
# Activate Security Rules (S) to replace bandit
# Enable the isort rules (I) to replace isort
# flake8-bugbear (B)
# flake8-simplify (SIM)
select = ["F", "E4", "E7", "E9", "W", "S", "I", "B", "SIM", "PGH004"]
ignore = [] # List any rules to be ignored, currently empty.
 
# Allow auto-fixing of all enabled rules when using the `--fix` option.
fixable = ["ALL"]
unfixable = [] # Specify rules that cannot be auto-fixed, if any.
 
# Define a regex pattern for allowed unused variables (typically underscore-prefixed).
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
 
[tool.ruff.format]
# Enforce double quotes for strings, following Black's style.
quote-style = "double"
 
# Use spaces for indentation, in line with Black's formatting style.
indent-style = "space"
 
# Keep magic trailing commas, a feature of Black's formatting.
skip-magic-trailing-comma = false
 
# Automatically detect and use the appropriate line ending style.
line-ending = "auto"
 
# Update this with your package name or directory to be scanned by pytest-cov
[tool.pytest.ini_options]
addopts = "-s --no-header --cov=src --cov-fail-under=50" # force cmd flags
testpaths = [ # what directories contain tests
    "tests",
]
pythonpath = [ # what to add to the python path
    "."
]

Instead of adding your dependencies manually, you can now use uv add <package_name>, and it will be added to your pyproject.toml.

Be careful though, as some external tools still rely on a requirements.txt file. You might therefore need to run the command uv pip compile -o requirements.txt in your CI/CD pipeline.

Virtual Environment

Like me, you might have been used to using the command source .venv/bin/activate to activate your virtual environment. With uv, it’s a bit different. Your .venv will still be created, but by using uv, it will default to your virtual environment. Let’s take an example.

Where previously you would have had to do these kinds of commands to run your script:

python -m venv .venv
source .venv/bin/activate
pip install requests
python main.py

Now, uv removes virtual environment management from your workflow. Thus, the commands above will be replaced by these:

1
2

uv add requests
uv run main.py

When you add a new package to your project (with the uv add command), uv automatically takes care of creating a virtual environment if one doesn’t already exist.

Then, by running your script via uv run, uv will automatically place itself in your virtual environment.

Python Installation

You might have used pyenv to install your different Python versions. With uv, all that is a thing of the past!

When you want to start a project with a specific Python version, you have several ways to do it:

Use the .python-version file
Create a virtual environment: uv venv --python 3.11.6
Launch a specific Python version: uvx python@3.12

What you need to remember is that uv doesn’t rely on a globally installed Python version but will always try to use the version specific to your project.

Alright, I think you know enough to begin your initiation. Let’s go!

Installation

Installing uv is child’s play. You have several options, choose the one that suits you best (feel free to consult the official installation documentation if needed):

# On macOS and Linux - I recommend using this if possible
curl -LsSf [https://astral.sh/uv/install.sh](https://astral.sh/uv/install.sh) | sh

# On Windows (with PowerShell)
powershell -c "irm [https://astral.sh/uv/install.ps1](https://astral.sh/uv/install.ps1) | iex"

# With pip (if you already have a Python environment)
pip install uv

Once installed, you can update it very easily:

`1`	`uv self update`

And there you have it, uv is ready to use!

Managing Python Versions with uv

One of uv’s nice features is its ability to install specific Python versions. No more need to go through the official Python website, juggle with pyenv, or other tools.

To install the latest stable version of Python:

`1`	`uv python install`

Need a particular version? No problem:

1
2

# Install Python 3.9
uv python install 3.9

You can then use this version to run a script:

`1`	`uv run --python 3.9 python my_script.py`

As I explained a little earlier, Python versions installed by uv are not “globally” available on your system via the simple python command. To use them, you must go through uv run --python <version>, or activate them in a virtual environment created with that specific version.

uvx for Running Packages on the Fly

You might be familiar with npx in the JavaScript world? uvx (the shortcut for uv tool run) is the equivalent offered by uv. It allows you to run a Python CLI command (like ruff, black, ipython, etc.) in a temporary environment with the specified dependencies, without polluting your project or your system.

For example, to run a specific version of ruff:

`1`	`uvx --python 3.12 ruff@0.9.6 check main.py`

Or to run ipython with requests temporarily available:

`1`	`uvx --with requests -p 3.13 ipython`

Extremely practical for one-shots or for testing tools!

Project Management with uv

Alright, all these commands are very nice, but I imagine you’re wondering how to integrate uv into a new project, or an existing one.

So let me show you the full power of uv for creating and managing a Python project.

Starting a New Project

To initialize a new project with uv:

uv init

This command will create a few files for you:

.python-version: Specifies the Python version to use for this project (e.g., 3.11).
main.py: An example Python file.
pyproject.toml: The central file for your project’s configuration, including its dependencies. This is the modern standard in Python.

Now, let’s see how to manage our dependencies.

With uv, pyproject.toml becomes your source of truth for dependencies.

To add a new dependency to your project:

`1`	`uv add requests`

To add a development dependency (only for dev, like ruff or pytest):

1
2

uv add ruff --dev
uv add pytest --dev

⚠️ Very important: The first time you add a dependency, uv will generate a uv.lock file. This file contains the exact versions of all your dependencies (direct and indirect) that were resolved. This uv.lock file is crucial and MUST be committed to your GitHub repository. It ensures that the versions of your packages are the same for everyone, according to your working environment.

Once your dependencies (especially dev dependencies) are added, you can use them with uv run:

1
2

uv run ruff format .
uv run pytest

To remove a package:

# For a normal dependency
uv remove requests

# For a development dependency (note the --group dev)
uv remove ruff --group dev

In case you are using uv in an enterprise with a private index, you can configure that too!

By default, uv uses PyPI. If you work in a corporate environment (with an Artifactory or another private index), you can configure uv to use it via the pyproject.toml file.

[[tool.uv.index]]
name = "your-index"
url = "[https://my-artifactory.corp/api/pypi/my-virtual-repo](https://my-artifactory.corp/api/pypi/my-virtual-repo)"
publish-url = "[https://my-artifactory.corp/api/pypi/my-local-repo](https://my-artifactory.corp/api/pypi/my-local-repo)"
default = true

Then, you can, for example, export environment variables to authenticate.

# Environment variables must follow this format: UV_INDEX_{name}_USERNAME
# where {name} is the index name you defined in your pyproject.toml file
export UV_INDEX_YOUR_INDEX_USERNAME="delia, antoine"
export UV_INDEX_YOUR_INDEX_PASSWORD="cmVmdG**************************FJUjNw"

Feel free to consult the uv documentation on indexes for more details.

Migrating an Existing Project to uv

Do you have a project with a good old requirements.txt? Migration is quite simple:

If you don’t have a pyproject.toml, create one (feel free to use the one provided above).
Run the following commands to add your requirements to your pyproject.toml: uv add -r requirements.txt and uv add --dev -r requirements-dev.txt.
Once all dependencies are migrated, you can delete your old requirements files.

If you were using other tools like Poetry, migration tools exist (such as: uvx migrate-to-uv).

And there you have it, your project is powered by uv!

Joining a Project That Already Uses uv

If you clone a project that is already managed by uv (so it should have a pyproject.toml and a uv.lock), setting it up is incredibly simple. You just need to run:

uv sync

This magic command will read the uv.lock file and install exactly the same versions of all the packages listed there. Isn’t that beautiful?

Build and Publish Your Project

uv doesn’t stop there and also offers commands for building and publishing.

To build your package:

`1`	`uv build`

To publish to PyPI (or a private index):

1
2
3

uv publish
# For a private index, you might need to specify the URL (unless already specified in your pyproject.toml)
# uv publish --repository-url [https://my-artifactory.corp/api/pypi/my-local-repo](https://my-artifactory.corp/api/pypi/my-local-repo)

And to quickly test if your freshly built package installs and imports correctly:

1
2

# Replace <MY_PACKAGE> with your package name
uv run --with <MY_PACKAGE>-<VERSION>.whl --no-project --python -c "import <MY_PACKAGE>"

And there you go! You’ve published your package in the blink of an eye!

Cleaning Your Cache

Over time, uv (just like pip) accumulates a cache of downloaded packages. While this allows for quick installation of your dependencies across multiple projects, in the long run, it could take up quite a bit of space on your computer. To clean it, simply run the following command:

`1`	`uv cache clean`

Conclusion

As you’ve understood, uv isn’t just “another package manager.” It’s a real breath of fresh air in the Python ecosystem.

Speed: This is the first thing that shocks you when you use it. Installations, resolutions, everything is incredibly faster than pip. On large projects, the time savings are phenomenal.
Unification: uv brings together functionalities that previously required multiple tools (pip, venv, pip-tools, or even pyenv for basic needs). Having a single, coherent interface greatly simplifies the workflow.
Compliance: The uv developers base all their choices on Python guidelines (those famous PEPs). You can therefore be sure that your pyproject.toml respects modern Python packaging standards.
Reliability: The lockfile system (uv.lock) ensures reproducible builds, an essential point for teamwork and continuous integration.

Since I started using uv, my interactions with Python dependency management have become faster, simpler, and more enjoyable. It’s the kind of tool that, once adopted, makes you wonder how you ever managed before.

So, if you’re looking to modernize your Python usage and gain productivity (and peace of mind!), I can only encourage you to give uv a try. Trying it is very often adopting it!

References

Official uv documentation: https://docs.astral.sh/uv
uv GitHub repo: https://github.com/astral-sh/uv
A year of uv: pros, cons, and should you switch? (bitecode.dev)
uv tricks (bitecode.dev)

Bonus: uv cheatsheet

Here’s a list of super handy uv commands. A cheatsheet, if you will. Feel free to come back and consult it as needed!

# Update uv
uv self update

# Run a script with a specific Python version
uv run --python 3.12.3 main.py

# Quickly run a tool with a specific version
uvx --python 3.12 ruff@0.9.6 check

# Quickly run ipython with requests installed
uvx --with requests -p 3.13 ipython

# Update your project's version
uv version --bump [major/minor/patch]

# Clean the cache
uv cache clean

How We Built a CI/CD Strategy That Onboards 100+ Python Projects in Under a Minute

Sun, 19 Jan 2025 16:45:00 +0200

We all know CI/CD is important. In fact, it seems impossible to imagine a world where we would ship a project without checking the quality of its code, and having a detailed test suite. Moreover, to enable developers to focus on development, all of this should be automated.

Well, things are not always so easy.

Today, I’ll share how we transitioned from having no CI/CD strategy to onboarding 100+ Python projects in under a minute.

The Real World

When I joined my current company, I noticed we were managing a large number of Python projects. But when I tried to check for the CI/CD of these, well it was a bit of a mess. Some projects had CI/CD in place, but not too many. And the ones with one were not using the same rules to verify code compliance. It was clear that putting in place a CI/CD for any given project was treated as best effort. And, to be honest, I cannot blame this on anyone.

Indeed, while CI/CD is widely recognized as a core component of any project, implementing it in the real world often proves challenging.

But I knew we could change that somehow. So before jumping into a strategy to put in place, I wanted to observe what had prevented people to implement a CI/CD to begin with.

Lack of Permissions

The first thing that I realized, is that not all developers had the same level of access to our Jenkins instance. So while some were able to create new pipelines for their projects, some couldn’t. In large organizations, it is not uncommon to have this kind of scenario.

Lesson learned: a lack of permissions should not be a drawback to use a CI/CD pipeline.

Lack of Knowledge

When people had the ability to create a pipeline, some did not, as they simply did not have the proper knowledge to do so. Here, I need to mention that our company uses a Jenkins instance, so developers familiar with other CI/CD tools (e.g., GitHub Actions, CircleCI) could not fully transfer these knowledge to Jenkins. Moreover, we did not have a clear documentation on the process to follow to create a new pipeline, so beginners, fearing to break something, would simply do not take the risk to mess with it.

Lesson learned: we should ensure people without proper Jenkins knowledge can use a CI/CD pipeline.

Lack of Time

It takes effort to put in place a CI/CD at the beginning of a project, something that might be overlooked by managers that want to ship a product as fast as possible. Moreover, it is sometimes difficult to quantify the return on investment of putting in place a CI/CD pipeline. And if it is difficult to prove this can bring business value, it will end up in the “we’ll deal with this later” box. And we know all too well that the tasks that end up in this box will never see the light of day again.

Lesson learned: setting up a CI/CD pipeline for a new or existing project should be easy and straightforward.

Lack of Clear Guidelines

Finally, I had a look at the projects that did have a CI/CD pipeline. They were working fine, but I could clearly see that they lacked a common vision. Some of them used the same formatter (black), but they not always used the same line-length. Some included a testing stage, some didn’t. Not only did this led to projects not having the same code quality and compliance, I also thought that this could potentially led to confusion for a newcomer, not knowing which standard to use.

Lesson learned: people should use the same CI/CD pipeline to ensure everyone follows the same guidelines (and we should ensure these guidelines are documented somewhere).

The Global Vision

After reviewing what could go wrong, it is now important to think of a solution that could address all of these, while following the company’s best practices, and using the tools at our disposal.

Formatting

The first stage should use a formatter to ensure every line of code in our codebase looks the same. This makes sure we are not ending up with different coding standards across our projects.

Previously, we used black as our formatter. But after hearing all the good news and testing the new cool kid in the block, we decided to switch to ruff, as it has the same benefits as black, but with a faster execution.

Linting

The next stage should use a linter to find potential issues with our code. This makes sure we avoid complexity in our code, as well as identifying code smells or security issues.

In the past, I used flake8 a lot. But given that we were already using ruff, and as it can also act as a linter, it was a no-brainer to keep it for this task.

There are many rules that ruff can apply. We decided to use some of them by default, while letting developers the choice to update the ones their project would follow.

Unit Tests

Tests are a critical part of the development of any project. It ensures we ship quality code to production, while also being able to trust that our code would run just fine.

We decided to use pytest to run these tests. The default code directory would be called src, and all tests should be in a tests folder, with files being prefixed by test_.

Code Coverage

Closely related to unit tests, code coverage ensures we are able to know how much of our code has been tested. This could quickly tell us if we sufficiently tested our code, as well as pointing out the remaining lines to cover.

As we were using pytest, we decided to use pytest-cov to generate a coverage report, as it integrates nicely with pytest.

We set the minimum coverage threshold at 50%. Anything lower would risk overlooking significant portions of code, while setting it higher might discourage developers from writing the necessary tests.

Organization Folders for Jenkins

We had our different stages ready. Now all we needed to do, was find a way to globally apply said pipeline to our Python repositories. So I tried looking for a way to easily do that in Jenkins.

That’s when I stumbled upon Organization Folders.

Organization Folders are designed for scenarios like ours: automatically scan an organization (as in, a GitHub organization), filter the repositories you want, and apply a Jenkins pipeline to them.

In our example, we are able to look for all repositories with the “python” topic, and identify them as Python projects. They will then be automatically built. If a new repository is created with this topic, it will also get picked up by Jenkins.

So, in less than 5 seconds, your project could be onboarded, without having to create it in Jenkins. All is done automatically so you can focus on your code.

Examples and Documentation

All of this was great, but I was fearing of one last obstacle. What would happen if developers adopted the CI/CD pipeline only to find stages failing, with no clear documentation to resolve issues? They would probably give up or try to fix it later, which would destroy the initial goal.

So I knew that if I wanted to onboard people in this, we needed to deliver clear documentation with direct examples, so they would be able to understand why these errors might appear, and how to fix them.

Especially on the unit tests stage, as I know this is always a daunting task to start with. So I prepared a project in advance with some unit tests that I knew they could have a look at to take some inspiration, or that directly covered some tricky parts (mocking boto3 API calls, etc.).

The last step was to make a presentation on all of the above. This was key to give meaning to people, so they could really understand the point of doing all this, while making sure they had all the keys to be autonomous.

What We Have Today

From a developer’s view, all he has to do to get his Python project onboarded, is to add the python topic in his repository, and ensures a pyproject.toml file is created at the root of the repository.

These two requirements are here to tell Jenkins which project it should take into account. Moreover, the pyproject.toml file is mandatory for the ruff stages in the pipeline.

With that done, his Python project will now check for formatting issues, linting errors, validation of unit tests, and code coverage.

All in all, we are now able to setup CI/CD pipelines in less than a minute, whether you have a new or existing project!

Some Flaws

While this simplifies things a lot, there is still room for improvements.

The pipeline cannot be enforced

Currently, this CI/CD pipeline cannot be enforced because developers can simply remove the python topic to bypass it. And while this is fine at first, as we do not want to block developers in their work, at some point, the goal is still to make sure we are applying the same best practices in all Python projects.

This might be resolved in the future by the use of GitHub Rulesets.

They essentially act quite the same as branch protection rules, expect you define these rules at the organization level.

This way, we could be able to protect our main branches for all our repositories that matches a specific custom properties, and require them to successfully pass the CI/CD pipeline before they are able to merge.

The pipeline’s stages can be ignored

The pyproject.toml is used to say to ruff which format it should apply, or which rules to follow for the linting part.

And because we are currently using the pyproject.toml inside each repository, a developer could just update the rules on his own, bypassing all the guidelines we were trying to apply in the first place.

Again, while we are allowing this for now to account for the number of fixes to resolve at first, in the end, we might want to prevent this from happening.

We could either use a common and fixed pyproject.toml file, or add it in the GitHub’s CODEOWNERS file to ensure it cannot be modified without strict approval.

What’s Next?

With these in place, we can now think about the evolution of this pipeline.

For example, our company has a SonarQube instance. We would be interested to add a stage that could scan the repository for code smells.

We are also exploring the use of mkdocs, so that projects can share a common style for documentation.

And one of my personal favorites, we might want to explore the use of uv to install requirements, as it is significantly faster than the old pip guy!

You might have also noticed that while I talked about CI/CD throughout this post, at no point do we have a step that, well, deploys anything (which basically leaves us with a CI pipeline). We are already thinking about a way to build and deploy Python packages to our Artifactory repository manager, which would finally make this a CI/CD pipeline!

Finally, we only covered Python in this post, but the same logic could apply to other types of projects. To give you an example, we are also working on a CI/CD pipeline for Terraform.

Conclusion

By tackling permissions, knowledge gaps, and inconsistent guidelines, we built a unified CI/CD strategy that now supports over 100 Python projects. It’s proof that with the right approach, automation is achievable for any organization

It’s been a long but rewarding journey!

I hope this post proved the value of CI/CD, helped you understand what could prevent it from being applied, and gave you some ideas on how to implement a similar strategy in your organization!