파이썬
Manage Python package dependencies with Poetry!
Wait, is it really this easy to manage? (Now I know)
Wait, is it really this easy to manage? (Now I know)
I began to realize how cumbersome pip is when it comes to managing Python packages. While installing packages with pip wasn’t difficult, managing them became challenging because I couldn’t keep track of the dependencies between packages when writing the requirements.txt file. So I started looking into “packages for package management” (?). I researched packages such as pipreqs and pipdeptree, but none of them fully addressed the issues I was facing.
Then I came across a tool called poetry. Since it provides Python virtual environments while making package dependency management easy, it was already widely used among Python developers. In this post, I’ll summarize my thoughts after installing and using Poetry myself.
As of this writing (September 18, 2024), you need to be running python 3.8+ or later to install Poetry.
Here’s how to install it on Windows.
I referred to this page in the official documentation. I followed the steps in the order listed in 1) Install Poetry > 3) Add Poetry to your PATH > 4) Use Poetry (2번 단계는 건너 뜀).
It can be easily installed using Homebrew
brew install poetry
You can start a Poetry project with the following command.
shellpoetry init
Enter the following options sequentially in the shell. If you leave a field blank, it will be temporarily set to the default value indicated in the brackets.
shellPackage name [test]: lets-do-poetry Version [0.1.0]: Description []: test for poetry Author [Your Name <you@example.com>, n to skip]: n License []: Compatible Python versions [^3.8]:
A file named pypoetry.toml will be created as shown below.
toml[tool.poetry] name = "lets-do-poetry" version = "0.1.0" description = "test for poetry" authors = ["Your Name <you@example.com>"] readme = "README.md" [tool.poetry.dependencies] python = "3.8.5" [build-system] requires = ["poetry-core"] build-backend = "poetry.core.masonry.api"
Running poetry init creates a virtual environment based on the default settings. If you enter the following command,
poetry env list
As configured in the example above, a virtual environment named lets-do-poetry with Python version 3.8.5 will be created.
shelllets-do-poetry-Bsf-ihqJ-py3.8 (Activated) # 해시와 함께 버전이 명시된 가상환경이 생김
💡 By default, poetry uses
virtualenv. It is also possible to switch topyenvorAnaconda. You can do this by modifying thepoetry configsettings. More on this later.
Here’s how to activate the virtual environment:
poetry shell
If you’ve already activated a virtual environment using poetry shell, you can simply type commands as you normally would.
python test.pyIf you want to run commands while the virtual environment is not active, execute poetry run python test.py.
poetry add flask
You can install packages using the command above.
flask@2.1.0, github_url#developIf you only need to install packages in the dev environment, use --dev.
poetry add --dev pytest
When you add a package for the first time, a file named poetry.lock is created. This file manages the dependencies for each package.
poetry remove flask
You can remove packages using the command above. When you remove a package, not only the selected package but also its dependencies are removed.
poetry show
Use the command above to view the list of packages currently installed in the virtual environment. If any packages are missing compared to potery.lock, they may be marked in red.
(In the example above, tomli has been removed from the virtual environment.)
To understand the dependencies between these packages and check their supported versions, use the --tree option.
poetry show --tree
To install packages based on poetry.lock, simply run the poetry install command.
When managing packages with poetry add and poetry remove, the poetry.lock file is automatically updated. However, if you’ve manually edited the TOML file and it no longer matches the lock file, you can synchronize them using the following command.
poetry lock
When I actually tried to run poetry install while the lock file and the TOML file were out of sync, the following error was displayed.
Fortunately (?), you can also extract the dependencies to requirements.txt using Poetry. Use the following command:
poetry export -f requirements.txt --output requirements.txt [--without-hashes]
-f : The format of the exported file; according to the official docs, only requirements.txt is supported. (Since the default is requirements.txt, this can be omitted.)--output: The name of the file to be exported--without-hashes: If this option is not used, the hash is included.To include development packages, use the --with option.
poetry export -f requirements.txt --output requirements-dev.txt --with dev --without-hashes
Previously, you had to create a virtual environment for each project. But with Poetry, there’s absolutely no need to do that. This is because Poetry automatically creates a virtual environment when you initialize a project. In theory, you can also freely configure the Python version.
(However, since I’ll need to install a higher Python version locally to test this, I haven’t covered it here. I’ll try it out separately.)
In some ways, it’s similar to the Python package pipdeptree. However, while pipdeptree focuses solely on identifying dependencies, Poetry makes it easy to manage them as well. This is because when you use the poetry add/remove [package] command, it automatically manages the dependent packages.
requirements.txt file.I’d like to adopt Poetry to identify package dependencies for a project already in progress, but it’s difficult to determine dependencies from the requirements.txt file I’ve been managing with pip. The top priority is to distinguish between packages that are actually intended for use in the project and those that were simply installed as dependencies. I’ll experiment with how to migrate this and post about it later.
While managing the project by fixing package versions, I hadn’t given much thought to dependencies between packages. However, since some packages might be deprecated in the future, and the versions of packages in my actual project were too low—leaving me feeling quite overwhelmed as I started worrying about version management—I was relieved to find that a good tool already exists. I plan to keep track of any issues that arise while using this in real-world projects and document them as knowledge assets.