Create an executable from your python program
One thing notable when working with Python is that it is not easy to install and run a program written in this programming language on a random computer. This is normal since it is an interpreted language, so we need to have an interpreter program (the most known is CPython, remember that many other Python implementations like PyPy exist) installed on the target machine where we want to run the program.
But in this article, I will present two projects solving this issue with a different approach: pyinstaller and nuitka.
Installation
Pyinstaller starts working with Python 3.8 and Nuitka still supports Python 2 projects!
$ pip install pyinstaller nuitka
# with poetry
$ poetry add pysinstaller nuitka -G dev
If you don’t know poetry, I have a nice introduction here.
You may have an incompatibility error when installing pyinstaller with poetry. This is because pyinstaller narrowed the version of Python you can install with an upper constraint forbidding a Python version not available. This condition is <3.13
at the moment of writing. On the other hand, if you create your poetry project with the defaults provided, you will have a Python constraint in this form ^<python version>
which allows versions greater than pyinstaller upper bound and poetry complaints. The solution is to also narrow your Python version in the poetry configuration, something like >3.8,<3.13
.
[tool.poetry.dependencies]
python = ">3.8,<3.13" # <-- changed here
Usage
The basic usage is simple. Consider this simple program greeting a user passing his name as input.
# hello.py
import sys
if __name__ == '__main__':
print(f'Hello {sys.argv[1]}!')
If you run this program in your terminal, you will see a message greeting the user you enter.
$ python hello.py Kevin
Hello Kevin!
Now let’s see how to run this program without invoking the interpreter, you know the python
we tape first in the command.
Pyinstaller
All we need to do is call pyinstaller with the program file.
$ pyinstaller hello.py
It will create one file and two folders:
hello.spec
: The configuration file it uses to build the application. You can specify in this file some metadata like non-Python files to add before building the application. For more information, read the appropriate documentation page.build
: this folder is where some intermediary processing takes place.dist
: the final folder containing the executable. It should be found in the dist/hello folder.
So to run the application as before, you can type:
# on Linux/Mac
$ ./dist/hello/hello Kevin
# on Windows
$ .\dist\hello\hello.exe Kevin
We can distribute this folder to other users and they can run the program without needing a Python interpreter, Houra!
If you don’t want to distribute a folder but just a single binary, you should add the —onefile
flag when calling pyinstaller. This way it will bundle all the necessary files into the executable, so there will be no need to have an interpreter on the target machine.
Before proceeding with the following command, I encourage you to check the size of the executable you have now. On my Windows 11 laptop, it is approximately 1.5 MB. On my embedded Ubuntu (WSL) it is less than 1 MB.
$ pyinstaller --onefile hello.py
Now the executable can be found directly in the dist folder. You can run it as before:
$ .\dist\hello Kevin
hello Kevin!
This file can be distributed to other users. This file is approximately 6.1 MB on my Windows and 5.8 MB on my embedded Ubuntu.
Notes:
You can change the name of the executable by passing the
—name
option.On Windows and macOS, if you are building a graphical application, you can pass the
—windowed
option to have an installer provided with the executable.Pyinstaller is not a cross-platform compiler, say another way, if you want to produce executables working on Windows, macOS, and Linux, you have to compile the project on each of those three platforms. An executable produced on Windows will not work on macOS, and so on.
Nuitka
Before trying to run the following command, I recommend you delete all the previous files created since nuitka creates folders with names almost similar to some we have seen above. 😜
The usage is pretty straightforward:
$ python -m nuitka hello.py
It may install some dependencies useful for building the project. The result file should be in the same folder where hello.py is. On Windows, it is called hello.exe
, on other platforms, it can be called hello
or hello.bin
. I will just use hello from now on.
$ .\hello Kevin
hello Kevin!
Awesome, we have a standalone binary we can copy on other computers… Not really. In fact, the executable produced still depends on the interpreter nuitka uses during the build. This is most obvious on Windows where a hello.cmd
file resides next to the executable and clearly shows how the interpreter path is kept for use. I will show you in a few seconds how to get rid of it but before that let’s look at the folder created by nuitka, hello.build
.
This folder contains C files and this is the fundamental difference with pyinstaller. Nuitka tries to transpile all Python files into C files before building the executable. The goal is to produce the smallest binary possible. I don’t know all the details, but it seems like it tries to create C instructions like what can be found in the CPython interpreter.
Now, let’s create a standalone executable not depending on an interpreter. We need to add one flag as follows.
$ python -m nuitka --standalone hello.py
Nuitka will maybe prompt you to install some caching tools, say yes, or install them (I needed to install patchelf
and ccache
dependencies on my Ubuntu). Now you have a folder hello.dist
containing the executable you can distribute to other users.
$ .\hello.dist\hello Kevin
hello Kevin!
If you don’t want a folder but a single file, add the —onefile
flag to the command.
$ python -m nuitka --standalone --onefile hello.py
Now you must have a hello binary alongside the Python file. You can run it as previously:
$ .\hello Kevin
hello Kevin!
This binary is about 3.7 MB on my Windows (smaller than with pyinstaller) and 3.3 MB on my embedded Ubuntu.
Notes:
There is a folder
hello-onefile.build
alongside the executable, it was necessary to create the executable but you don’t need to distribute it :)In more realistic cases where you have many modules/packages and an entry point Python file, you probably want to add the option
—follow-imports
to bundle all the files in the executable.
For more information, I recommend you to read this documentation page.
A realistic use case
To see these projects in action in a more interesting use case, we will consider this small project I created when writing my previous article.
Feel free to clone it to follow what I will do. It is a command line interface to transcribe audio files and add subtitles to videos. Here are some examples of how to use it.
$ whp --help
Usage: whp [OPTIONS] COMMAND [ARGS]...
A command line interface to transcribe your audio files and create subtitles
for your videos.
Note that you will need ffmpeg (https://ffmpeg.org/) installed on your machine to run this command.
Example usage:
# transcribe an audio file in spanish in all formats supported
# this will create many files like audio.json, audio.srt, etc...
$ whp audio transcribe audio.mp3 -l es
# transcribe multiple audio files. They must have the same source language.
$ whp audio transcribe audio1.mp3 audio2.wav -l es
# transcribe an audio file using the medium model
$ whp audio transcribe audio.mp3 -l en -m medium
# transcribe an audio file in json and srt formats
# this will create files audio.json and audio.srt
$ whp audio transcribe audio.mp3 -f json -f srt
# extract an audio file from a video file
$ whp video ea video.mp4 -o audio.mp3
# create video with subtitles
$ whp video subtitles video.mp4 -o video_with_subtitles.mp4
# specify the whisper model to use and the source language of the video
$ whp video subtitles video.mp4 -l en -m medium -o video_with_subtitles.mp4
Options:
--version Show the version and exit.
-h, --help Show this message and exit.
Commands:
audio Audio related subcommands.
install-completion Install completion script for bash, zsh and fish...
video Video related subcommands.
Now let’s say we want to distribute it to users on Windows, macOS, and Ubuntu-based OS.
The first step is to try to build the executable locally before trying to automate the process with a CI/CD platform.
This project uses click to create the command line. I wrote an article on it if you are curious. The entry point of the program is the whisperer/main.py
module.
Pyinstaller
The entry point of the program is the whisperer/main.py
module. If you look at its content, you will see this at the end of the file.
if __name__ == '__main__':
cli()
If you try to run this file with Python, it will be the same as invoking the command whp
.
$ python -m whisperer.main -h
Usage: python -m whisperer.main [OPTIONS] COMMAND [ARGS]...
A command line interface to transcribe your audio files and create subtitles
for your videos.
Note that you will need ffmpeg (https://ffmpeg.org/) installed on your machine to run this command.
Example usage:
# transcribe an audio file in spanish in all formats supported
# this will create many files like audio.json, audio.srt, etc...
$ whp audio transcribe audio.mp3 -l es
# transcribe multiple audio files. They must have the same source language.
$ whp audio transcribe audio1.mp3 audio2.wav -l es
# transcribe an audio file using the medium model
$ whp audio transcribe audio.mp3 -l en -m medium
# transcribe an audio file in json and srt formats
# this will create files audio.json and audio.srt
$ whp audio transcribe audio.mp3 -f json -f srt
# extract an audio file from a video file
$ whp video ea video.mp4 -o audio.mp3
# create video with subtitles
$ whp video subtitles video.mp4 -o video_with_subtitles.mp4
# specify the whisper model to use and the source language of the video
$ whp video subtitles video.mp4 -l en -m medium -o video_with_subtitles.mp4
Options:
--version Show the version and exit.
-h, --help Show this message and exit.
Commands:
audio Audio related subcommands.
install-completion Install completion script for bash, zsh and fish...
video Video related subcommands.
Invoking the program python -m whisperer.main
allows Python to take into account the context of the whisperer package and avoids import errors.
Now let’s create the executable for the program. It is a good practice to bundle a folder first. This allows you to see what is put in the executable and debug in case of issues.
$ pyinstaller whisperer/main.py
You should have an executable main
(I don’t add the possible exe or bin extension but you should consider it) in the dist/main
folder. You can cd to this directory and run the following command to extract audio from a video.
# replace video.mp4 with a valid video
$ main video ea video.mp4 -o audio.mp3
processing video.mp4 ⚙
Done! 🎉
If you have an output like the previous one, congratulations!
Note: I discovered, while writing this section of the article that pyinstaller doesn’t recognize relative imports, so I had to change them where they were used.
For example, before the main.py
module imported commands like this:
from .commands.audio import audio
from .commands.completion import install_completion
from .commands.video import video
I changed it to this 😁
from whisperer.commands.audio import audio
from whisperer.commands.completion import install_completion
from whisperer.commands.video import video
Ok, now that we know that it works, we don’t want the program to be called main. The command line interface is called whp and we want to maintain this name. Also, we want a single file which is more convenient to handle than a folder, so we run the following command to have a standalone executable.
$ pyinstaller --onefile --name whp whisperer/main.py
The final executable can be found in the dist folder. Perfect!
Nuitka
So to create an executable with nuitka we can run
$ python -m nuitka --follow-imports --standalone --onefile whisperer/main.py
Except that it will not work. 🙃
Unfortunately, nuitka doesn’t work with libraries doing Just-In-Time compilation like numba or triton. More information can be found in this issue.
This is the paradox I have with nuitka, I prefer it because it produces smaller executables than pyinstaller, on the other hand, it does not work in all cases like pyinstaller, so you should be aware of that and try both projects for your program.
Automate the creation of executables
Now that you know how to create an executable locally, let’s try to automate the process through a CI/CD platform. I will show you an example of what can be done using GitHub Actions, but it can be adapted to your CI/CD provider without much difficulty.
name: Publish
# run on each tag, you can change the strategy if you want
on:
push:
tags:
- '*'
jobs:
publish:
timeout-minutes: 10
runs-on: ${{ matrix.os }}
strategy:
max-parallel: 6
matrix:
os: [ 'ubuntu-latest', 'macos-latest' ,'windows-latest' ]
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.11
uses: actions/setup-python@v4
with:
python-version: 3.11
- name: Install python pre-requisites
run: python -m pip install -U pip
- name: Install poetry
run: pipx install "poetry<1.8.0"
# I don't have a macOS, so I don't know the dependencies needed
# you will have to find them yourself :)
- name: Install Ubuntu dependencies
run: apt-get install patchelf ccache gcc
if: matrix.os == 'ubuntu-latest'
# we run tests to ensure everything is OK
- name: Run tests
run: pytest
# when everything is OK, we build our executable, I assume pyinstaller is
# installed by poetry
- name: Build executable
run: pyinstaller --onefile --name whp whisperer/main.py
- name: deploy to a store / package repository
run: echo "Since the method depends on the platform, it is up to you to see how to do it :D"
Before ending, I want to mention another library I discovered recently, PyApp. Roughly said, it does the same job as the two projects I showed you but I don’t find it appealing because it works a bit like pipx.
It tries to create a virtual environment in the target machine before running from what I understood. Since we don’t know the constraints we can have on a target machine, this solution is not optimal.
This is all for this article, hope you enjoy reading it. Take care of yourself and see you soon. 😁