How To Package And Distribute Python Applications

Introduction #

All Python libraries (i.e. application packages) that you download using a package manager (e.g. pip) are distributed using a utility dedicated to do the job. These utilities create “Python distributions” which are basically versioned (and compressed) archives. All related elements to what’s being distributed, such as source files and resource files, are contained within it.
In this DigitalOcean article, we are going to talk about the necessary tools for distribution and go over the key steps to allow you to package your own useful libraries, modules, or applications – which should help you when deploying your project on a droplet or sharing on the internet.

Python Distributions and Packages #

Even if you have worked only a little with Python, you will be familiar with the concept of using a package manager (e.g. pip, easy_install) to download modules and libraries (e.g. application development frameworks) which are then imported and used to create a new one.
These package management tools, operating locally, connect to a source (i.e. Python Package Index – PyPI) and perform a desired action (e.g. search and install) as they work these resources which are actually called Python distributions.
The way to distribute an application consists of wrapping its directory with some must-have files (along with a few recommended ones), specifying related elements (e.g. resources, dependencies etc.) and releasing it or using it elsewhere…that simple.
Note: You are highly encouraged to work with virtual environments to isolate Python downloads, modules, and applications you are working with.

Python Packages #

In Python, a package [technically] is an importable directory (with __init__.py) containing source files (i.e. modules). This shall not be confused with operating-system packages, which are [technically] actual applications (i.e. a Debian package). However, it must be noted that Python distributions are indeed called packages as well.
Example package structure:

package
  |
  |-- __init__.py

Python Applications #

Although anything from a single file to one with hundreds scattered across various packages can be considered an application in Python, in most realistic scenarios, an application will consist of multiple modules and a certain amount of external imports (from libraries).
Example application structure:

myapp
  |
  |-- __init__.py
  |-- amodule.py
  |-- anothermod.py
  |__ tests
  |     |
  |     |-- __init__.py
  |     |-- ..
  |     |-- .
  | ..

Python Distribution Tools and Libraries #

Given the popular nature of Python and the rich amount of third-party libraries / applications written for it, a simpler and unified way of distributing has always been a necessity. There have been several different tools and libraries used for creating Python distributions.
In order to deal with the tasks of distribution, Python distribution utilities toolset distutils was created.

Python Package Index (PyPI) #

Python Package Index, or PyPI, is a central [online] repository for projects (Python distributions). Package managing tools such as pip use this repository in order to host, find and install them.

Getting Started #

Let”s begin with creating a simple, general Python flask application [structure] which we then can use to package.

Creating the Application Structure #

We aim to create an example that resembles most real-world projects. Therefore, it will be best to imagine a scenario with modularised components.
Example structure:

/MyApplication
    |-- run.py
    |-- config.py
    |__ /app
         |-- __init__.py
         |-- /module_one
             |-- __init__.py
             |-- controllers.py
             |-- models.py                
         |__ /templates
             |-- module_one
                 |-- hello.html
         |__ /static
         |__ ..
         |__ .

Create the folders: #

mkdir ~/MyApplication
cd    ~/MyApplication
touch run.py
touch config.py
mkdir app
cd    app
touch __init__.py
mkdir templates
mkdir static
mkdir module_one
cd    module_one
touch __init__.py
touch controllers.py
touch models.py
cd    ../templates
mkdir module_one
cd    module_one
touch hello.html

Edit run.py using nano: #

nano ~/MyApplication/run.py

Place the contents:

# Run a test server.
from app import app
app.run(debug=True)

Save and exit using CTRL+X and confirm with with Y.

Edit config.py using nano: #

nano ~/MyApplication/config.py

Place the contents:

DEBUG = True

THREADS_PER_PAGE = 4

CSRF_ENABLED     = True
CSRF_SESSION_KEY = "secret"

Save and exit using CTRL+X and confirm with with Y.

Edit app/init.py using nano: #

nano ~/MyApplication/app/__init__.py

Place the contents:

from flask import Flask, render_template

app = Flask(__name__)
app.config.from_object("config")

from app.module_one.controllers import module_one

app.register_blueprint(module_one)

Save and exit using CTRL+X and confirm with with Y.

Edit app/module_one/controllers.py using nano: #

nano app/module_one/controllers.py

Place the contents:

from flask import Blueprint, request, render_template

module_one = Blueprint("auth", __name__, url_prefix="/auth")

@module_one.route("/hello")
def hello():
    return render_template("module_one/hello.html")

Save and exit using CTRL+X and confirm with with Y.
Place the contents:

Edit app/templates/module_one/hello.html using nano: #

nano app/templates/module_one/hello.html

Place the contents:

  
<!DOCTYPE html>  
<html lang=“en”>  
<head>  
<title>{% block title %}My Site{% endblock %}  
{% block css %}  
{% endblock %}  
<meta name=“viewport” content=“width=device-width, initial-scale=1.0”>  
</head>  
<body>  
Hello, world!  
</body>  
</html>  

Save and exit using CTRL+X and confirm with with Y.

Beginning with Application Distribution / Packaging #

Having created an exemplary application structure of a web site that uses flask, we can continue with taking the first step into preparing the distribution.

Altering the Folder Structure #

In order to package our application well, we need to make some additions to our folder structure.

/MyApplication
    |-- run.py
    |__ /app
         |-- __init__.py
         |-- /module_one
             |-- __init__.py
             |-- controllers.py
             |-- models.py                
         |__ /templates
             |-- module_one
                 |-- hello.html
         |__ /static
         |__ ..
         |__ .
    |-- setup.py    # Distribution setup file
    |-- README.txt  # Read-me file
    |-- MANIFEST.in # Distribution manifest file
    |-- CHANGES.txt # Changes log

Alter the folder structure to create necessary files:

touch ~/MyApplication/setup.py
touch ~/MyApplication/README.py
touch ~/MyApplication/MANIFEST.py
touch ~/MyApplication/CHANGES.py
mv    ~/MyApplication/run.py ~/MyApplication/bin/run

Create the setup.py #

nano ~/MyApplication/setup.py

Place the below self explanatory contents:

from distutils.core import setup

setup(
    # Application name:
    name="MyApplication",
    
    # Version number (initial):
    version="0.1.0",
    
    # Application author details:
    author="name surname",
    author_email="name@addr.ess",
    
    # Packages
    packages=["app"],
    
    # Include additional files into the package
    include_package_data=True,
    
    # Details
    url="http://pypi.python.org/pypi/MyApplication_v010/",
    
    #
    # license="LICENSE.txt",
    description="Useful towel-related stuff.",
    
    # long_description=open("README.txt").read(),
    
    # Dependent packages (distributions)
    install_requires=[
        "flask",
    ],
)

Save and exit using CTRL+X and confirm with with Y.

Create the MANIFEST.in #

If you need to ship extra directories (e.g. static or templates), you need to explicitly state them in the manifest to be packaged. We will do this inside the MANIFEST.in.

nano ~/MyApplication/MANIFEST.in

Place the below self explanatory contents:

recursive-include app/templates *
recursive-include app/static *

Save and exit using CTRL+X and confirm with with Y.
And that’s it! Your Python distribution package is ready to be installed and shipped.

Additional Files #

Please remember that in order to have a complete distribution, your file/directory must contain (and linked):

README.txt

MANIFEST.in

LICENSE.txt

Working With the Distribution Ready Application #

As we have finalized creation of our application followed by making necessary amendments to the file structure to prepare it for a flawless distribution build, we can begin with going through the packaging operations.

How to Create The Distribution File #

In order to generate a distribution file copy, run the following:

cd     ~/MyApplication
python setup.py sdist

This command will go through your setup, print out the operations being performed and generate a tar archive inside the newly created dist directory, similar to:

# root@hostname:~/MyApplication# ls dist
# MyApplication-0.1.0.tar.gz

Note: Since we did not populate all the sub-folders (i.e. static) and worked with additional files (e.g. README.txt), you might see some warnings during the creation process.

How to Install The Application #

From now on, your application can be installed and used by others using the setup.py file created.
In order to install the application, run the following:

python setup.py install

If this installation is for development and the requirements are also to be installed, run the following:

python setup.py develop

How to Share Your Application #

If you would like to share your code on the Python Packaging Index, you can do so by initiating the “register” procedure as per the following:

python setup.py register

You will need to complete the procedure by following the on-screen instructions.
If you have a registered login, in order to just upload, you can use the following:

python setup.py sdist upload

How to Create Packages of Your Application’s New Versions #

Edit the setup.py file with a text editor (e.g. nano) and set the new version number: version="0.1.1"

Edit the CHANGES.txt to reflect the changes

Make the necessary adjustments to the LICENSE.txt and README.txt

Upload your code following the previous step.