Python Build and Deployment Tools Guide

Python has become one of the most popular programming languages for web development, data science, automation, and cloud-native applications. However, building and deploying Python applications presents unique challenges compared to other languages. This guide covers the essential Python build tools, deployment options, and provides decision matrices to help you choose the right approach for your project.

Python Build Tools Overview

Python’s build ecosystem has evolved significantly over the years, moving from traditional setup scripts to modern standardized packaging formats.

1. setuptools

setuptools is the most widely used packaging library for Python. It provides the foundation for building and distributing Python packages.

Key Features:

• Creates source distributions (sdist) and wheel distributions (bdist_wheel)
• Supports dependency management
• Allows inclusion of non-Python files
• Integrates with easy_install (legacy) and pip

Basic Usage:

# setup.py
from setuptools import setup, find_packages

setup(
    name="my-package",
    version="0.1.0",
    packages=find_packages(),
    install_requires=[
        "requests>=2.28.0",
        "numpy>=1.24.0"
    ],
    extras_require={
        "dev": ["pytest", "black"],
        "docs": ["sphinx"]
    },
    python_requires=">=3.8",
)

Building:

# Create source distribution
python setup.py sdist

# Create wheel distribution
python setup.py bdist_wheel

# Install in development mode
pip install -e .

2. pip

While not a build tool per se, pip is essential for installing and managing Python packages.

Key Features:

• Installs packages from PyPI and other indexes
• Manages virtual environments
• Handles dependencies automatically

Common Commands:

# Install package
pip install package-name

# Install from requirements file
pip install -r requirements.txt

# Upgrade package
pip install --upgrade package-name

# List installed packages
pip list

# Export dependencies
pip freeze > requirements.txt

3. poetry

poetry is a modern dependency management and packaging tool that simplifies the entire Python development workflow.

Key Features:

• Unified dependency management (runtime + dev)
• Lock file for reproducible builds
• Built-in virtual environment management
• Simple CLI interface

Installation:

# Using pipx (recommended)
pipx install poetry

# Or using curl
curl -sSL https://install.python-poetry.org | python3 -

Basic Workflow:

# Create new project
poetry new my-project

# Add dependency
poetry add requests

# Add dev dependency
poetry add --group dev pytest

# Build package
poetry build

# Install dependencies from lock file
poetry install

# Run command in virtual environment
poetry run python script.py

pyproject.toml example:

[tool.poetry]
name = "my-package"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]

[tool.poetry.dependencies]
python = "^3.8"
requests = "^2.28.0"

[tool.poetry.group.dev.dependencies]
pytest = "^7.0.0"
black = "^22.0.0"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

4. pyproject.toml

The pyproject.toml file is the standardized build configuration file introduced in PEP 518 and standardized in PEP 621.

Benefits:

• Single source of truth for project metadata
• Tool-agnostic format (works with setuptools, poetry, hatchling)
• Human-readable TOML format

PEP 621 Standard Metadata:

[project]
name = "my-package"
version = "0.1.0"
description = "A sample Python package"
readme = "README.md"
requires-python = ">=3.8"
dependencies = [
    "requests>=2.28.0",
    "numpy>=1.24.0"
]

[project.optional-dependencies]
dev = ["pytest", "black"]
docs = ["sphinx"]

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

# For poetry compatibility
[tool.poetry]
name = "my-package"
version = "0.1.0"

Comparison of Build Tools

Feature	setuptools	pip	poetry
Dependency Management	Basic	Yes (with requirements.txt)	Full (lock files, virtual envs)
Virtual Environment	No	Manual or venv	Built-in
Lock Files	No	Manual with requirements.lock	Yes (poetry.lock)
Build System	Yes	No	Yes
Modern Standards	Legacy	Standard installer	PEP 621 compatible
Configuration Format	Python	CLI flags / INI	TOML

Deployment Tooling

Docker

Docker is the most popular containerization platform for deploying Python applications.

Basic Dockerfile:

FROM python:3.11-slim

WORKDIR /app

# Install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY . .

# Create non-root user
RUN useradd -m appuser
USER appuser

# Expose port
EXPOSE 8000

# Run application
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Docker Compose for Multi-Service Apps:

version: '3.8'

services:
  web:
    build: .
    ports:
      - "8000:8000"
    environment:
      - DATABASE_URL=postgresql://user:pass@db:5432/mydb
    depends_on:
      - db
  
  db:
    image: postgres:15
    environment:
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=pass
      - POSTGRES_DB=mydb
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

AWS SAM (Serverless Application Model)

AWS SAM is a framework for building serverless applications on AWS.

SAM Template Example:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:
  PythonFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: app.lambda_handler
      Runtime: python3.11
      CodeUri: src/
      Timeout: 30
      MemorySize: 512
      Environment:
        Variables:
          ENVIRONMENT: production
          LOG_LEVEL: info
      Events:
        ApiEvent:
          Type: Api
          Properties:
            Path: /python-function
            Method: get

Outputs:
  ApiUrl:
    Description: URL of the deployed function
    Value: !Sub "https://${ServerlessApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/python-function"

Building and Deploying:

# Build the application
sam build

# Validate template
sam validate

# Deploy to AWS
sam deploy --guided

# Local testing
sam local start-api

Azure Functions CLI

Azure Functions allows you to run Python code serverlessly on Microsoft’s cloud platform.

Project Structure:

function-app/
├── host.json
├── local.settings.json
├── requirements.txt
└── functions/
    ├── __init__.py
    └── function_app.py

host.json:

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[4.*, 5.0.0)"
  }
}

Python Function Example:

import azure.functions as func
import logging

app = func.FunctionApp()

@app.route(route="python-function")
def python_function(req: func.HttpRequest) -> func.HttpResponse:
    logging.info("Python HTTP trigger function processed a request.")
    
    name = req.params.get('name')
    if not name:
        try:
            req_body = req.get_json()
            name = req_body.get('name')
        except ValueError:
            pass
    
    if name:
        return func.HttpResponse(f"Hello, {name}!")
    else:
        return func.HttpResponse(
            "Please pass a name on the query string or in the request body",
            status_code=400
        )

Building and Deploying:

# Install dependencies
pip install -r requirements.txt

# Local development
func start

# Build for deployment
func azure functionapp publish <function-app-name>

# Or deploy using Azure CLI
az functionapp deployment source config-zip \
    --resource-group <rg-name> \
    --name <function-app-name> \
    --src <zip-file>

Decision Matrix: Choosing the Right Tools

By Project Type

Project Type	Build Tool	Deployment	Notes
Simple Script	pip + venv	Direct execution	No packaging needed
Library/Package	setuptools or poetry	PyPI	Use poetry for modern projects
Web Application	poetry	Docker	Containerize for consistency
Microservices	poetry	Docker/Kubernetes	Manage multiple services
Serverless Function	pip (AWS) / poetry (local)	AWS SAM/Azure Functions	Keep packages small

By Scale

Scale	Build Tool	Deployment	Rationale
Small (1-3 developers)	poetry	Docker	Easy setup, good defaults
Medium (4-10 developers)	poetry + CI/CD	Docker/Kubernetes	Team collaboration features
Large (10+ developers)	poetry/pdm + monorepo	Kubernetes	Scalable workflows
Enterprise	Custom build system	Cloud-native	Compliance and governance

By Deployment Target

Target	Best Build Tool	Deployment Method
Local Development	poetry	poetry install
Docker Container	poetry/pdm	Multi-stage builds
AWS Lambda	pip	SAM or Zappa
Azure Functions	pip	Azure CLI / VS Code
Heroku	pip + requirements.txt	Git push
Kubernetes	poetry	Container images

Best Practices

1. Use Virtual Environments

# Create virtual environment
python -m venv .venv

# Activate (Linux/macOS)
source .venv/bin/activate

# Activate (Windows)
.\.venv\Scripts\activate

# Deactivate
deactivate

2. Pin Dependencies

Always pin production dependencies to ensure reproducible builds:

# Using poetry
poetry add requests==2.28.0

# Or specify exact versions in pyproject.toml
[tool.poetry.dependencies]
python = "^3.8"
requests = "2.28.0"  # Exact version

3. Use Multi-Stage Docker Builds

# Build stage
FROM python:3.11-slim AS builder

WORKDIR /build
COPY pyproject.toml poetry.lock ./
RUN pip install poetry && poetry build

# Production stage
FROM python:3.11-slim AS production

WORKDIR /app
COPY --from=builder /build/dist/*.whl .
RUN pip install *.whl

COPY . .
CMD ["python", "main.py"]

4. Optimize Dependency Installation

# Install dependencies before copying code for better caching
COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .

5. Use CI/CD Pipelines

# GitHub Actions example
name: Python Package

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      
      - name: Install dependencies
        run: |
          pip install poetry
          poetry install
      
      - name: Run tests
        run: |
          poetry run pytest
      
      - name: Build package
        run: |
          poetry build
      
      - name: Upload artifact
        uses: actions/upload-artifact@v4
        with:
          name: dist
          path: dist/

Conclusion

Python’s build and deployment ecosystem offers multiple options to suit different project needs. The key is to:

1. Start simple: Use pip and venv for basic projects
2. Adopt modern tools: Transition to poetry for better dependency management
3. Containerize: Use Docker for consistent deployments across environments
4. Automate: Implement CI/CD pipelines for reliable releases

By following the guidelines in this post, you can build and deploy Python applications efficiently while maintaining scalability and maintainability.

---

Further Reading:

• Python Packaging User Guide
• Poetry Documentation
• Docker Python Best Practices
• AWS SAM Documentation
• Azure Functions Python Developer Guide

Related Tools to Explore:

• hatch: Modern Python project manager
• pipenv: Another dependency management tool
• build: PEP 517 build backend
• uv: Ultra-fast Python package installer

Related posts

• Build Ecosystems by Programming Language
• Python Build with pip, setuptools and build
• Python Build with uv and Poetry