pypi-server

Original source / Raw SKILL.md

---
name: pypi-server
description: Guide for setting up local PyPI servers to host and serve Python packages. This skill should be used when tasks involve creating a local PyPI repository, serving Python packages over HTTP, building distributable Python packages, or testing pip installations from a custom index URL.
---

# PyPI Server Setup

This skill provides guidance for creating local PyPI servers to host and distribute Python packages.

## When to Use This Skill

- Setting up a local PyPI repository or package index
- Building Python packages for distribution (wheel/sdist)
- Serving packages via HTTP for pip installation
- Testing package installation from custom index URLs

## Environment Reconnaissance (First Step)

Before planning the approach, gather critical environment information:

1. **Check Python version**: Run `python3 --version` to determine compatibility constraints
   - Python 3.13+ removed the `cgi` module, breaking many older tools like `pypiserver`
   - Plan fallback strategies based on available Python version

2. **Check available system tools**: Verify what utilities exist
   - Process management: `which ps pkill lsof kill`
   - Network utilities: `which curl wget nc`
   - Package tools: `pip list` to see pre-installed packages

3. **Identify port availability**: Check if target ports are free before starting servers

## Approach Selection

### Decision Framework

| Condition | Recommended Approach |
|-----------|---------------------|
| Python 3.13+ | Use `python3 -m http.server` with proper directory structure |
| Python 3.12 or earlier | Either `pypiserver` or `http.server` works |
| Simple single-package hosting | `http.server` is sufficient |
| Full PyPI mirroring needed | Consider `pypiserver` or `devpi` |

### Recommended Default: Python's Built-in HTTP Server

For most local PyPI hosting tasks, use `python3 -m http.server` with the correct directory structure. This approach:
- Has no external dependencies
- Works across all Python versions
- Is simpler to configure and debug

## Directory Structure Requirements

Pip expects a specific directory structure when using `--index-url`:

```
server_root/
└── simple/
    └── <package-name>/
        └── <package-name>-<version>-py3-none-any.whl
```

**Critical Details:**
- The `simple/` directory must be at the root of the served directory
- Package names in the directory should be normalized (lowercase, hyphens to underscores for some cases)
- The server must be started from the directory containing `simple/`, not from within it

## Building Python Packages

### Standard Package Structure

```
package_name/
├── setup.py
├── pyproject.toml (optional but recommended)
└── package_name/
    ├── __init__.py
    └── module.py
```

### Minimal setup.py Example

```python
from setuptools import setup, find_packages

setup(
    name="package-name",
    version="0.1.0",
    packages=find_packages(),
)
```

### Build Commands

```bash
# Build wheel and source distribution
python3 -m pip install build
python3 -m build

# Output appears in dist/
```

## Server Setup Steps

### Step 1: Create Directory Structure

```bash
mkdir -p pypi-server/simple/packagename/
cp dist/*.whl pypi-server/simple/packagename/
```

### Step 2: Start HTTP Server

```bash
cd pypi-server
python3 -m http.server 8080
```

**Important:** Start the server from the directory that contains `simple/`, not from within `simple/`.

### Step 3: Verify Server

Test that the structure is correct:
```bash
curl http://localhost:8080/simple/
curl http://localhost:8080/simple/packagename/
```

## Verification Strategy

### Pre-Installation Checks

1. Verify server is running: `curl -I http://localhost:PORT/`
2. Verify simple index exists: `curl http://localhost:PORT/simple/`
3. Verify package directory exists: `curl http://localhost:PORT/simple/packagename/`
4. Verify wheel file is accessible: `curl -I http://localhost:PORT/simple/packagename/file.whl`

### Installation Test

```bash
pip install --index-url http://localhost:PORT/simple packagename==version
```

### Post-Installation Verification

```python
import packagename
# Test core functionality
```

## Common Pitfalls and Solutions

### 1. Port Already in Use

**Symptom:** `OSError: [Errno 98] Address already in use`

**Solutions:**
- Use a different port
- Find and kill the existing process:
  ```python
  import socket
  sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
  ```
- Create a custom server script with `SO_REUSEADDR` option

### 2. Wrong Directory Structure

**Symptom:** `404 Not Found` when pip tries to access `/simple/`

**Cause:** Server started from wrong directory or missing `simple/` prefix

**Solution:** Verify the URL `http://localhost:PORT/simple/` returns a directory listing

### 3. Python 3.13+ Compatibility

**Symptom:** `ModuleNotFoundError: No module named 'cgi'` when using pypiserver

**Cause:** Python 3.13 removed the deprecated `cgi` module

**Solution:** Use `python3 -m http.server` instead of pypiserver

### 4. Process Management Without Standard Tools

**Symptom:** Cannot find or kill background processes (ps/pkill not available)

**Solutions:**
- Use Python's process management:
  ```python
  import subprocess
  proc = subprocess.Popen(['python3', '-m', 'http.server', '8080'])
  # Save proc.pid for later termination
  ```
- Use `/proc` filesystem on Linux: `ls /proc/*/cmdline`
- Track PIDs explicitly when starting background processes

### 5. Shell Compatibility

**Symptom:** `source: not found` when sourcing files

**Cause:** Using `source` in sh shell (source is a bash feature)

**Solution:** Use `.` instead of `source` for POSIX compatibility:
```bash
. ./venv/bin/activate
```

### 6. Package Name Normalization

**Symptom:** Package not found despite correct structure

**Cause:** Pip normalizes package names (underscores to hyphens, lowercase)

**Solution:** Use lowercase names and be consistent with hyphens/underscores

## Custom Server Script (Robust Alternative)

For better process control and port reuse, consider a custom server script:

```python
#!/usr/bin/env python3
import http.server
import socketserver
import os

PORT = 8080
DIRECTORY = "/path/to/pypi-server"

os.chdir(DIRECTORY)

class ReuseAddrServer(socketserver.TCPServer):
    allow_reuse_address = True

handler = http.server.SimpleHTTPRequestHandler
with ReuseAddrServer(("", PORT), handler) as httpd:
    print(f"Serving at port {PORT}")
    httpd.serve_forever()
```

## Checklist Before Starting

- [ ] Verified Python version and tool compatibility
- [ ] Created correct directory structure with `simple/` prefix
- [ ] Built package successfully (wheel exists in dist/)
- [ ] Copied wheel to correct location under `simple/packagename/`
- [ ] Confirmed target port is available
- [ ] Plan for process management (how to stop the server later)