--- name: rocm_vllm_deployment description: Production-ready vLLM deployment on AMD ROCm GPUs. Combines environment auto-check, model parameter detection, Docker Compose deployment, health verification, and functional testing with comprehensive logging and security best practices. version: 1.0.0 author: Alex He timeout: 3600s platform: Linux (AMD GPU ROCm) tags: - LLM - Deployment - AMD - ROCm - Docker Compose - vLLM - Automation - EnvCheck - AutoRepair --- # ROCm vLLM Deployment Skill Production-ready automation for deploying vLLM inference services on AMD ROCm GPUs using Docker Compose. ## Features - Environment Auto-Check - Detects and repairs missing dependencies - Model Parameter Detection - Auto-reads config.json for optimal settings - VRAM Estimation - Calculates memory requirements before deployment - Secure Token Handling - Never writes tokens to compose files - **Structured Output** - All logs and test results saved per-model - **Deployment Reports** - Human-readable summary for each deployment - Health Verification - Automated health checks and functional tests - Troubleshooting Guide - Common issues and solutions ## Environment Prerequisites **Recommended (for production):** Add to `~/.bash_profile`: ```bash # HuggingFace authentication token (required for gated models) export HF_TOKEN="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # Model cache directory (optional) export HF_HOME="$HOME/models" # Apply changes source ~/.bash_profile ``` **Not required for testing:** The skill will proceed without these set: - **HF_TOKEN**: Optional — public models work without it; gated models fail at download with clear error - **HF_HOME**: Optional — defaults to `/root/.cache/huggingface/hub` ### Environment Variable Detection **Priority Order:** 1. **Explicit parameter** (highest) — Provided in task/request (e.g., `hf_token: "xxx"`) 2. **Environment variable** — Already set in shell or from parent process 3. **~/.bash_profile** — Source to load variables 4. **Default value** (lowest) — HF_HOME defaults to `/root/.cache/huggingface/hub` | Variable | Required | If Missing | |----------|----------|------------| | `HF_TOKEN` | **Conditional** | Continue without token (public models work; gated models fail at download with clear error) | | `HF_HOME` | No | **Warning + Default** — Use `/root/.cache/huggingface/hub` | **Philosophy:** Fail fast for configuration errors, fail at download time for authentication errors. --- ## Helper Scripts **Location:** `/scripts/` ### check-env.sh Validate and load environment variables before deployment. **Usage:** ```bash # Basic check (HF_TOKEN optional, HF_HOME optional with default) ./scripts/check-env.sh # Strict mode (HF_HOME required, fails if not set) ./scripts/check-env.sh --strict # Quiet mode (minimal output, for automation) ./scripts/check-env.sh --quiet # Test with environment variables HF_TOKEN="hf_xxx" HF_HOME="/models" ./scripts/check-env.sh ``` **Exit Codes:** | Code | Meaning | |------|---------| | 0 | Environment check completed (variables loaded or defaulted) | | 2 | Critical error (e.g., cannot source ~/.bash_profile) | **Note:** This script is optional. You can also directly run `source ~/.bash_profile`. --- ### generate-report.sh Generate human-readable deployment report after successful deployment. **Usage:** ```bash ./scripts/generate-report.sh

[model-load-time] [memory-used] # Example: ./scripts/generate-report.sh \ "Qwen-Qwen3-0.6B" \ "vllm-qwen3-0-6b" \ "8001" \ "✅ Success" \ "3.6" \ "1.2" ``` **Parameters:** | Parameter | Required | Description | |-----------|----------|-------------| | `model-id` | Yes | Model ID (with `/` replaced by `-`) | | `container-name` | Yes | Docker container name | | `port` | Yes | Host port for API endpoint | | `status` | Yes | Deployment status (e.g., "✅ Success") | | `model-load-time` | No | Model loading time in seconds | | `memory-used` | No | Memory consumption in GiB | **Output:** `$HOME/vllm-compose//DEPLOYMENT_REPORT.md` **Exit Codes:** | Code | Meaning | |------|---------| | 0 | Report generated successfully | | 1 | Missing required parameters | | 2 | Output directory not found | **Integration:** This script is automatically called in **Phase 7** of the deployment workflow. --- ## Input Schema | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | model_id | String | Yes | - | HuggingFace model ID | | docker_image | String | No | rocm/vllm-dev:nightly | vLLM Docker image | | tensor_parallel_size | Integer | No | 1 | Number of GPUs | | port | Integer | No | 9999 | API server port | | hf_home | String | No | `${HF_HOME}` or `/root/.cache/huggingface/hub` | Model cache directory | | hf_token | Secret | Conditional | `${HF_TOKEN}` | HuggingFace token (optional for public models, required for gated models) | | max_model_len | Integer | No | Auto-detect | Maximum sequence length | | gpu_memory_utilization | Float | No | 0.85 | GPU memory utilization | | auto_install | Boolean | No | true | Auto-install dependencies | | log_level | String | No | INFO | Logging verbosity | ## Output Structure **All deployment artifacts MUST be saved to:** ``` $HOME/vllm-compose// ``` Convert model ID to directory name by replacing `/` with `-`: - `openai/gpt-oss-20b` → `$HOME/vllm-compose/openai-gpt-oss-20b/` - `Qwen/Qwen3-Coder-Next-FP8` → `$HOME/vllm-compose/Qwen-Qwen3-Coder-Next-FP8/` **Per-model directory structure:** ``` $HOME/vllm-compose// ├── deployment.log # Full deployment logs (stdout + stderr) ├── test-results.json # Functional test results (JSON format) ├── docker-compose.yml # Generated Docker Compose file ├── .env # HF_TOKEN environment (chmod 600, optional) └── DEPLOYMENT_REPORT.md # Human-readable deployment summary ``` **File requirements:** - `deployment.log` — Capture ALL container logs during deployment - `test-results.json` — Save API response from functional test request - `DEPLOYMENT_REPORT.md` — Generated in Phase 7 - All three files MUST exist before marking deployment as complete ## Execution Workflow ### Phase 0: Environment Check & Auto-Repair **Step 0.1: Load Environment Variables** ```bash # Source ~/.bash_profile to load HF_HOME and HF_TOKEN source ~/.bash_profile # If HF_HOME is not defined, it defaults to /root/.cache/huggingface/hub ``` If HF_HOME is not defined in ~/.bash_profile, it defaults to `/root/.cache/huggingface/hub`. **Step 0.2: Create Output Directory** - Create: `$HOME/vllm-compose//` **Step 0.3: Initialize Logging** - All output → `$HOME/vllm-compose//deployment.log` **Step 0.4: System Checks** - Detect OS and package manager - Check Python, pip, huggingface_hub - Check Docker, docker compose - Check ROCm tools (rocm-smi/amd-smi) - Check GPU access (/dev/kfd, /dev/dri) - Check disk space (20GB minimum) ### Phase 1: Model Download **Use HF_HOME from Phase 0 (environment variable or default):** ```bash # Download model to HF_HOME huggingface-cli download --local-dir "$HF_HOME/hub/models----" # Or use snapshot_download via Python: python -c "from huggingface_hub import snapshot_download; snapshot_download(repo_id='', cache_dir='$HF_HOME')" ``` **Authentication Handling:** | Scenario | Behavior | |----------|----------| | Public model + no token | ✅ Download succeeds | | Public model + token provided | ✅ Download succeeds | | Gated model + no token | ❌ Download fails with "authentication required" error | | Gated model + invalid token | ❌ Download fails with "invalid token" error | | Gated model + valid token | ✅ Download succeeds | **On Authentication Failure:** ```bash echo "ERROR: Model download failed - authentication required" echo "This model requires a valid HF_TOKEN." echo "" echo "Please add to ~/.bash_profile:" echo " export HF_TOKEN=\"hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\"" echo "Then run: source ~/.bash_profile" exit 1 ``` - Locate model path in HF cache: `$HF_HOME/hub/models----/` - Log download progress to `deployment.log` ### Phase 2: Model Parameter Detection - Read config.json from model - Auto-detect: max_model_len, hidden_size, num_attention_heads, num_hidden_layers, vocab_size, dtype - Validate TP size divides attention heads - Estimate VRAM requirement ### Phase 3: Docker Compose Configuration **Generate files in output directory:** - **docker-compose.yml** → `$HOME/vllm-compose//docker-compose.yml` - Mount HF_HOME as volume (read-only for models) - NO hardcoded tokens in compose file - **.env** → `$HOME/vllm-compose//.env` (optional) - Contains: `HF_TOKEN=` - Permissions: `chmod 600` - Only created if user explicitly requests persistent token storage **Volume mount example:** ```yaml volumes: - ${HF_HOME}:/root/.cache/huggingface/hub:ro - /dev/kfd:/dev/kfd - /dev/dri:/dev/dri ``` **Important:** Docker Compose reads `${HF_HOME}` from the host environment at runtime. Before running docker compose, source ~/.bash_profile: `source ~/.bash_profile` ### Phase 4: Container Launch **Important:** Before deploying, pull the latest image to ensure updates: ```bash docker pull rocm/vllm-dev:nightly ``` **Note:** Default port is 9999. Before running docker compose, check if port is available: `ss -tlnp | grep :`. If port is in use, specify a different port in docker-compose.yml. - Pass HF_TOKEN at runtime: HF_TOKEN=$HF_TOKEN docker compose up -d - Wait for container initialization ### Phase 5: Health Verification - Check container status - Test /health endpoint - Test /v1/models endpoint ### Phase 6: Functional Testing - Run completion test via `/v1/chat/completions` API - **Save response to:** `$HOME/vllm-compose//test-results.json` - Verify response contains valid completion - **Log deployment complete** → Append to `deployment.log` - **Deployment is complete only when both files exist:** - `deployment.log` - `test-results.json` ### Phase 7: Deployment Report **Generate human-readable deployment report using the helper script.** **Step 7.1: Extract Deployment Metrics** ```bash # Parse deployment.log for metrics MODEL_LOAD_TIME=$(grep -o "model loading took [0-9.]* seconds" deployment.log | grep -o '[0-9.]*' || echo "N/A") MEMORY_USED=$(grep -o "took [0-9.]* GiB memory" deployment.log | grep -o '[0-9.]*' || echo "N/A") ``` **Step 7.2: Generate Report** ```bash # Execute the report generation script /scripts/generate-report.sh \ "" \ "" \ "" \ "" \ "$MODEL_LOAD_TIME" \ "$MEMORY_USED" # Example: ./scripts/generate-report.sh \ "Qwen-Qwen3-0.6B" \ "vllm-qwen3-0-6b" \ "8001" \ "✅ Success" \ "3.6" \ "1.2" ``` **Output:** `$HOME/vllm-compose//DEPLOYMENT_REPORT.md` **Report Contents:** - Output structure verification (file checklist) - Deployment summary table (health, test, metrics) - Test results (request/response preview) - Environment configuration - Quick commands for operations **Completion Criteria:** - `DEPLOYMENT_REPORT.md` exists in output directory - Report contains all required sections - All file checks show ✅ ## Security Best Practices 1. **Never commit tokens to version control** — Add `.env` to `.gitignore` 2. **Use .env files with chmod 600** — Restrict access to owner only 3. **Mask tokens in logs** — Show only first 10 chars: `${TOKEN:0:10}...` 4. **Pass tokens at runtime** — `HF_TOKEN=$HF_TOKEN docker compose up -d` 5. **Store tokens in ~/.bash_profile** — For production environments, set `HF_TOKEN` in user's shell config 6. **Set token for gated models** — HF_TOKEN is validated at download time; set in ~/.bash_profile for production ## Troubleshooting ### Environment Variables | Issue | Solution | |-------|----------| | `HF_TOKEN not set` | Add `export HF_TOKEN="hf_xxx"` to `~/.bash_profile`, then `source ~/.bash_profile`. Or provide via parameter. | | `HF_HOME not set` | defaults to `/root/.cache/huggingface/hub`. For production, add `export HF_HOME="/path"` to `~/.bash_profile`. | | `~/.bash_profile not found` | Create `~/.bash_profile` and add environment variables. | | `Changes not taking effect` | Run `source ~/.bash_profile` or restart terminal. | | `HF_TOKEN provided but download still fails` | Token may be invalid or lack access to the model. Verify token at https://huggingface.co/settings/tokens | ### Model Download | Issue | Solution | |-------|----------| | `Authentication required` (gated model) | Set `HF_TOKEN` in `~/.bash_profile` or provide via parameter. Ensure token has access to the model. | | `Model not found` | Verify model ID is correct (case-sensitive). Check model exists on HuggingFace. | | `Download timeout` | Check network connection. Large models may take time. | ### Deployment | Issue | Solution | |-------|----------| | hf CLI not found | `pip install huggingface_hub` | | Docker Compose fails | Use `docker compose` (no hyphen) | | GPU access fails | Add user to `render` group: `sudo usermod -aG render $USER` | | Port in use | Change `port` parameter | | OOM | Reduce `gpu_memory_utilization` | ## Cleanup ```bash cd $HOME/vllm-compose/ docker compose down ``` ## Status Check **Check deployment status and logs:** ```bash # View deployment directory ls -la $HOME/vllm-compose// # View live logs tail -f $HOME/vllm-compose//deployment.log # View test results cat $HOME/vllm-compose//test-results.json # Check container status docker ps | grep # Verify environment variables echo "HF_TOKEN: ${HF_TOKEN:0:10}..." echo "HF_HOME: $HF_HOME" ``` ## Quick Start (Production) **Step 1: Add environment variables to ~/.bash_profile** ```bash # Required: HuggingFace token export HF_TOKEN="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # Recommended: Custom model storage path (production) export HF_HOME="/data/models/huggingface" # Apply changes source ~/.bash_profile ``` **Step 2: Verify environment is ready** ```bash # Source ~/.bash_profile to load variables source ~/.bash_profile # Expected output: # === Environment Ready === # Summary: # HF_TOKEN: hf_xxxxxx... # HF_HOME: /data/models/huggingface ``` **Step 3: Run deployment** ```bash # The skill will automatically: # 1. Source ~/.bash_profile to load HF_HOME and HF_TOKEN # 2. Use HF_TOKEN and HF_HOME from environment (or ~/.bash_profile, or defaults) # 3. Proceed without token for public models # 4. Fail at download time with clear error if gated model requires token ``` ## Version History | Version | Changes | |---------|---------| | 1.0.0 | Initial release | --- ## Referenced Files > The following files are referenced in this skill and included for context. ### scripts/check-env.sh ```bash #!/bin/bash # # check-env.sh - Check and load HF_TOKEN and HF_HOME environment variables # # This script is part of the rocm_vllm_deployment skill. # It sources ~/.bashrc and validates environment variables. # # Priority Order: # 1. Environment variables already set (e.g., from parent process/parameters) # 2. ~/.bashrc (sourced by this script) # 3. Default values (HF_HOME only; HF_TOKEN remains unset) # # Usage: # ./check-env.sh [--strict] [--quiet] # # Options: # --strict Fail if HF_HOME is not set (default: warn only) # --quiet Suppress informational output # # Exit Codes: # 0 - Environment check completed (variables loaded or defaulted) # 2 - Critical error (e.g., cannot source ~/.bashrc) # # Environment Variables: # HF_TOKEN - Optional here, required only for gated models (checked at download time) # HF_HOME - Optional: Custom model cache directory (default: ~/.cache/huggingface/hub) # set -e # Parse arguments STRICT_MODE=false QUIET_MODE=false for arg in "$@"; do case $arg in --strict) STRICT_MODE=true shift ;; --quiet) QUIET_MODE=true shift ;; esac done # Helper function for output log() { if [ "$QUIET_MODE" = false ]; then echo "$@" fi } # Track source of variables HF_TOKEN_SOURCE="not set" HF_HOME_SOURCE="not set" #------------------------------------------------------------------------------ # Step 1: Check if already set in environment (from parameters/parent process) #------------------------------------------------------------------------------ log "=== Environment Check ===" log "" if [ -n "$HF_TOKEN" ]; then HF_TOKEN_SOURCE="environment/parameter" log "✓ HF_TOKEN already set in environment: ${HF_TOKEN:0:10}..." fi if [ -n "$HF_HOME" ]; then HF_HOME_SOURCE="environment/parameter" log "✓ HF_HOME already set in environment: $HF_HOME" fi #------------------------------------------------------------------------------ # Step 2: Source ~/.bashrc to load missing variables #------------------------------------------------------------------------------ if [ -f "$HOME/.bashrc" ]; then # Source .bashrc to load HF_TOKEN and HF_HOME if not already set source "$HOME/.bashrc" log "✓ Loaded ~/.bashrc" # Check if variables were loaded from .bashrc if [ -z "$HF_TOKEN_SOURCE" ] && [ -n "$HF_TOKEN" ]; then HF_TOKEN_SOURCE="~/.bashrc" log "✓ HF_TOKEN loaded from ~/.bashrc: ${HF_TOKEN:0:10}..." fi if [ -z "$HF_HOME_SOURCE" ] && [ -n "$HF_HOME" ]; then HF_HOME_SOURCE="~/.bashrc" log "✓ HF_HOME loaded from ~/.bashrc: $HF_HOME" fi else log "⚠️ WARNING: ~/.bashrc not found" log "Creating empty ~/.bashrc for future use..." touch "$HOME/.bashrc" fi log "" #------------------------------------------------------------------------------ # Step 3: Apply defaults for missing variables #------------------------------------------------------------------------------ # HF_TOKEN: No default, will fail at download time if required if [ -z "$HF_TOKEN" ]; then HF_TOKEN_SOURCE="not set (public models only)" log "⚠️ HF_TOKEN not set" log " - Public models: Will proceed without authentication" log " - Gated models: Will fail at download time with clear error" log "" log "To avoid authentication errors, add to ~/.bashrc:" log " export HF_TOKEN=\"hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\"" else log "✓ HF_TOKEN: ${HF_TOKEN:0:10}... (source: $HF_TOKEN_SOURCE)" fi # HF_HOME: Default to ~/.cache/huggingface/hub if [ -z "$HF_HOME" ]; then if [ "$STRICT_MODE" = true ]; then log "❌ ERROR: HF_HOME is not set (strict mode)" log "" log "Please add the following to ~/.bashrc:" log "" log " export HF_HOME=\"/path/to/models\"" log "" exit 1 else HF_HOME_SOURCE="default" HF_HOME="$HOME/.cache/huggingface/hub" log "⚠️ HF_HOME not set, using default: $HF_HOME" log "" log "For production, consider adding to ~/.bashrc:" log " export HF_HOME=\"/data/models/huggingface\"" fi else log "✓ HF_HOME: $HF_HOME (source: $HF_HOME_SOURCE)" fi #------------------------------------------------------------------------------ # Step 4: Export for child processes #------------------------------------------------------------------------------ export HF_TOKEN export HF_HOME log "" log "=== Environment Ready ===" log "" log "Summary:" log " HF_TOKEN: ${HF_TOKEN:0:10}... ($HF_TOKEN_SOURCE)" log " HF_HOME: $HF_HOME ($HF_HOME_SOURCE)" log "" # Output for parsing (optional, for automation) if [ "$QUIET_MODE" = true ]; then echo "HF_TOKEN_SOURCE=$HF_TOKEN_SOURCE" echo "HF_HOME_SOURCE=$HF_HOME_SOURCE" fi exit 0 ``` ### scripts/generate-report.sh ```bash #!/bin/bash # # generate-report.sh - Generate deployment report for vLLM deployment # # This script is part of the rocm_vllm_deployment skill. # It creates a human-readable deployment report in markdown format. # # Usage: # ./generate-report.sh

[model-load-time] [memory-used] # # Example: # ./generate-report.sh "Qwen-Qwen3-0.6B" "vllm-qwen3-0-6b" "8001" "✅ Success" "3.6" "1.2" # # Exit Codes: # 0 - Report generated successfully # 1 - Missing required parameters # 2 - Output directory not found # set -e # Parameters MODEL_ID="$1" CONTAINER_NAME="$2" PORT="$3" STATUS="$4" MODEL_LOAD_TIME="${5:-N/A}" MEMORY_USED="${6:-N/A}" # Directories SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SKILL_DIR="$(dirname "$SCRIPT_DIR")" OUTPUT_DIR="$HOME/vllm-compose/$MODEL_ID" REPORT_FILE="$OUTPUT_DIR/DEPLOYMENT_REPORT.md" #------------------------------------------------------------------------------ # Validate parameters #------------------------------------------------------------------------------ if [ -z "$MODEL_ID" ] || [ -z "$CONTAINER_NAME" ] || [ -z "$PORT" ] || [ -z "$STATUS" ]; then echo "ERROR: Missing required parameters" echo "" echo "Usage: $0

[model-load-time] [memory-used]" echo "" echo "Example:" echo " $0 \"Qwen-Qwen3-0.6B\" \"vllm-qwen3-0-6b\" \"8001\" \"✅ Success\" \"3.6\" \"1.2\"" exit 1 fi # Check if output directory exists if [ ! -d "$OUTPUT_DIR" ]; then echo "ERROR: Output directory not found: $OUTPUT_DIR" exit 2 fi #------------------------------------------------------------------------------ # Helper functions #------------------------------------------------------------------------------ check_file() { if [ -f "$1" ]; then echo "✅" else echo "❌" fi } get_compose_value() { local key="$1" local file="$OUTPUT_DIR/docker-compose.yml" if [ -f "$file" ]; then # Try JSON array format first: "--key", "value" local val=$(grep -o "\"--$key\", \"[^\"]*\"" "$file" 2>/dev/null | sed 's/.*"$[^"]*$"$/\1/' | head -1) if [ -n "$val" ]; then echo "$val" return fi # Try YAML format: key: value val=$(grep -o "$key: [^ ]*" "$file" 2>/dev/null | cut -d' ' -f2 | head -1) if [ -n "$val" ]; then echo "$val" return fi echo "N/A" else echo "N/A" fi } #------------------------------------------------------------------------------ # Read test results if available #------------------------------------------------------------------------------ TEST_RESULTS_FILE="$OUTPUT_DIR/test-results.json" if [ -f "$TEST_RESULTS_FILE" ]; then TEST_RESPONSE=$(cat "$TEST_RESULTS_FILE" | head -c 2000) PROMPT_TOKENS=$(cat "$TEST_RESULTS_FILE" | grep -o '"prompt_tokens":[0-9]*' | grep -o '[0-9]*' || echo "N/A") COMPLETION_TOKENS=$(cat "$TEST_RESULTS_FILE" | grep -o '"completion_tokens":[0-9]*' | grep -o '[0-9]*' || echo "N/A") TOTAL_TOKENS=$(cat "$TEST_RESULTS_FILE" | grep -o '"total_tokens":[0-9]*' | grep -o '[0-9]*' || echo "N/A") TEST_STATUS="✅ PASSED" else TEST_RESPONSE="Test results not available" PROMPT_TOKENS="N/A" COMPLETION_TOKENS="N/A" TOTAL_TOKENS="N/A" TEST_STATUS="❌ FAILED" fi #------------------------------------------------------------------------------ # Get deployment info #------------------------------------------------------------------------------ # Check health status from deployment.log HEALTH_STATUS="⚠️ Not recorded" if [ -f "$OUTPUT_DIR/deployment.log" ]; then if grep -q "Health OK" "$OUTPUT_DIR/deployment.log" 2>/dev/null; then HEALTH_STATUS="✅ PASSED" elif grep -q "health" "$OUTPUT_DIR/deployment.log" 2>/dev/null; then HEALTH_STATUS="⚠️ In progress" fi fi # Get max model len MAX_MODEL_LEN=$(get_compose_value "max-model-len") # Get tensor parallel size TP_SIZE=$(get_compose_value "tensor-parallel-size") # Get GPU memory utilization GPU_MEM_UTIL=$(get_compose_value "gpu-memory-utilization") # Get Docker image DOCKER_IMAGE="N/A" if [ -f "$OUTPUT_DIR/docker-compose.yml" ]; then DOCKER_IMAGE=$(grep -o 'image: [^ ]*' "$OUTPUT_DIR/docker-compose.yml" 2>/dev/null | cut -d' ' -f2 || echo "N/A") fi # Get environment info HF_TOKEN_STATUS="Not set" if [ -n "$HF_TOKEN" ]; then HF_TOKEN_STATUS="Set (${HF_TOKEN:0:10}...)" fi HF_HOME_STATUS="${HF_HOME:-default: ~/.cache/huggingface/hub}" #------------------------------------------------------------------------------ # Generate report #------------------------------------------------------------------------------ cat > "$REPORT_FILE" << EOF # Deployment Report | | | |---|---| | **Model** | $MODEL_ID | | **Status** | $STATUS | | **Timestamp** | $(date '+%Y-%m-%d %H:%M:%S %Z') | | **Container** | $CONTAINER_NAME | --- ## 📁 Output Structure \`\`\` $OUTPUT_DIR/ ├── deployment.log $(check_file "$OUTPUT_DIR/deployment.log") ├── test-results.json $(check_file "$OUTPUT_DIR/test-results.json") ├── docker-compose.yml $(check_file "$OUTPUT_DIR/docker-compose.yml") └── DEPLOYMENT_REPORT.md ✅ \`\`\` --- ## 📊 Deployment Summary | Metric | Value | |--------|-------| | **Health Check** | $HEALTH_STATUS | | **Functional Test** | $TEST_STATUS | | **Model Load Time** | $MODEL_LOAD_TIME seconds | | **Memory Used** | $MEMORY_USED GiB | | **Max Context Length** | $MAX_MODEL_LEN tokens | | **Tensor Parallel Size** | $TP_SIZE | | **GPU Memory Utilization** | ${GPU_MEM_UTIL}% | --- ## 🧪 Test Results **Endpoint:** \`http://localhost:$PORT/v1/chat/completions\` **Request:** \`\`\`json { "model": "$MODEL_ID", "messages": [{"role": "user", "content": ""}], "max_tokens": } \`\`\` **Response:** \`\`\`json $TEST_RESPONSE \`\`\` **Token Usage:** - Prompt Tokens: $PROMPT_TOKENS - Completion Tokens: $COMPLETION_TOKENS - Total Tokens: $TOTAL_TOKENS --- ## 🔧 Environment | Variable | Value | |----------|-------| | **HF_TOKEN** | $HF_TOKEN_STATUS | | **HF_HOME** | $HF_HOME_STATUS | | **Docker Image** | $DOCKER_IMAGE | | **Port Mapping** | $PORT:8000 | --- ## 🚀 Quick Commands \`\`\`bash # View live logs tail -f $OUTPUT_DIR/deployment.log # Test endpoint curl http://localhost:$PORT/v1/chat/completions \\ -H "Content-Type: application/json" \\ -d '{"model":"$MODEL_ID","messages":[{"role":"user","content":"Hello"}],"max_tokens":50}' # Check container status docker ps | grep $CONTAINER_NAME # Stop container cd $OUTPUT_DIR && docker compose down \`\`\` --- ## 📝 Notes - Report generated by: generate-report.sh - Skill: rocm_vllm_deployment - Skill directory: $SKILL_DIR - Generated at: $(date '+%Y-%m-%d %H:%M:%S %Z') EOF echo "✅ Report generated: $REPORT_FILE" exit 0 ``` --- ## Skill Companion Files > Additional files collected from the skill directory layout. ### _meta.json ```json { "owner": "alexhegit", "slug": "rocm-vllm-deployment", "displayName": "ROCm vLLM Deployment", "latest": { "version": "1.0.0", "publishedAt": 1772357570473, "commit": "https://github.com/openclaw/skills/commit/ff2d5f29eb0777bb92e5f866235bc1c257a22e10" }, "history": [] } ```

Find the right skill for the job.

Content Repurposer

instagram-collector

polymarket-weather-trader

creative-illustration

ideaspark-navigator

outlook-email

xiaohongshu-assistant-operator

agile-product-owner

feishu-chat

Proposal Writer

music-analysis

openmm-grid-trading

rocm_vllm_deployment

Art

news-content-extractor

lead-gen-pipeline

epragma-redmine-issue

Google Services (gog CLI)

openclaw-continuous-learning

daily-market-insight

nestjs-modular-monolith

technical-design-doc-creator

react-best-practices

doc