Merge pull request #1002 from IlumCI/dockernew

[FEAT/BUGF-DOCKER][Fixed Docker CI-CD workflows to better publish swarms to dockerhub]
pull/1029/head
Kye Gomez 3 weeks ago committed by GitHub
commit 943e0b6946
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

@ -165,6 +165,24 @@ $ cd swarms
$ pip install -e . $ pip install -e .
``` ```
### Using Docker
The easiest way to get started with Swarms is using our pre-built Docker image:
```bash
# Pull and run the latest image
$ docker pull kyegomez/swarms:latest
$ docker run --rm kyegomez/swarms:latest python -c "import swarms; print('Swarms is ready!')"
# Run interactively for development
$ docker run -it --rm -v $(pwd):/app kyegomez/swarms:latest bash
# Using docker-compose (recommended for development)
$ docker-compose up -d
```
For more Docker options and advanced usage, see our [Docker documentation](/scripts/docker/DOCKER.md).
--- ---
## Environment Configuration ## Environment Configuration

@ -294,4 +294,3 @@ flycheck_*.el
# network security # network security
/network-security.data /network-security.data

@ -0,0 +1,225 @@
# Swarms Docker Image
This repository includes a Docker image for running Swarms, an AI agent framework. The image is automatically built and published to DockerHub on every push to the main branch and on version tags.
## 🐳 Quick Start
### Pull and Run
```bash
# Pull the latest image
docker pull kyegomez/swarms:latest
# Run a simple test
docker run --rm kyegomez/swarms:latest python test_docker.py
# Run with interactive shell
docker run -it --rm kyegomez/swarms:latest bash
```
### Using Specific Versions
```bash
# Pull a specific version
docker pull kyegomez/swarms:v8.0.4
# Run with specific version
docker run --rm kyegomez/swarms:v8.0.4 python -c "import swarms; print(swarms.__version__)"
```
## 🏗️ Building Locally
### Prerequisites
- Docker installed on your system
- Git to clone the repository
### Build Steps
```bash
# Clone the repository
git clone https://github.com/kyegomez/swarms.git
cd swarms
# Build the image
docker build -t swarms:latest .
# Test the image
docker run --rm swarms:latest python test_docker.py
```
## 🚀 Usage Examples
### Basic Agent Example
```bash
# Create a Python script (agent_example.py)
cat > agent_example.py << 'EOF'
from swarms import Agent
# Create an agent
agent = Agent(
agent_name="test_agent",
system_prompt="You are a helpful AI assistant."
)
# Run the agent
result = agent.run("Hello! How are you today?")
print(result)
EOF
# Run in Docker
docker run --rm -v $(pwd):/app swarms:latest python /app/agent_example.py
```
### Interactive Development
```bash
# Run with volume mount for development
docker run -it --rm \
-v $(pwd):/app \
-w /app \
swarms:latest bash
# Inside the container, you can now run Python scripts
python your_script.py
```
### Using Environment Variables
```bash
# Run with environment variables
docker run --rm \
-e OPENAI_API_KEY=your_api_key_here \
-e ANTHROPIC_API_KEY=your_anthropic_key_here \
swarms:latest python your_script.py
```
## 🔧 Configuration
### Environment Variables
The Docker image supports the following environment variables:
- `OPENAI_API_KEY`: Your OpenAI API key
- `ANTHROPIC_API_KEY`: Your Anthropic API key
- `GOOGLE_API_KEY`: Your Google API key
- `PYTHONPATH`: Additional Python path entries
- `PYTHONUNBUFFERED`: Set to 1 for unbuffered output
### Volume Mounts
Common volume mount patterns:
```bash
# Mount current directory for development
-v $(pwd):/app
# Mount specific directories
-v $(pwd)/data:/app/data
-v $(pwd)/models:/app/models
# Mount configuration files
-v $(pwd)/config:/app/config
```
## 🐛 Troubleshooting
### Common Issues
1. **Permission Denied**
```bash
# Fix permission issues
docker run --rm -v $(pwd):/app:rw swarms:latest python your_script.py
```
2. **Memory Issues**
```bash
# Increase memory limit
docker run --rm --memory=4g swarms:latest python your_script.py
```
3. **Network Issues**
```bash
# Use host network
docker run --rm --network=host swarms:latest python your_script.py
```
### Debug Mode
```bash
# Run with debug output
docker run --rm -e PYTHONUNBUFFERED=1 swarms:latest python -u your_script.py
# Run with interactive debugging
docker run -it --rm swarms:latest python -m pdb your_script.py
```
## 🔄 CI/CD Integration
The Docker image is automatically built and published via GitHub Actions:
- **Triggers**: Push to main branch, version tags (v*.*.*)
- **Platforms**: linux/amd64, linux/arm64
- **Registry**: DockerHub (kyegomez/swarms)
### GitHub Actions Secrets Required
- `DOCKERHUB_USERNAME`: Your DockerHub username
- `DOCKERHUB_TOKEN`: Your DockerHub access token
## 📊 Image Details
### Base Image
- Python 3.11-slim-bullseye
- Multi-stage build for optimization
- UV package manager for faster installations
### Image Size
- Optimized for minimal size
- Multi-stage build reduces final image size
- Only necessary dependencies included
### Security
- Non-root user execution
- Minimal system dependencies
- Regular security updates
## 🤝 Contributing
To contribute to the Docker setup:
1. Fork the repository
2. Make your changes to the Dockerfile
3. Test locally: `docker build -t swarms:test .`
4. Submit a pull request
### Testing Changes
```bash
# Build test image
docker build -t swarms:test .
# Run tests
docker run --rm swarms:test python test_docker.py
# Test with your code
docker run --rm -v $(pwd):/app swarms:test python your_test_script.py
```
## 📝 License
This Docker setup is part of the Swarms project and follows the same MIT license.
## 🆘 Support
For issues with the Docker image:
1. Check the troubleshooting section above
2. Review the GitHub Actions logs for build issues
3. Open an issue on GitHub with detailed error information
4. Include your Docker version and system information
---
**Note**: This Docker image is automatically updated with each release. For production use, consider pinning to specific version tags for stability.

@ -1,25 +1,37 @@
# Use a lightweight Python image # Multi-stage build for optimized Docker image
FROM python:3.11-slim-bullseye as builder
# Install system dependencies for building
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential gcc curl \
&& rm -rf /var/lib/apt/lists/*
# Install UV for faster package management
RUN curl -LsSf https://astral.sh/uv/install.sh | sh
ENV PATH="/root/.cargo/bin:${PATH}"
# Create a virtual environment and install dependencies
RUN uv venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
# Install the swarms package using UV
RUN uv pip install --system -U swarms
# Final stage
FROM python:3.11-slim-bullseye FROM python:3.11-slim-bullseye
# Environment config for speed and safety # Environment config for speed and safety
ENV PYTHONDONTWRITEBYTECODE=1 \ ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \ PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1 \ PATH="/opt/venv/bin:${PATH}" \
PIP_DISABLE_PIP_VERSION_CHECK=1 \
PATH="/app:${PATH}" \
PYTHONPATH="/app:${PYTHONPATH}" \ PYTHONPATH="/app:${PYTHONPATH}" \
USER=swarms USER=swarms
# Set working directory # Set working directory
WORKDIR /app WORKDIR /app
# System dependencies (minimal) # Copy virtual environment from builder stage
RUN apt-get update && apt-get install -y --no-install-recommends \ COPY --from=builder /opt/venv /opt/venv
build-essential gcc \
&& rm -rf /var/lib/apt/lists/*
# Install the swarms package
RUN pip install --upgrade pip && pip install -U swarms
# Add non-root user # Add non-root user
RUN useradd -m -s /bin/bash -U $USER && \ RUN useradd -m -s /bin/bash -U $USER && \

@ -0,0 +1,71 @@
version: '3.8'
services:
swarms:
build:
context: .
dockerfile: Dockerfile
image: swarms:latest
container_name: swarms-container
environment:
- PYTHONUNBUFFERED=1
- PYTHONPATH=/app
# Add your API keys here or use .env file
# - OPENAI_API_KEY=${OPENAI_API_KEY}
# - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
# - GOOGLE_API_KEY=${GOOGLE_API_KEY}
volumes:
- .:/app
- ./data:/app/data
- ./models:/app/models
working_dir: /app
command: python test_docker.py
restart: unless-stopped
healthcheck:
test: ["CMD", "python", "-c", "import swarms; print('Health check passed')"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
swarms-dev:
build:
context: .
dockerfile: Dockerfile
image: swarms:dev
container_name: swarms-dev-container
environment:
- PYTHONUNBUFFERED=1
- PYTHONPATH=/app
volumes:
- .:/app
- ./data:/app/data
- ./models:/app/models
working_dir: /app
command: bash
stdin_open: true
tty: true
restart: unless-stopped
swarms-api:
build:
context: .
dockerfile: Dockerfile
image: swarms:api
container_name: swarms-api-container
environment:
- PYTHONUNBUFFERED=1
- PYTHONPATH=/app
volumes:
- .:/app
working_dir: /app
ports:
- "8000:8000"
command: python -m uvicorn main:app --host 0.0.0.0 --port 8000 --reload
restart: unless-stopped
depends_on:
- swarms
networks:
default:
name: swarms-network

@ -58,6 +58,7 @@ jobs:
type=semver,pattern={{major}}.{{minor}} type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}} type=semver,pattern={{major}}
type=sha type=sha
type=raw,value=latest,enable={{is_default_branch}}
# Build and push Docker image # Build and push Docker image
- name: Build and push Docker image - name: Build and push Docker image
@ -71,3 +72,5 @@ jobs:
platforms: linux/amd64,linux/arm64 platforms: linux/amd64,linux/arm64
cache-from: type=gha cache-from: type=gha
cache-to: type=gha,mode=max cache-to: type=gha,mode=max
build-args: |
BUILDKIT_INLINE_CACHE=1

@ -0,0 +1,58 @@
name: Docker Test Build
on:
pull_request:
branches: [ "master" ]
workflow_dispatch:
env:
REGISTRY: docker.io
IMAGE_NAME: ${{ github.repository }}
jobs:
test-build:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
# Setup QEMU for multi-platform builds
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
# Setup Docker BuildX
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# Build Docker image (without pushing)
- name: Build Docker image
id: build
uses: docker/build-push-action@v6
with:
context: .
push: false
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:test
platforms: linux/amd64
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
BUILDKIT_INLINE_CACHE=1
# Test the built image
- name: Test Docker image
run: |
docker run --rm ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:test python test_docker.py
# Show image size
- name: Show image size
run: |
docker images ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:test --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}"
# Clean up test image
- name: Clean up test image
if: always()
run: |
docker rmi ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:test || true

@ -0,0 +1,139 @@
# Docker utilities for Swarms project (PowerShell version)
# Usage: .\scripts\docker-utils.ps1 [command]
param(
[Parameter(Position=0)]
[string]$Command = "help"
)
# Configuration
$ImageName = "swarms"
$Registry = "kyegomez"
$FullImageName = "$Registry/$ImageName"
# Functions
function Write-Usage {
Write-Host "Docker Utilities for Swarms" -ForegroundColor Blue
Write-Host ""
Write-Host "Usage: .\scripts\docker-utils.ps1 [command]"
Write-Host ""
Write-Host "Commands:"
Write-Host " build Build the Docker image locally"
Write-Host " test Test the Docker image"
Write-Host " run Run the Docker image interactively"
Write-Host " push Push to DockerHub (requires login)"
Write-Host " clean Clean up Docker images and containers"
Write-Host " logs Show logs from running containers"
Write-Host " shell Open shell in running container"
Write-Host " compose-up Start services with docker-compose"
Write-Host " compose-down Stop services with docker-compose"
Write-Host " help Show this help message"
Write-Host ""
}
function Build-Image {
Write-Host "Building Docker image..." -ForegroundColor Green
docker build -t "$ImageName`:latest" .
Write-Host " Image built successfully!" -ForegroundColor Green
}
function Test-Image {
Write-Host "Testing Docker image..." -ForegroundColor Green
docker run --rm "$ImageName`:latest" python test_docker.py
Write-Host " Image test completed!" -ForegroundColor Green
}
function Run-Interactive {
Write-Host "Running Docker image interactively..." -ForegroundColor Green
docker run -it --rm -v "${PWD}:/app" -w /app "$ImageName`:latest" bash
}
function Push-ToDockerHub {
Write-Host "⚠ Make sure you're logged into DockerHub first!" -ForegroundColor Yellow
Write-Host "Pushing to DockerHub..." -ForegroundColor Green
# Tag the image
docker tag "$ImageName`:latest" "$FullImageName`:latest"
# Push to DockerHub
docker push "$FullImageName`:latest"
Write-Host " Image pushed to DockerHub!" -ForegroundColor Green
}
function Clean-Docker {
Write-Host "Cleaning up Docker resources..." -ForegroundColor Yellow
# Stop and remove containers
docker ps -aq | ForEach-Object { docker rm -f $_ }
# Remove images
docker images "$ImageName" -q | ForEach-Object { docker rmi -f $_ }
# Remove dangling images
docker image prune -f
Write-Host " Docker cleanup completed!" -ForegroundColor Green
}
function Show-Logs {
Write-Host "Showing logs from running containers..." -ForegroundColor Green
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"
Write-Host ""
# Show logs for swarms containers
$containers = docker ps --filter "name=swarms" --format "{{.Names}}"
foreach ($container in $containers) {
Write-Host "Logs for $container:" -ForegroundColor Blue
docker logs $container --tail 20
Write-Host ""
}
}
function Open-Shell {
Write-Host "Opening shell in running container..." -ForegroundColor Green
# Find running swarms container
$container = docker ps --filter "name=swarms" --format "{{.Names}}" | Select-Object -First 1
if (-not $container) {
Write-Host " No running swarms container found!" -ForegroundColor Red
Write-Host "Start a container first with: .\scripts\docker-utils.ps1 run"
exit 1
}
Write-Host "Opening shell in $container..." -ForegroundColor Blue
docker exec -it $container bash
}
function Compose-Up {
Write-Host "Starting services with docker-compose..." -ForegroundColor Green
docker-compose up -d
Write-Host " Services started!" -ForegroundColor Green
Write-Host "Use 'docker-compose logs -f' to view logs"
}
function Compose-Down {
Write-Host "Stopping services with docker-compose..." -ForegroundColor Yellow
docker-compose down
Write-Host " Services stopped!" -ForegroundColor Green
}
# Main script logic
switch ($Command.ToLower()) {
"build" { Build-Image }
"test" { Test-Image }
"run" { Run-Interactive }
"push" { Push-ToDockerHub }
"clean" { Clean-Docker }
"logs" { Show-Logs }
"shell" { Open-Shell }
"compose-up" { Compose-Up }
"compose-down" { Compose-Down }
"help" { Write-Usage }
default {
Write-Host " Unknown command: $Command" -ForegroundColor Red
Write-Usage
exit 1
}
}

@ -0,0 +1,167 @@
#!/bin/bash
# Docker utilities for Swarms project
# Usage: ./scripts/docker-utils.sh [command]
set -e
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Configuration
IMAGE_NAME="swarms"
REGISTRY="kyegomez"
FULL_IMAGE_NAME="${REGISTRY}/${IMAGE_NAME}"
# Functions
print_usage() {
echo -e "${BLUE}Docker Utilities for Swarms${NC}"
echo ""
echo "Usage: $0 [command]"
echo ""
echo "Commands:"
echo " build Build the Docker image locally"
echo " test Test the Docker image"
echo " run Run the Docker image interactively"
echo " push Push to DockerHub (requires login)"
echo " clean Clean up Docker images and containers"
echo " logs Show logs from running containers"
echo " shell Open shell in running container"
echo " compose-up Start services with docker-compose"
echo " compose-down Stop services with docker-compose"
echo " help Show this help message"
echo ""
}
build_image() {
echo -e "${GREEN}Building Docker image...${NC}"
docker build -t "${IMAGE_NAME}:latest" .
echo -e "${GREEN} Image built successfully!${NC}"
}
test_image() {
echo -e "${GREEN}Testing Docker image...${NC}"
docker run --rm "${IMAGE_NAME}:latest" python test_docker.py
echo -e "${GREEN} Image test completed!${NC}"
}
run_interactive() {
echo -e "${GREEN}Running Docker image interactively...${NC}"
docker run -it --rm \
-v "$(pwd):/app" \
-w /app \
"${IMAGE_NAME}:latest" bash
}
push_to_dockerhub() {
echo -e "${YELLOW}⚠ Make sure you're logged into DockerHub first!${NC}"
echo -e "${GREEN}Pushing to DockerHub...${NC}"
# Tag the image
docker tag "${IMAGE_NAME}:latest" "${FULL_IMAGE_NAME}:latest"
# Push to DockerHub
docker push "${FULL_IMAGE_NAME}:latest"
echo -e "${GREEN} Image pushed to DockerHub!${NC}"
}
clean_docker() {
echo -e "${YELLOW}Cleaning up Docker resources...${NC}"
# Stop and remove containers
docker ps -aq | xargs -r docker rm -f
# Remove images
docker images "${IMAGE_NAME}" -q | xargs -r docker rmi -f
# Remove dangling images
docker image prune -f
echo -e "${GREEN} Docker cleanup completed!${NC}"
}
show_logs() {
echo -e "${GREEN}Showing logs from running containers...${NC}"
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"
echo ""
# Show logs for swarms containers
for container in $(docker ps --filter "name=swarms" --format "{{.Names}}"); do
echo -e "${BLUE}Logs for $container:${NC}"
docker logs "$container" --tail 20
echo ""
done
}
open_shell() {
echo -e "${GREEN}Opening shell in running container...${NC}"
# Find running swarms container
container=$(docker ps --filter "name=swarms" --format "{{.Names}}" | head -1)
if [ -z "$container" ]; then
echo -e "${RED} No running swarms container found!${NC}"
echo "Start a container first with: $0 run"
exit 1
fi
echo -e "${BLUE}Opening shell in $container...${NC}"
docker exec -it "$container" bash
}
compose_up() {
echo -e "${GREEN}Starting services with docker-compose...${NC}"
docker-compose up -d
echo -e "${GREEN} Services started!${NC}"
echo "Use 'docker-compose logs -f' to view logs"
}
compose_down() {
echo -e "${YELLOW}Stopping services with docker-compose...${NC}"
docker-compose down
echo -e "${GREEN} Services stopped!${NC}"
}
# Main script logic
case "${1:-help}" in
build)
build_image
;;
test)
test_image
;;
run)
run_interactive
;;
push)
push_to_dockerhub
;;
clean)
clean_docker
;;
logs)
show_logs
;;
shell)
open_shell
;;
compose-up)
compose_up
;;
compose-down)
compose_down
;;
help|--help|-h)
print_usage
;;
*)
echo -e "${RED} Unknown command: $1${NC}"
print_usage
exit 1
;;
esac

@ -0,0 +1,113 @@
# Setting up DockerHub Secrets for GitHub Actions
This guide will help you set up the required secrets for the Docker workflow to automatically build and push images to DockerHub.
## Prerequisites
1. A DockerHub account
2. Admin access to the GitHub repository
3. DockerHub access token
## Step 1: Create a DockerHub Access Token
1. Log in to [DockerHub](https://hub.docker.com/)
2. Go to your account settings
3. Navigate to "Security" → "Access Tokens"
4. Click "New Access Token"
5. Give it a name (e.g., "GitHub Actions")
6. Set the permissions to "Read & Write"
7. Copy the generated token (you won't be able to see it again!)
## Step 2: Add Secrets to GitHub Repository
1. Go to your GitHub repository
2. Navigate to "Settings" → "Secrets and variables" → "Actions"
3. Click "New repository secret"
4. Add the following secrets:
### Required Secrets
| Secret Name | Value | Description |
|-------------|-------|-------------|
| `DOCKERHUB_USERNAME` | Your DockerHub username | Your DockerHub username (e.g., `kyegomez`) |
| `DOCKERHUB_TOKEN` | Your DockerHub access token | The access token you created in Step 1 |
## Step 3: Verify Setup
1. Push a commit to the `main` branch
2. Go to the "Actions" tab in your GitHub repository
3. You should see the "Docker Build and Publish" workflow running
4. Check that it completes successfully
## Troubleshooting
### Common Issues
1. **Authentication Failed**
- Double-check your DockerHub username and token
- Ensure the token has "Read & Write" permissions
- Make sure the token hasn't expired
2. **Permission Denied**
- Verify you have admin access to the repository
- Check that the secrets are named exactly as shown above
3. **Workflow Not Triggering**
- Ensure you're pushing to the `main` branch
- Check that the workflow file is in `.github/workflows/`
- Verify the workflow file has the correct triggers
### Testing Locally
You can test the Docker build locally before pushing:
```bash
# Build the image locally
docker build -t swarms:test .
# Test the image
docker run --rm swarms:test python test_docker.py
# If everything works, push to GitHub
git add .
git commit -m "Add Docker support"
git push origin main
```
## Security Notes
- Never commit secrets directly to your repository
- Use repository secrets for sensitive information
- Regularly rotate your DockerHub access tokens
- Consider using organization-level secrets for team repositories
## Additional Configuration
### Custom Registry
If you want to use a different registry (not DockerHub), update the workflow file:
```yaml
env:
REGISTRY: your-registry.com
IMAGE_NAME: your-org/your-repo
```
### Multiple Tags
The workflow automatically creates tags based on:
- Git branch name
- Git commit SHA
- Version tags (v*.*.*)
- Latest tag for main branch
You can customize this in the workflow file under the "Extract Docker metadata" step.
## Support
If you encounter issues:
1. Check the GitHub Actions logs for detailed error messages
2. Verify your DockerHub credentials
3. Ensure the workflow file is properly configured
4. Open an issue in the repository with the error details

@ -0,0 +1,66 @@
#!/usr/bin/env python3
"""
Test script to verify Swarms installation in Docker container.
"""
import sys
from typing import Dict, Any
def test_swarms_import() -> Dict[str, Any]:
"""
Test that swarms can be imported and basic functionality works.
Returns:
Dict[str, Any]: Test results
"""
try:
import swarms
print(f" Swarms imported successfully. Version: {swarms.__version__}")
# Test basic functionality
from swarms import Agent
print(" Agent class imported successfully")
return {
"status": "success",
"version": swarms.__version__,
"message": "Swarms package is working correctly"
}
except ImportError as e:
print(f" Failed to import swarms: {e}")
return {
"status": "error",
"error": str(e),
"message": "Swarms package import failed"
}
except Exception as e:
print(f" Unexpected error: {e}")
return {
"status": "error",
"error": str(e),
"message": "Unexpected error occurred"
}
def main() -> None:
"""Main function to run tests."""
print(" Testing Swarms Docker Image...")
print("=" * 50)
# Test Python version
print(f"Python version: {sys.version}")
# Test swarms import
result = test_swarms_import()
print("=" * 50)
if result["status"] == "success":
print(" All tests passed! Docker image is working correctly.")
sys.exit(0)
else:
print(" Tests failed! Please check the Docker image.")
sys.exit(1)
if __name__ == "__main__":
main()
Loading…
Cancel
Save