[CLEANUP][Tests] [DOCS][Contributors Guide] [SCRIPTS][setup.sh] [cleanup un-used files

pull/967/head
Kye Gomez 4 days ago
parent 234138c75d
commit f88f4fab13

@ -0,0 +1,73 @@
name: Code Quality and Tests
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
code-quality-and-test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10"]
steps:
# Step 1: Check out the repository
- name: Checkout repository
uses: actions/checkout@v4
# Step 2: Set up Python
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
# Step 3: Install Poetry for dependency management
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
# Step 4: Cache dependencies to speed up subsequent runs
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ hashFiles('**/poetry.lock') }}
# Step 5: Install dependencies including dev dependencies
- name: Install dependencies
run: poetry install --no-interaction --with dev --all-extras
# Step 7: Run Black formatting check on swarms folder
- name: Check Black formatting on swarms folder
run: |
poetry run black swarms/ --check --diff
# Step 8: Run Ruff linting on swarms folder
- name: Run Ruff linting on swarms folder
run: |
poetry run ruff check swarms/
# Step 10: Run tests with API keys
- name: Run tests
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
poetry run pytest tests/ -v --tb=short
# Step 11: Upload test results as artifacts (optional)
- name: Upload test results
if: always()
uses: actions/upload-artifact@v4
with:
name: test-results-${{ matrix.python-version }}
path: |
tests/
.coverage
retention-days: 7

@ -0,0 +1,692 @@
# Environment Setup Guide for Swarms Contributors
Welcome to the Swarms development environment setup guide! This comprehensive guide will walk you through setting up your development environment from scratch, whether you're a first-time contributor or an experienced developer.
!!! success "🚀 One-Click Setup (Recommended)"
**New!** Use our automated setup script that handles everything:
```bash
git clone https://github.com/kyegomez/swarms.git
cd swarms
chmod +x scripts/setup.sh
./scripts/setup.sh
```
This script automatically installs Poetry, creates a virtual environment, installs all dependencies, sets up pre-commit hooks, and more!
!!! info "Manual Setup"
**Alternative**: For manual control, install Python 3.10+, Git, and Poetry, then run:
```bash
git clone https://github.com/kyegomez/swarms.git
cd swarms
poetry install --with dev
```
---
## :material-list-status: Prerequisites
Before setting up your development environment, ensure you have the following installed:
### System Requirements
| Tool | Version | Purpose |
|------|---------|---------|
| **Python** | 3.10+ | Core runtime |
| **Git** | 2.30+ | Version control |
| **Poetry** | 1.4+ | Dependency management (recommended) |
| **Node.js** | 16+ | Documentation tools (optional) |
### Operating System Support
=== "macOS"
```bash
# Install Homebrew if not already installed
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
# Install prerequisites
brew install python@3.10 git poetry node
```
=== "Ubuntu/Debian"
```bash
# Update package list
sudo apt update
# Install Python 3.10 and pip
sudo apt install python3.10 python3.10-venv python3-pip git curl
# Install Poetry
curl -sSL https://install.python-poetry.org | python3 -
# Add Poetry to PATH
export PATH="$HOME/.local/bin:$PATH"
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
```
=== "Windows"
1. **Install Python 3.10+** from [python.org](https://python.org/downloads/)
2. **Install Git** from [git-scm.com](https://git-scm.com/download/win)
3. **Install Poetry** using PowerShell:
```powershell
(Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python -
```
---
## :material-auto-fix: Automated Setup (Recommended)
We provide a comprehensive setup script that automates the entire development environment setup process. This is the **recommended approach** for new contributors.
### What the Setup Script Does
The `scripts/setup.sh` script automatically handles:
- ✅ **Python Version Check**: Verifies Python 3.10+ is installed
- ✅ **Poetry Installation**: Installs Poetry if not present
- ✅ **Virtual Environment**: Creates and configures a project-specific virtual environment
- ✅ **Dependencies**: Installs all main, development, lint, and test dependencies
- ✅ **Pre-commit Hooks**: Sets up and installs pre-commit hooks for code quality
- ✅ **Environment Template**: Creates a `.env` file template with common variables
- ✅ **Verification**: Runs initial setup verification checks
- ✅ **Helpful Output**: Provides colored output and next steps
### Running the Automated Setup
```bash
# Clone the repository
git clone https://github.com/kyegomez/swarms.git
cd swarms
# Make the script executable and run it
chmod +x scripts/setup.sh
./scripts/setup.sh
```
### Script Features
=== "🎯 Smart Detection"
The script intelligently detects your system state:
- Checks if Poetry is already installed
- Verifies Python version compatibility
- Detects existing virtual environments
- Checks for Git repository status
=== "🔧 Comprehensive Setup"
Installs everything you need:
```bash
# All dependency groups
poetry install --with dev,lint,test
# Pre-commit hooks
pre-commit install
pre-commit install --hook-type commit-msg
# Initial verification run
pre-commit run --all-files
```
=== "📋 Environment Template"
Creates a starter `.env` file:
```bash
# Generated .env template
OPENAI_API_KEY=your_openai_api_key_here
ANTHROPIC_API_KEY=your_anthropic_key_here
LOG_LEVEL=INFO
DEVELOPMENT=true
```
=== "💡 Helpful Guidance"
Provides next steps and useful commands:
- How to activate the virtual environment
- Essential Poetry commands
- Testing and development workflow
- Troubleshooting tips
### When to Use Manual Setup
Use the manual setup approach if you:
- Want full control over each step
- Have specific system requirements
- Are troubleshooting installation issues
- Prefer to understand each component
---
## :material-git: Repository Setup
### Step 1: Fork and Clone
1. **Fork the repository** on GitHub: [github.com/kyegomez/swarms](https://github.com/kyegomez/swarms)
2. **Clone your fork**:
```bash
git clone https://github.com/YOUR_USERNAME/swarms.git
cd swarms
```
3. **Add upstream remote**:
```bash
git remote add upstream https://github.com/kyegomez/swarms.git
```
4. **Verify remotes**:
```bash
git remote -v
# origin https://github.com/YOUR_USERNAME/swarms.git (fetch)
# origin https://github.com/YOUR_USERNAME/swarms.git (push)
# upstream https://github.com/kyegomez/swarms.git (fetch)
# upstream https://github.com/kyegomez/swarms.git (push)
```
---
## :material-package-variant: Dependency Management
Choose your preferred method for managing dependencies:
=== "Poetry (Recommended)"
Poetry provides superior dependency resolution and virtual environment management.
### Installation
```bash
# Navigate to project directory
cd swarms
# Install all dependencies including development tools
poetry install --with dev,lint,test
# Activate the virtual environment
poetry shell
```
### Useful Poetry Commands
```bash
# Add a new dependency
poetry add package_name
# Add a development dependency
poetry add --group dev package_name
# Update dependencies
poetry update
# Show dependency tree
poetry show --tree
# Run commands in the virtual environment
poetry run python your_script.py
```
=== "pip + venv"
Traditional pip-based setup with virtual environments.
### Installation
```bash
# Navigate to project directory
cd swarms
# Create virtual environment
python -m venv venv
# Activate virtual environment
# On macOS/Linux:
source venv/bin/activate
# On Windows:
venv\Scripts\activate
# Upgrade pip
pip install --upgrade pip
# Install core dependencies
pip install -r requirements.txt
# Install documentation dependencies (optional)
pip install -r docs/requirements.txt
```
---
## :material-tools: Development Tools Setup
### Code Quality Tools
Swarms uses several tools to maintain code quality:
=== "Formatting"
**Black** - Code formatter
```bash
# Format code
poetry run black swarms/
# or with pip:
black swarms/
# Check formatting without making changes
black swarms/ --check --diff
```
=== "Linting"
**Ruff** - Fast Python linter
```bash
# Run linter
poetry run ruff check swarms/
# or with pip:
ruff check swarms/
# Auto-fix issues
ruff check swarms/ --fix
```
=== "Type Checking"
**MyPy** - Static type checker
```bash
# Run type checking
poetry run mypy swarms/
# or with pip:
mypy swarms/
```
### Pre-commit Hooks (Optional but Recommended)
Set up pre-commit hooks to automatically run quality checks:
```bash
# Install pre-commit
poetry add --group dev pre-commit
# or with pip:
pip install pre-commit
# Install git hooks
pre-commit install
# Run on all files
pre-commit run --all-files
```
The project uses the latest ruff-pre-commit configuration with separate hooks for linting and formatting:
- **ruff-check**: Runs the linter with automatic fixes (`--fix` flag)
- **ruff-format**: Runs the formatter for code styling
- **types_or: [python, pyi]**: Excludes Jupyter notebooks from processing
This configuration ensures consistent code quality and style across the project while avoiding conflicts with Jupyter notebook files.
---
## :material-test-tube: Testing Setup
### Running Tests
```bash
# Run all tests
poetry run pytest
# or with pip:
pytest
# Run tests with coverage
poetry run pytest --cov=swarms tests/
# Run specific test file
poetry run pytest tests/test_specific_file.py
# Run tests matching a pattern
poetry run pytest -k "test_agent"
```
### Test Structure
The project uses pytest with the following structure:
```
tests/
├── agents/ # Agent-related tests
├── structs/ # Multi-agent structure tests
├── tools/ # Tool tests
├── utils/ # Utility tests
└── conftest.py # Test configuration
```
### Writing Tests
```python
# Example test file: tests/test_example.py
import pytest
from swarms import Agent
def test_agent_creation():
"""Test that an agent can be created successfully."""
agent = Agent(
agent_name="test_agent",
system_prompt="You are a helpful assistant"
)
assert agent.agent_name == "test_agent"
@pytest.mark.parametrize("input_val,expected", [
("hello", "HELLO"),
("world", "WORLD"),
])
def test_uppercase(input_val, expected):
"""Example parametrized test."""
assert input_val.upper() == expected
```
---
## :material-book-open-page-variant: Documentation Setup
### Building Documentation Locally
```bash
# Install documentation dependencies
pip install -r docs/requirements.txt
# Navigate to docs directory
cd docs
# Serve documentation locally
mkdocs serve
# Documentation will be available at http://127.0.0.1:8000
```
### Documentation Structure
```
docs/
├── index.md # Homepage
├── mkdocs.yml # MkDocs configuration
├── swarms/ # Core documentation
├── examples/ # Examples and tutorials
├── contributors/ # Contributor guides
└── assets/ # Images and static files
```
### Writing Documentation
Use Markdown with MkDocs extensions:
```markdown
# Page Title
!!! tip "Pro Tip"
Use admonitions to highlight important information.
=== "Python"
```python
from swarms import Agent
agent = Agent()
```
=== "CLI"
```bash
swarms create-agent --name myagent
```
```
---
## :material-application-variable: Environment Variables
Create a `.env` file for local development:
```bash
# Copy example environment file
cp .env.example .env # if it exists
# Or create your own .env file
touch .env
```
Common environment variables:
```bash
# .env file
OPENAI_API_KEY=your_openai_api_key_here
ANTHROPIC_API_KEY=your_anthropic_api_key_here
GROQ_API_KEY=your_groq_api_key_here
# Development settings
DEBUG=true
LOG_LEVEL=INFO
# Optional: Database settings
DATABASE_URL=sqlite:///swarms.db
```
---
## :material-check-circle: Verification Steps
!!! tip "Automated Verification"
If you used the automated setup script (`./scripts/setup.sh`), most verification steps are handled automatically. The script runs verification checks and reports any issues.
For manual setups, verify your setup is working correctly:
### 1. Basic Import Test
```bash
poetry run python -c "from swarms import Agent; print('✅ Import successful')"
```
### 2. Run a Simple Agent
```python
# test_setup.py
from swarms import Agent
agent = Agent(
agent_name="setup_test",
system_prompt="You are a helpful assistant for testing setup.",
max_loops=1
)
response = agent.run("Say hello!")
print(f"✅ Agent response: {response}")
```
### 3. Code Quality Check
```bash
# Run all quality checks
poetry run black swarms/ --check
poetry run ruff check swarms/
poetry run pytest tests/ -x
```
### 4. Documentation Build
```bash
cd docs
mkdocs build
echo "✅ Documentation built successfully"
```
---
## :material-rocket-launch: Development Workflow
### Creating a Feature Branch
```bash
# Sync with upstream
git fetch upstream
git checkout master
git rebase upstream/master
# Create feature branch
git checkout -b feature/your-feature-name
# Make your changes...
# Add and commit
git add .
git commit -m "feat: add your feature description"
# Push to your fork
git push origin feature/your-feature-name
```
### Daily Development Commands
```bash
# Start development session
cd swarms
poetry shell # or source venv/bin/activate
# Pull latest changes
git fetch upstream
git rebase upstream/master
# Run tests during development
poetry run pytest tests/ -v
# Format and lint before committing
poetry run black swarms/
poetry run ruff check swarms/ --fix
# Run a quick smoke test
poetry run python -c "from swarms import Agent; print('✅ All good')"
```
---
## :material-bug: Troubleshooting
!!! tip "First Step: Try the Automated Setup"
If you're experiencing setup issues, try running our automated setup script first:
```bash
chmod +x scripts/setup.sh
./scripts/setup.sh
```
This script handles most common setup problems automatically and provides helpful error messages.
### Common Issues and Solutions
=== "Poetry Issues"
**Problem**: Poetry command not found
```bash
# Solution: Add Poetry to PATH
export PATH="$HOME/.local/bin:$PATH"
# Add to your shell profile (.bashrc, .zshrc, etc.)
```
**Problem**: Poetry install fails
```bash
# Solution: Clear cache and reinstall
poetry cache clear --all pypi
poetry install --with dev
```
=== "Python Version Issues"
**Problem**: Wrong Python version
```bash
# Check Python version
python --version
# Use pyenv to manage Python versions
curl https://pyenv.run | bash
pyenv install 3.10.12
pyenv local 3.10.12
```
=== "Import Errors"
**Problem**: Cannot import swarms modules
```bash
# Ensure you're in the virtual environment
poetry shell
# or
source venv/bin/activate
# Install in development mode
poetry install --with dev
# or
pip install -e .
```
=== "Test Failures"
**Problem**: Tests fail due to missing dependencies
```bash
# Install test dependencies
poetry install --with test
# or
pip install pytest pytest-cov pytest-mock
```
### Getting Help
If you encounter issues:
1. **Check the FAQ** in the main documentation
2. **Search existing issues** on GitHub
3. **Ask in the Discord community**: [discord.gg/jM3Z6M9uMq](https://discord.gg/jM3Z6M9uMq)
4. **Create a GitHub issue** with:
- Your operating system
- Python version
- Error messages
- Steps to reproduce
---
## :material-next-step: Next Steps
Now that your environment is set up:
1. **Read the Contributing Guide**: [contributors/main.md](main.md)
2. **Explore the Codebase**: Start with `swarms/structs/agent.py`
3. **Run Examples**: Check out `examples/` directory
4. **Pick an Issue**: Look for `good-first-issue` labels on GitHub
5. **Join the Community**: Discord, Twitter, and GitHub discussions
!!! success "You're Ready!"
Your Swarms development environment is now set up! You're ready to contribute to the most important technology for multi-agent collaboration.
---
## :material-bookmark-outline: Quick Reference
### Essential Commands
```bash
# Setup (choose one)
./scripts/setup.sh # Automated setup (recommended)
poetry install --with dev # Manual dependency install
# Daily workflow
poetry shell # Activate environment
poetry run pytest # Run tests
poetry run black swarms/ # Format code
poetry run ruff check swarms/ # Lint code
# Git workflow
git fetch upstream # Get latest changes
git rebase upstream/master # Update your branch
git checkout -b feature/name # Create feature branch
git push origin feature/name # Push your changes
# Documentation
cd docs && mkdocs serve # Serve docs locally
mkdocs build # Build docs
```
### Project Structure
```
swarms/
├── swarms/ # Core package
│ ├── agents/ # Agent implementations
│ ├── structs/ # Multi-agent structures
│ ├── tools/ # Agent tools
│ └── utils/ # Utilities
├── examples/ # Usage examples
├── tests/ # Test suite
├── docs/ # Documentation
├── pyproject.toml # Poetry configuration
└── requirements.txt # Pip dependencies
```
Happy coding! 🚀

@ -1,287 +0,0 @@
# BaseWorkflow
The `BaseWorkflow` class serves as a foundational structure for defining and managing workflows. It allows users to add, remove, update, and manage tasks and agents within a workflow, offering flexibility and extensibility for various applications.
### Key Concepts
- **Agents**: Entities participating in the workflow.
- **Tasks**: Units of work to be executed within the workflow.
- **Models**: Computational models used within the workflow.
- **Workflow State**: The state of the workflow, which can be saved and restored.
## Attributes
### Arguments
| Argument | Type | Default | Description |
|----------|------|---------|-------------|
| `agents` | `List[Agent]` | `None` | A list of agents participating in the workflow. |
| `task_pool` | `List[Task]` | `None` | A list of tasks in the workflow. |
| `models` | `List[Any]` | `None` | A list of models used in the workflow. |
| `*args` | | | Variable length argument list. |
| `**kwargs` | | | Arbitrary keyword arguments. |
### Attributes
| Attribute | Type | Description |
|-----------|------|-------------|
| `agents` | `List[Agent]` | A list of agents participating in the workflow. |
| `task_pool` | `List[Task]` | A list of tasks in the workflow. |
| `models` | `List[Any]` | A list of models used in the workflow. |
## Methods
### add_task
Adds a task or a list of tasks to the task pool.
**Arguments:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `task` | `Task` | `None` | A single task to add. |
| `tasks` | `List[Task]` | `None` | A list of tasks to add. |
**Raises:**
- `ValueError`: If neither task nor tasks are provided.
**Examples:**
```python
workflow = BaseWorkflow()
task1 = Task(description="Task 1")
task2 = Task(description="Task 2")
# Adding a single task
workflow.add_task(task=task1)
# Adding multiple tasks
workflow.add_task(tasks=[task1, task2])
```
### add_agent
Adds an agent to the workflow.
**Arguments:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `agent` | `Agent` | The agent to add to the workflow. |
**Examples:**
```python
workflow = BaseWorkflow()
agent = Agent(name="Agent 1")
# Adding an agent to the workflow
workflow.add_agent(agent=agent)
```
### run
Abstract method to run the workflow.
### __sequential_loop
Abstract method for the sequential loop.
### __log
Logs a message if verbose mode is enabled.
**Arguments:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `message` | `str` | The message to log. |
### __str__
Returns a string representation of the workflow.
### __repr__
Returns a string representation of the workflow for debugging.
### reset
Resets the workflow by clearing the results of each task.
**Examples:**
```python
workflow = BaseWorkflow()
workflow.reset()
```
### get_task_results
Returns the results of each task in the workflow.
**Returns:**
| Return Type | Description |
|-------------|-------------|
| `Dict[str, Any]` | The results of each task in the workflow. |
**Examples:**
```python
workflow = BaseWorkflow()
results = workflow.get_task_results()
```
### remove_task
Removes a task from the workflow.
**Arguments:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `task` | `str` | The description of the task to remove. |
**Examples:**
```python
workflow = BaseWorkflow()
workflow.remove_task(task="Task 1")
```
### update_task
Updates the arguments of a task in the workflow.
**Arguments:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `task` | `str` | The description of the task to update. |
| `**updates` | | The updates to apply to the task. |
**Raises:**
- `ValueError`: If the task is not found in the workflow.
**Examples:**
```python
workflow = BaseWorkflow()
task = Task(description="Task 1", kwargs={"param": 1})
# Adding a task to the workflow
workflow.add_task(task=task)
# Updating the task
workflow.update_task("Task 1", param=2)
```
### delete_task
Deletes a task from the workflow.
**Arguments:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `task` | `str` | The description of the task to delete. |
**Raises:**
- `ValueError`: If the task is not found in the workflow.
**Examples:**
```python
workflow = BaseWorkflow()
task = Task(description="Task 1")
# Adding a task to the workflow
workflow.add_task(task=task)
# Deleting the task
workflow.delete_task("Task 1")
```
### save_workflow_state
Saves the workflow state to a json file.
**Arguments:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `filepath` | `Optional[str]` | `"sequential_workflow_state.json"` | The path to save the workflow state to. |
**Examples:**
```python
workflow = BaseWorkflow()
workflow.save_workflow_state(filepath="workflow_state.json")
```
### add_objective_to_workflow
Adds an objective to the workflow.
**Arguments:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `task` | `str` | The description of the task. |
| `**kwargs` | | Additional keyword arguments for the task. |
**Examples:**
```python
workflow = BaseWorkflow()
workflow.add_objective_to_workflow(task="New Objective", agent=agent, args=[], kwargs={})
```
### load_workflow_state
Loads the workflow state from a json file and restores the workflow state.
**Arguments:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `filepath` | `str` | `None` | The path to load the workflow state from. |
**Examples:**
```python
workflow = BaseWorkflow()
workflow.load_workflow_state(filepath="workflow_state.json")
```
### workflow_dashboard
Displays a dashboard for the workflow.
**Arguments:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `**kwargs` | | Additional keyword arguments to pass to the dashboard. |
**Examples:**
```python
workflow = BaseWorkflow()
workflow.workflow_dashboard()
```
### workflow_bootup
Initializes the workflow.
**Examples:**
```python
workflow = BaseWorkflow()
workflow.workflow_bootup()
```

@ -1,6 +1,7 @@
import os
import concurrent.futures import concurrent.futures
from typing import List, Optional, Dict, Any import os
from typing import Any
from loguru import logger from loguru import logger
try: try:
@ -26,12 +27,12 @@ class VLLMWrapper:
def __init__( def __init__(
self, self,
model_name: str = "meta-llama/Llama-2-7b-chat-hf", model_name: str = "meta-llama/Llama-2-7b-chat-hf",
system_prompt: Optional[str] = None, system_prompt: str | None = None,
stream: bool = False, stream: bool = False,
temperature: float = 0.5, temperature: float = 0.5,
max_tokens: int = 4000, max_tokens: int = 4000,
max_completion_tokens: int = 4000, max_completion_tokens: int = 4000,
tools_list_dictionary: Optional[List[Dict[str, Any]]] = None, tools_list_dictionary: list[dict[str, Any]] | None = None,
tool_choice: str = "auto", tool_choice: str = "auto",
parallel_tool_calls: bool = False, parallel_tool_calls: bool = False,
*args, *args,
@ -121,8 +122,8 @@ class VLLMWrapper:
return self.run(task, *args, **kwargs) return self.run(task, *args, **kwargs)
def batched_run( def batched_run(
self, tasks: List[str], batch_size: int = 10 self, tasks: list[str], batch_size: int = 10
) -> List[str]: ) -> list[str]:
""" """
Run the model for multiple tasks in batches. Run the model for multiple tasks in batches.

@ -329,7 +329,8 @@ class CouncilJudgeEvaluator:
return ( return (
final_answer.group(1) == correct_answer_lower final_answer.group(1) == correct_answer_lower
) )
except: except Exception as e:
logger.error(f"Error checking answer: {str(e)}")
pass pass
# For other datasets, check if the correct answer is contained in the evaluation # For other datasets, check if the correct answer is contained in the evaluation

@ -96,6 +96,12 @@ mypy-protobuf = "^3.0.0"
[tool.poetry.group.test.dependencies] [tool.poetry.group.test.dependencies]
pytest = "^8.1.1" pytest = "^8.1.1"
[tool.poetry.group.dev.dependencies]
black = "*"
ruff = "*"
pytest = "*"
# pre-commit = "*"
[tool.ruff] [tool.ruff]
line-length = 70 line-length = 70

@ -12,13 +12,13 @@ psutil
python-dotenv python-dotenv
PyYAML PyYAML
docstring_parser==0.16 docstring_parser==0.16
black>=23.1,<26.0 black
ruff>=0.0.249,<0.4.5 ruff
types-toml>=0.10.8.1 types-toml>=0.10.8.1
types-pytz>=2023.3,<2026.0 types-pytz>=2023.3,<2026.0
types-chardet>=5.0.4.6 types-chardet>=5.0.4.6
mypy-protobuf>=3.0.0 mypy-protobuf>=3.0.0
pytest>=8.1.1 pytest
networkx networkx
aiofiles aiofiles
httpx httpx

@ -0,0 +1,155 @@
#!/bin/bash
# Swarms Setup Script
# This script sets up the complete development environment for the Swarms project
set -e # Exit on any error
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Function to print colored output
print_status() {
echo -e "${BLUE}[INFO]${NC} $1"
}
print_success() {
echo -e "${GREEN}[SUCCESS]${NC} $1"
}
print_warning() {
echo -e "${YELLOW}[WARNING]${NC} $1"
}
print_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
# Function to check if command exists
command_exists() {
command -v "$1" >/dev/null 2>&1
}
print_status "Starting Swarms development environment setup..."
# Check Python version
print_status "Checking Python version..."
if command_exists python3; then
PYTHON_VERSION=$(python3 -c 'import sys; print(".".join(map(str, sys.version_info[:2])))')
print_status "Found Python $PYTHON_VERSION"
# Check if Python version meets requirements (>=3.10)
if python3 -c 'import sys; exit(0 if sys.version_info >= (3, 10) else 1)'; then
print_success "Python version is compatible (>=3.10)"
else
print_error "Python 3.10 or higher is required. Please install a compatible Python version."
exit 1
fi
else
print_error "Python3 is not installed. Please install Python 3.10 or higher."
exit 1
fi
# Install Poetry if not present
if ! command_exists poetry; then
print_status "Poetry not found. Installing Poetry..."
curl -sSL https://install.python-poetry.org | python3 -
# Add Poetry to PATH for current session
export PATH="$HOME/.local/bin:$PATH"
# Check if installation was successful
if command_exists poetry; then
print_success "Poetry installed successfully"
else
print_error "Failed to install Poetry. Please install manually from https://python-poetry.org/"
exit 1
fi
else
print_success "Poetry is already installed"
poetry --version
fi
# Configure Poetry to create virtual environments in project directory
print_status "Configuring Poetry..."
poetry config virtualenvs.in-project true
poetry config virtualenvs.prefer-active-python true
# Install dependencies
print_status "Installing project dependencies..."
poetry install --with dev,lint,test
print_success "All dependencies installed successfully"
# Activate virtual environment and run additional setup
print_status "Setting up development tools..."
# Setup pre-commit hooks
print_status "Installing pre-commit hooks..."
poetry run pre-commit install
poetry run pre-commit install --hook-type commit-msg
print_success "Pre-commit hooks installed"
# Run pre-commit on all files to ensure everything is set up correctly
print_status "Running pre-commit on all files (this may take a while on first run)..."
poetry run pre-commit run --all-files || print_warning "Some pre-commit checks failed - this is normal on first setup"
# Create .env file if it doesn't exist
if [ ! -f ".env" ]; then
print_status "Creating .env file template..."
cat > .env << 'EOF'
# Swarms Environment Variables
# Copy this file and fill in your actual values
# OpenAI Configuration
OPENAI_API_KEY=your_openai_api_key_here
# Other API Keys (add as needed)
# ANTHROPIC_API_KEY=your_anthropic_key_here
# COHERE_API_KEY=your_cohere_key_here
# Logging Level
LOG_LEVEL=INFO
# Development Settings
DEVELOPMENT=true
EOF
print_success ".env template created - please fill in your API keys"
else
print_status ".env file already exists"
fi
# Check if Git is initialized
if [ ! -d ".git" ]; then
print_warning "Git repository not initialized. Run 'git init' if you want version control."
else
print_success "Git repository detected"
fi
# Display virtual environment information
print_status "Virtual environment information:"
poetry env info
# Display installed packages
print_status "Installed packages:"
poetry show --tree
print_success "Setup completed successfully!"
print_status "Next steps:"
echo " 1. Activate the virtual environment: poetry shell"
echo " 2. Fill in your API keys in the .env file"
echo " 3. Run tests to verify installation: poetry run pytest"
echo " 4. Start developing!"
print_status "Useful commands:"
echo " - poetry shell # Activate virtual environment"
echo " - poetry run python <script> # Run Python scripts"
echo " - poetry run pytest # Run tests"
echo " - poetry run pre-commit run # Run pre-commit hooks manually"
echo " - poetry add <package> # Add new dependencies"
echo " - poetry update # Update dependencies"

@ -0,0 +1,120 @@
import pathlib
from typing import List, Tuple
def rename_test_files(
tests_dir: str = "tests", dry_run: bool = True
) -> List[Tuple[str, str]]:
"""
Recursively finds all Python files in the tests directory and adds 'test_'
prefix to their names if they don't already contain 'test_' in the filename.
Args:
tests_dir (str): Path to the tests directory. Defaults to "tests".
dry_run (bool): If True, only shows what would be renamed without actually renaming.
If False, performs the actual renaming. Defaults to True.
Returns:
List[Tuple[str, str]]: List of tuples containing (old_path, new_path) for each renamed file.
"""
renamed_files = []
tests_path = pathlib.Path(tests_dir)
if not tests_path.exists():
print(f"Error: Tests directory '{tests_dir}' does not exist.")
return renamed_files
# Find all Python files recursively
python_files = list(tests_path.rglob("*.py"))
print(
f"Found {len(python_files)} Python files in '{tests_dir}' directory"
)
print("=" * 60)
for file_path in python_files:
filename = file_path.name
# Skip files that already have 'test_' in their name
if "test_" in filename.lower():
print(f"✓ SKIP: {file_path} (already contains 'test_')")
continue
# Skip __init__.py files
if filename == "__init__.py":
print(f"✓ SKIP: {file_path} (__init__.py file)")
continue
# Create new filename with 'test_' prefix
new_filename = f"test_{filename}"
new_path = file_path.parent / new_filename
# Check if target filename already exists
if new_path.exists():
print(
f"⚠️ WARNING: {new_path} already exists, skipping {file_path}"
)
continue
if dry_run:
print(
f"🔍 DRY RUN: Would rename {file_path} -> {new_path}"
)
else:
try:
file_path.rename(new_path)
print(f"✅ RENAMED: {file_path} -> {new_path}")
except OSError as e:
print(f"❌ ERROR: Failed to rename {file_path}: {e}")
continue
renamed_files.append((str(file_path), str(new_path)))
print("=" * 60)
print(
f"Summary: {len(renamed_files)} files {'would be' if dry_run else 'were'} renamed"
)
return renamed_files
def main():
"""
Main function to demonstrate the rename functionality.
"""
print("Python Test File Renamer")
print("=" * 60)
# First run in dry-run mode to see what would be changed
print("Running in DRY RUN mode first...")
print()
renamed_files = rename_test_files(tests_dir="tests", dry_run=True)
if not renamed_files:
print("No files need to be renamed.")
return
print("\nFiles that would be renamed:")
for old_path, new_path in renamed_files:
print(f" {old_path} -> {new_path}")
# Ask user if they want to proceed with actual renaming
print(f"\nFound {len(renamed_files)} files that need renaming.")
user_input = (
input(
"Do you want to proceed with the actual renaming? (y/N): "
)
.strip()
.lower()
)
if user_input in ["y", "yes"]:
print("\nProceeding with actual renaming...")
rename_test_files(tests_dir="tests", dry_run=False)
else:
print("Renaming cancelled.")
if __name__ == "__main__":
main()

@ -3,7 +3,6 @@ from swarms.structs.agent_builder import AgentsBuilder
from swarms.structs.auto_swarm_builder import AutoSwarmBuilder from swarms.structs.auto_swarm_builder import AutoSwarmBuilder
from swarms.structs.base_structure import BaseStructure from swarms.structs.base_structure import BaseStructure
from swarms.structs.base_swarm import BaseSwarm from swarms.structs.base_swarm import BaseSwarm
from swarms.structs.base_workflow import BaseWorkflow
from swarms.structs.batch_agent_execution import batch_agent_execution from swarms.structs.batch_agent_execution import batch_agent_execution
from swarms.structs.concurrent_workflow import ConcurrentWorkflow from swarms.structs.concurrent_workflow import ConcurrentWorkflow
from swarms.structs.conversation import Conversation from swarms.structs.conversation import Conversation
@ -98,7 +97,6 @@ __all__ = [
"Agent", "Agent",
"BaseStructure", "BaseStructure",
"BaseSwarm", "BaseSwarm",
"BaseWorkflow",
"ConcurrentWorkflow", "ConcurrentWorkflow",
"Conversation", "Conversation",
"GroupChat", "GroupChat",

@ -1,343 +0,0 @@
import json
from typing import Any, Dict, List, Optional
from swarms.utils.formatter import formatter
from swarms.structs.agent import Agent
from swarms.structs.base_structure import BaseStructure
from swarms.utils.loguru_logger import initialize_logger
logger = initialize_logger("base-workflow")
class BaseWorkflow(BaseStructure):
"""
Base class for defining a workflow.
Args:
agents (List[Agent], optional): A list of agents participating in the workflow. Defaults to None.
task_pool (List[Task], optional): A list of tasks in the workflow. Defaults to None.
models (List[Any], optional): A list of models used in the workflow. Defaults to None.
*args: Variable length argument list.
**kwargs: Arbitrary keyword arguments.
Attributes:
agents (List[Agent]): A list of agents participating in the workflow.
task_pool (List[Task]): A list of tasks in the workflow.
models (List[Any]): A list of models used in the workflow.
Methods:
add_task: Adds a task or a list of tasks to the task pool.
add_agent: Adds an agent to the workflow.
run: Abstract method to run the workflow.
reset: Resets the workflow by clearing the results of each task.
get_task_results: Returns the results of each task in the workflow.
remove_task: Removes a task from the workflow.
update_task: Updates the arguments of a task in the workflow.
delete_task: Deletes a task from the workflow.
save_workflow_state: Saves the workflow state to a json file.
add_objective_to_workflow: Adds an objective to the workflow.
load_workflow_state: Loads the workflow state from a json file and restores the workflow state.
"""
def __init__(
self,
agents: List[Agent] = None,
task_pool: List[str] = None,
models: List[Any] = None,
*args,
**kwargs,
):
super().__init__(*args, **kwargs)
self.agents = agents
self.task_pool = task_pool
self.models = models
self.task_pool = []
self.agent_pool = []
# Logging
logger.info("Number of agents activated:")
if self.agents:
logger.info(f"Agents: {len(self.agents)}")
else:
logger.info("No agents activated.")
if self.task_pool:
logger.info(f"Task Pool Size: {len(self.task_pool)}")
else:
logger.info("Task Pool is empty.")
def add_task(
self,
task: str = None,
tasks: List[str] = None,
*args,
**kwargs,
):
"""
Adds a task or a list of tasks to the task pool.
Args:
task (Task, optional): A single task to add. Defaults to None.
tasks (List[Task], optional): A list of tasks to add. Defaults to None.
Raises:
ValueError: If neither task nor tasks are provided.
"""
if task:
self.task_pool.append(task)
elif tasks:
self.task_pool.extend(tasks)
else:
raise ValueError(
"You must provide a task or a list of tasks"
)
def add_agent(self, agent: Agent, *args, **kwargs):
return self.agent_pool(agent)
def run(self):
"""
Abstract method to run the workflow.
"""
...
def __sequential_loop(self):
"""
Abstract method for the sequential loop.
"""
# raise NotImplementedError("You must implement this method")
...
def __log(self, message: str):
"""
Logs a message if verbose mode is enabled.
Args:
message (str): The message to log.
"""
if self.verbose:
print(message)
def __str__(self):
return f"Workflow with {len(self.task_pool)} tasks"
def __repr__(self):
return f"Workflow with {len(self.task_pool)} tasks"
def reset(self) -> None:
"""Resets the workflow by clearing the results of each task."""
try:
for task in self.tasks:
task.result = None
except Exception as error:
formatter.print_panel(
f"Error resetting workflow: {error}"
)
raise error
def get_task_results(self) -> Dict[str, Any]:
"""
Returns the results of each task in the workflow.
Returns:
Dict[str, Any]: The results of each task in the workflow
"""
try:
return {
task.description: task.result for task in self.tasks
}
except Exception as error:
formatter.print_panel(
f"Error getting task results: {error}"
)
def remove_task(self, task: str) -> None:
"""Remove tasks from sequential workflow"""
try:
self.tasks = [
task
for task in self.tasks
if task.description != task
]
except Exception as error:
formatter.print_panel(
f"Error removing task from workflow: {error}",
)
raise error
def update_task(self, task: str, **updates) -> None:
"""
Updates the arguments of a task in the workflow.
Args:
task (str): The description of the task to update.
**updates: The updates to apply to the task.
Raises:
ValueError: If the task is not found in the workflow.
Examples:
>>> from swarm_models import OpenAIChat
>>> from swarms.structs import SequentialWorkflow
>>> llm = OpenAIChat(openai_api_key="")
>>> workflow = SequentialWorkflow(max_loops=1)
>>> workflow.add("What's the weather in miami", llm)
>>> workflow.add("Create a report on these metrics", llm)
>>> workflow.update_task("What's the weather in miami", max_tokens=1000)
>>> workflow.tasks[0].kwargs
{'max_tokens': 1000}
"""
try:
for task in self.tasks:
if task.description == task:
task.kwargs.update(updates)
break
else:
raise ValueError(
f"Task {task} not found in workflow."
)
except Exception as error:
formatter.print_panel(
f"Error updating task in workflow: {error}"
),
def delete_task(self, task: str) -> None:
"""
Delete a task from the workflow.
Args:
task (str): The description of the task to delete.
Raises:
ValueError: If the task is not found in the workflow.
Examples:
>>> from swarm_models import OpenAIChat
>>> from swarms.structs import SequentialWorkflow
>>> llm = OpenAIChat(openai_api_key="")
>>> workflow = SequentialWorkflow(max_loops=1)
>>> workflow.add("What's the weather in miami", llm)
>>> workflow.add("Create a report on these metrics", llm)
>>> workflow.delete_task("What's the weather in miami")
>>> workflow.tasks
[Task(description='Create a report on these metrics', agent=Agent(llm=OpenAIChat(openai_api_key=''), max_loops=1, dashboard=False), args=[], kwargs={}, result=None, history=[])]
"""
try:
for task in self.tasks:
if task.description == task:
self.tasks.remove(task)
break
else:
raise ValueError(
f"Task {task} not found in workflow."
)
except Exception as error:
formatter.print_panel(
f"Error deleting task from workflow: {error}",
)
raise error
def save_workflow_state(
self,
filepath: Optional[str] = "sequential_workflow_state.json",
**kwargs,
) -> None:
"""
Saves the workflow state to a json file.
Args:
filepath (str): The path to save the workflow state to.
Examples:
>>> from swarm_models import OpenAIChat
>>> from swarms.structs import SequentialWorkflow
>>> llm = OpenAIChat(openai_api_key="")
>>> workflow = SequentialWorkflow(max_loops=1)
>>> workflow.add("What's the weather in miami", llm)
>>> workflow.add("Create a report on these metrics", llm)
>>> workflow.save_workflow_state("sequential_workflow_state.json")
"""
try:
filepath = filepath or self.saved_state_filepath
with open(filepath, "w") as f:
# Saving the state as a json for simplicuty
state = {
"tasks": [
{
"description": task.description,
"args": task.args,
"kwargs": task.kwargs,
"result": task.result,
"history": task.history,
}
for task in self.tasks
],
"max_loops": self.max_loops,
}
json.dump(state, f, indent=4)
except Exception as error:
formatter.print_panel(
f"Error saving workflow state: {error}",
)
raise error
def add_objective_to_workflow(self, task: str, **kwargs) -> None:
"""Adds an objective to the workflow."""
try:
formatter.print_panel(
"""
Adding Objective to Workflow...""",
"green",
)
self.tasks.append(task)
except Exception as error:
formatter.print_panel(
f"Error adding objective to workflow: {error}",
)
raise error
def workflow_dashboard(self, **kwargs) -> None:
"""
Displays a dashboard for the workflow.
Args:
**kwargs: Additional keyword arguments to pass to the dashboard.
Examples:
>>> from swarm_models import OpenAIChat
>>> from swarms.structs import SequentialWorkflow
>>> llm = OpenAIChat(openai_api_key="")
>>> workflow = SequentialWorkflow(max_loops=1)
>>> workflow.add("What's the weather in miami", llm)
>>> workflow.add("Create a report on these metrics", llm)
>>> workflow.workflow_dashboard()
"""
formatter.print_panel(
f"""
Sequential Workflow Dashboard
--------------------------------
Name: {self.name}
Description: {self.description}
task_pool: {len(self.task_pool)}
Max Loops: {self.max_loops}
Autosave: {self.autosave}
Autosave Filepath: {self.saved_state_filepath}
Restore Filepath: {self.restore_state_filepath}
--------------------------------
Metadata:
kwargs: {kwargs}
"""
)
def workflow_bootup(self, **kwargs) -> None:
"""
Workflow bootup.
"""
formatter.print_panel(
"""Sequential Workflow Initializing...""",
)

@ -120,11 +120,11 @@ def prepare_output_for_output_model(
Union[BaseModel, str]: The prepared output. Union[BaseModel, str]: The prepared output.
""" """
if output_type == BaseModel: if output_type is BaseModel:
return str_to_pydantic_model(output, output_type) return str_to_pydantic_model(output, output_type)
elif output_type == dict: elif output_type is dict:
return dict_to_json_str(output) return dict_to_json_str(output)
elif output_type == str: elif output_type is str:
return output return output
else: else:
return output return output

@ -1,33 +0,0 @@
# TESTING
# -==================
# Use an official Python runtime as a parent image
FROM python:3.11-slim
# Set environment variables to make Python output unbuffered and disable the PIP cache
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV PIP_NO_CACHE_DIR off
ENV PIP_DISABLE_PIP_VERSION_CHECK on
ENV PIP_DEFAULT_TIMEOUT 100
# Set environment variables for OpenAI API key and workspace directory
ENV OPENAI_API_KEY your_api_key_here
ENV WORKSPACE_DIR /path/to/your/workspace
# Set the working directory in the container
WORKDIR /usr/src/app
# Copy the current directory contents into the container at /usr/src/app
COPY . .
# Install Poetry
RUN pip install poetry
# Disable virtualenv creation by poetry and install dependencies
RUN poetry config virtualenvs.create false
# Install the 'swarms' package and any additional packages
RUN pip install swarms swarm-models swarms-memory pytest
# Run all tests in the tests directory
CMD python3 -m unittest discover -s tests

@ -1,79 +0,0 @@
The pseudocode for unit tests covering the WorkerNode and BossNode might look something like this:
1. Initialize the WorkerNode and BossNode instances with the necessary dependencies.
2. Test the `create_agent` method of the WorkerNode. Ensure it creates an agent as expected.
3. Test the `run_agent` method of the WorkerNode. Check if it runs the agent as expected.
4. Test the `create_task` method of the BossNode. Check if it creates a task as expected.
5. Test the `execute_task` method of the BossNode. Ensure it executes the task as expected.
In Python, this would look something like:
```python
import pytest
def test_WorkerNode_create_agent():
# assuming llm, tools, and vectorstore are initialized properly
worker_node = WorkerNode(llm, tools, vectorstore)
worker_node.create_agent("test_agent", "test_role", False, {})
assert worker_node.agent is not None
assert worker_node.agent.chain.verbose
def test_WorkerNode_run_agent():
worker_node = WorkerNode(llm, tools, vectorstore)
worker_node.create_agent("test_agent", "test_role", False, {})
worker_node.run_agent("test prompt") # check it runs without error
def test_BossNode_create_task():
# assuming llm, vectorstore, task_execution_chain are initialized properly
boss_node = BossNode(llm, vectorstore, task_execution_chain, False, 3)
task = boss_node.create_task("test task")
assert task == {"objective": "test task"}
def test_BossNode_execute_task():
boss_node = BossNode(llm, vectorstore, task_execution_chain, False, 3)
task = boss_node.create_task("test task")
boss_node.execute_task(task) # check it runs without error
```
You would run these tests with a testing tool such as `pytest`. This is just an example and does not cover all possible test cases. Ideally, your tests should be more comprehensive, and should include negative test cases as well, to check that your code handles errors correctly.
The code you have provided has quite a few interconnected components, so it would be good to design tests that examine not just the individual pieces but how well they integrate and interact. Here are three additional tests you could consider:
1. **Test that the tools in the WorkerNode are correctly instantiated and are working as expected:** Since the tools are a key part of the functionality in the WorkerNode, it's important to verify they're initialized correctly. You could choose one tool to test in detail, or write a generic test that loops through all tools and verifies they're properly set up.
2. **Test that the AgentExecutor in the BossNode is correctly instantiated:** This is an important component in the BossNode and it's important to make sure it's functioning correctly.
3. **Test that the LLMChain in the BossNode works as expected:** This is another critical component of the BossNode, so it's worth having a test that specifically targets it.
Here is an example of what these tests could look like:
```python
def test_WorkerNode_tools():
worker_node = WorkerNode(llm, tools, vectorstore)
worker_node.create_agent("test_agent", "test_role", False, {})
# Check that all tools are instantiated
for tool in worker_node.tools:
assert tool is not None
def test_BossNode_AgentExecutor():
boss_node = BossNode(llm, vectorstore, task_execution_chain, False, 3)
# Check that the AgentExecutor is correctly initialized
assert boss_node.baby_agi.task_execution_chain is not None
def test_BossNode_LLMChain():
boss_node = BossNode(llm, vectorstore, task_execution_chain, False, 3)
# Check that the LLMChain in ZeroShotAgent is working
assert boss_node.baby_agi.task_execution_chain.agent.llm_chain is not None
```
As before, these tests are somewhat simplistic and primarily check for existence and instantiation. Real-world testing would likely involve more complex and specific tests for functionality and error-handling.
Loading…
Cancel
Save