@ -6,16 +6,17 @@ The `ConcurrentWorkflow` class is designed to facilitate the concurrent executio
### Key Features
### Key Features
- **Concurrent Execution**: Runs multiple agents simultaneously using Python's `asyncio` and `ThreadPoolExecutor`.
- **Concurrent Execution**: Runs multiple agents simultaneously using Python's `ThreadPoolExecutor`
- **Metadata Collection**: Gathers detailed metadata about each agent's execution, including start and end times, duration, and output.
- **Interactive Mode**: Supports interactive task modification and execution
- **Customizable Output**: Allows the user to save metadata to a file or return it as a string or dictionary.
- **Caching System**: Implements LRU caching for repeated prompts
- **Error Handling**: Implements retry logic for improved reliability.
- **Progress Tracking**: Optional progress bar for task execution
- **Batch Processing**: Supports running tasks in batches and parallel execution.
- **Enhanced Error Handling**: Implements retry mechanism with exponential backoff
- **Asynchronous Execution**: Provides asynchronous run options for improved performance.
- **Input Validation**: Validates task inputs before execution
- **Batch Processing**: Supports running tasks in batches
## Class Definitions
- **Metadata Collection**: Gathers detailed metadata about each agent's execution
- **Customizable Output**: Allows saving metadata to file or returning as string/dictionary
The `ConcurrentWorkflow` class is the core class that manages the concurrent execution of agents. It inherits from `BaseSwarm` and includes several key attributes and methods to facilitate this process.
## Class Definition
### Attributes
### Attributes
@ -26,15 +27,18 @@ The `ConcurrentWorkflow` class is the core class that manages the concurrent exe
| `agents` | `List[Agent]` | A list of agents to be executed concurrently. |
| `agents` | `List[Agent]` | A list of agents to be executed concurrently. |
| `metadata_output_path` | `str` | Path to save the metadata output. Defaults to `"agent_metadata.json"`. |
| `metadata_output_path` | `str` | Path to save the metadata output. Defaults to `"agent_metadata.json"`. |
| `auto_save` | `bool` | Flag indicating whether to automatically save the metadata. |
| `auto_save` | `bool` | Flag indicating whether to automatically save the metadata. |
| `output_schema` | `BaseModel` | The output schema for the metadata, defaults to `MetadataSchema`. |
| `output_type` | `str` | The type of output format. Defaults to `"dict"`. |
| `max_loops` | `int` | Maximum number of loops for the workflow, defaults to `1`. |
| `max_loops` | `int` | Maximum number of loops for each agent. Defaults to `1`. |
| `return_str_on` | `bool` | Flag to return output as string. Defaults to `False`. |
| `return_str_on` | `bool` | Flag to return output as string. Defaults to `False`. |
| `agent_responses` | `List[str]` | List of agent responses as strings. |
| `auto_generate_prompts`| `bool` | Flag indicating whether to auto-generate prompts for agents. |
| `auto_generate_prompts`| `bool` | Flag indicating whether to auto-generate prompts for agents. |
| `output_type` | `OutputType` | Type of output format to return. Defaults to `"dict"`. |
| `return_entire_history`| `bool` | Flag to return entire conversation history. Defaults to `False`. |
| `return_entire_history`| `bool` | Flag to return entire conversation history. Defaults to `False`. |
@ -134,136 +157,71 @@ Runs the workflow for a batch of tasks.
#### Returns
#### Returns
- `List[Union[Dict[str, Any], str]]`: A list of final metadata for each task.
- `List[Any]`: A list of results for each task.
#### Example
```python
tasks = ["Task 1", "Task 2"]
results = workflow.run_batched(tasks)
print(results)
```
## Usage Examples
## Usage Examples
### Example 1: Basic Usage
### Example 1: Basic Usage with Interactive Mode
```python
```python
import os
from swarms import Agent, ConcurrentWorkflow
from swarms import Agent, ConcurrentWorkflow, OpenAIChat
# Initialize agents
# Define custom system prompts for each social media platform
TWITTER_AGENT_SYS_PROMPT = """
You are a Twitter marketing expert specializing in real estate. Your task is to create engaging, concise tweets to promote properties, analyze trends to maximize engagement, and use appropriate hashtags and timing to reach potential buyers.
"""
INSTAGRAM_AGENT_SYS_PROMPT = """
You are an Instagram marketing expert focusing on real estate. Your task is to create visually appealing posts with engaging captions and hashtags to showcase properties, targeting specific demographics interested in real estate.
"""
FACEBOOK_AGENT_SYS_PROMPT = """
You are a Facebook marketing expert for real estate. Your task is to craft posts optimized for engagement and reach on Facebook, including using images, links, and targeted messaging to attract potential property buyers.
"""
LINKEDIN_AGENT_SYS_PROMPT = """
You are a LinkedIn marketing expert for the real estate industry. Your task is to create professional and informative posts, highlighting property features, market trends, and investment opportunities, tailored to professionals and investors.
"""
EMAIL_AGENT_SYS_PROMPT = """
You are an Email marketing expert specializing in real estate. Your task is to write compelling email campaigns to promote properties, focusing on personalization, subject lines, and effective call-to-action strategies to drive conversions.
"""
# Initialize your agents for different social media platforms
description="Concurrent swarm of content generators for real estate!",
show_progress=True,
auto_save=True,
cache_size=100,
max_retries=3,
retry_delay=1.0
)
)
# Run workflow
# Run workflow
task = "Create a marketing campaign for a luxury beachfront property in Miami, focusing on its stunning ocean views, private beach access, and state-of-the-art amenities."
task = "What are the benefits of using Python for data analysis?"
metadata = workflow.run(task)
result = workflow.run(task)
print(metadata)
print(result)
```
```
### Example 2: Custom Output Handling
### Example 2: Batch Processing with Progress Bar
```python
```python
# Initialize workflow with string output
# Initialize workflow
workflow = ConcurrentWorkflow(
workflow = ConcurrentWorkflow(
name="Real Estate Marketing Swarm",
name="Batch Processing Workflow",
agents=agents,
agents=agents,
metadata_output_path="metadata.json",
show_progress=True,
description="Concurrent swarm of content generators for real estate!",
auto_save=True
auto_save=True,
return_str_on=True
)
)
# Run workflow
# Define tasks
task = "Develop a marketing strategy for a newly renovated historic townhouse in Boston, emphasizing its blend of classic architecture and modern amenities."
tasks = [
metadata_str = workflow.run(task)
"Analyze the impact of climate change on agriculture",
print(metadata_str)
"Evaluate renewable energy solutions",
"Assess water conservation strategies"
]
# Run batch processing
results = workflow.run_batched(tasks)
# Process results
for task, result in zip(tasks, results):
print(f"Task: {task}")
print(f"Result: {result}\n")
```
```
### Example 3: Error Handling and Debugging
### Example 3: Error Handling and Retries
```python
```python
import logging
import logging
@ -271,71 +229,38 @@ import logging
# Set up logging
# Set up logging
logging.basicConfig(level=logging.INFO)
logging.basicConfig(level=logging.INFO)
# Initialize workflow
# Initialize workflow with retry settings
workflow = ConcurrentWorkflow(
workflow = ConcurrentWorkflow(
name="Real Estate Marketing Swarm",
name="Reliable Workflow",
agents=agents,
agents=agents,
metadata_output_path="metadata.json",
max_retries=3,
description="Concurrent swarm of content generators for real estate!",
retry_delay=1.0,
auto_save=True
show_progress=True
)
)
# Run workflow with error handling
# Run workflow with error handling
try:
try:
task = "Create a marketing campaign for a eco-friendly tiny house community in Portland, Oregon."
task = "Generate a comprehensive market analysis report"
metadata = workflow.run(task)
result = workflow.run(task)
print(metadata)
print(result)
except Exception as e:
except Exception as e:
logging.error(f"An error occurred during workflow execution: {str(e)}")
logging.error(f"An error occurred: {str(e)}")
# Additional error handling or debugging steps can be added here
```
### Example 4: Batch Processing
```python
# Initialize workflow
workflow = ConcurrentWorkflow(
name="Real Estate Marketing Swarm",
agents=agents,
metadata_output_path="metadata_batch.json",
description="Concurrent swarm of content generators for real estate!",
auto_save=True
)
# Define a list of tasks
tasks = [
"Market a family-friendly suburban home with a large backyard and excellent schools nearby.",
"Promote a high-rise luxury apartment in New York City with panoramic skyline views.",
"Advertise a ski-in/ski-out chalet in Aspen, Colorado, perfect for winter sports enthusiasts."
]
# Run workflow in batch mode
results = workflow.run_batched(tasks)
# Process and print results
for task, result in zip(tasks, results):
print(f"Task: {task}")
print(f"Result: {result}\n")
```
```
## Tips and Best Practices
## Tips and Best Practices
- **Agent Initialization**: Ensure that all agents are correctly initialized with their required configurations before passing them to `ConcurrentWorkflow`.
- **Agent Initialization**: Ensure all agents are correctly initialized with required configurations.
- **Metadata Management**: Use the `auto_save` flag to automatically save metadata if you plan to run multiple workflows in succession.
- **Interactive Mode**: Use interactive mode for tasks requiring user input or modification.
- **Concurrency Limits**: Adjust the number of agents based on your system's capabilities to avoid overloading resources.
- **Caching**: Utilize the caching system for repeated tasks to improve performance.
- **Error Handling**: Implement try-except blocks when running workflows to catch and handle exceptions gracefully.
- **Progress Tracking**: Enable progress bar for long-running tasks to monitor execution.
- **Batch Processing**: For large numbers of tasks, consider using `run_batched` or `run_parallel` methods to improve overall throughput.
- **Error Handling**: Implement proper error handling and use retry mechanism for reliability.
- **Asynchronous Operations**: Utilize asynchronous methods (`run_async`, `run_batched_async`, `run_parallel_async`) when dealing with I/O-bound tasks or when you need to maintain responsiveness in your application.
- **Resource Management**: Monitor cache size and clear when necessary.
- **Logging**: Implement detailed logging to track the progress of your workflows and troubleshoot any issues that may arise.
- **Batch Processing**: Use batch processing for multiple related tasks.
- **Resource Management**: Be mindful of API rate limits and resource consumption, especially when running large batches or parallel executions.
- **Logging**: Implement detailed logging for debugging and monitoring.
- **Testing**: Thoroughly test your workflows with various inputs and edge cases to ensure robust performance in production environments.
@ -333,9 +362,11 @@ class ConcurrentWorkflow(BaseSwarm):
**kwargs,
**kwargs,
)->Any:
)->Any:
"""
"""
Executestheagent's run method on a specified device.
Executestheagent's run method on a specified device with optional interactive mode.
Thismethodattemptstoexecutetheagent's run method on a specified device, either CPU or GPU. It logs the device selection and the number of cores or GPU ID used. If the device is set to CPU, it can use all available cores or a specific core specified by `device_id`. If the device is set to GPU, it uses the GPU specified by `device_id`.
Thismethodattemptstoexecutetheagent's run method on a specified device, either CPU or GPU.
Kye Gomez
1 day ago