swarms/docs/swarms/structs/concurrentworkflow.md

# ConcurrentWorkflow Documentation

## Overview

The `ConcurrentWorkflow` class is designed to facilitate the concurrent execution of multiple agents, each tasked with solving a specific query or problem. This class is particularly useful in scenarios where multiple agents need to work in parallel, allowing for efficient resource utilization and faster completion of tasks. The workflow manages the execution, collects metadata, and optionally saves the results in a structured format.

### Key Features

- **Concurrent Execution**: Runs multiple agents simultaneously using Python's `asyncio` and `ThreadPoolExecutor`.
- **Metadata Collection**: Gathers detailed metadata about each agent's execution, including start and end times, duration, and output.
- **Customizable Output**: Allows the user to save metadata to a file or return it as a string or dictionary.
- **Error Handling**: Catches and logs errors during agent execution, ensuring the workflow can continue.

## Class Definitions

### AgentOutputSchema

The `AgentOutputSchema` class is a data model that captures the output and metadata for each agent's execution. It inherits from `pydantic.BaseModel` and provides structured fields to store essential information.

| Attribute     | Type           | Description                                               |
|---------------|----------------|-----------------------------------------------------------|
| `run_id`      | `Optional[str]`| Unique ID for the run, automatically generated using `uuid`. |
| `agent_name`  | `Optional[str]`| Name of the agent that executed the task.                 |
| `task`        | `Optional[str]`| The task or query given to the agent.                     |
| `output`      | `Optional[str]`| The output generated by the agent.                        |
| `start_time`  | `Optional[datetime]`| The time when the agent started the task.         |
| `end_time`    | `Optional[datetime]`| The time when the agent completed the task.        |
| `duration`    | `Optional[float]` | The total time taken to complete the task, in seconds. |

### MetadataSchema

The `MetadataSchema` class is another data model that aggregates the outputs from all agents involved in the workflow. It also inherits from `pydantic.BaseModel` and includes fields for additional workflow-level metadata.

| Attribute      | Type                   | Description                                               |
|----------------|------------------------|-----------------------------------------------------------|
| `swarm_id`     | `Optional[str]`         | Unique ID for the workflow run, generated using `uuid`.   |
| `task`         | `Optional[str]`         | The task or query given to all agents.                    |
| `description`  | `Optional[str]`         | A description of the workflow, typically indicating concurrent execution. |
| `agents`       | `Optional[List[AgentOutputSchema]]` | A list of agent outputs and metadata.  |
| `timestamp`    | `Optional[datetime]`    | The timestamp when the workflow was executed.             |

## ConcurrentWorkflow

The `ConcurrentWorkflow` class is the core class that manages the concurrent execution of agents. It inherits from `BaseSwarm` and includes several key attributes and methods to facilitate this process.

### Attributes

| Attribute              | Type                    | Description                                               |
|------------------------|-------------------------|-----------------------------------------------------------|
| `name`                 | `str`                   | The name of the workflow. Defaults to `"ConcurrentWorkflow"`. |
| `description`          | `str`                   | A brief description of the workflow.                      |
| `agents`               | `List[Agent]`           | A list of agents to be executed concurrently.             |
| `metadata_output_path` | `str`                   | Path to save the metadata output. Defaults to `"agent_metadata.json"`. |
| `auto_save`            | `bool`                  | Flag indicating whether to automatically save the metadata. |
| `output_schema`        | `BaseModel`             | The output schema for the metadata, defaults to `MetadataSchema`. |
| `max_loops`            | `int`                   | Maximum number of loops for the workflow, defaults to `1`. |
| `return_str_on`        | `bool`                  | Flag to return output as string. Defaults to `False`.     |
| `agent_responses`      | `List[str]`             | List of agent responses as strings.                       |

## Methods

### ConcurrentWorkflow.\_\_init\_\_

Initializes the `ConcurrentWorkflow` class with the provided parameters.

#### Parameters

| Parameter             | Type           | Default Value                          | Description                                               |
|-----------------------|----------------|----------------------------------------|-----------------------------------------------------------|
| `name`                | `str`          | `"ConcurrentWorkflow"`                 | The name of the workflow.                                 |
| `description`         | `str`          | `"Execution of multiple agents concurrently"` | A brief description of the workflow.               |
| `agents`              | `List[Agent]`  | `[]`                                   | A list of agents to be executed concurrently.             |
| `metadata_output_path`| `str`          | `"agent_metadata.json"`                | Path to save the metadata output.                         |
| `auto_save`           | `bool`         | `False`                                | Flag indicating whether to automatically save the metadata. |
| `output_schema`       | `BaseModel`    | `MetadataSchema`                       | The output schema for the metadata.                       |
| `max_loops`           | `int`          | `1`                                    | Maximum number of loops for the workflow.                 |
| `return_str_on`       | `bool`         | `False`                                | Flag to return output as string.                          |
| `agent_responses`     | `List[str]`    | `[]`                                   | List of agent responses as strings.                       |

#### Raises

- `ValueError`: If the list of agents is empty or if the description is empty.

### ConcurrentWorkflow._run_agent

Runs a single agent with the provided task and tracks its output and metadata.

#### Parameters

| Parameter   | Type                    | Description                                               |
|-------------|-------------------------|-----------------------------------------------------------|
| `agent`     | `Agent`                 | The agent instance to run.                                |
| `task`      | `str`                   | The task or query to give to the agent.                   |
| `executor`  | `ThreadPoolExecutor`    | The thread pool executor to use for running the agent task. |

#### Returns

- `AgentOutputSchema`: The metadata and output from the agent's execution.

#### Detailed Explanation

This method handles the execution of a single agent by offloading the task to a thread using `ThreadPoolExecutor`. It also tracks the time taken by the agent to complete the task and logs relevant information. If an exception occurs during execution, it captures the error and includes it in the output.

### ConcurrentWorkflow.transform_metadata_schema_to_str

Transforms the metadata schema into a string format.

#### Parameters

| Parameter   | Type                | Description                                               |
|-------------|---------------------|-----------------------------------------------------------|
| `schema`    | `MetadataSchema`    | The metadata schema to transform.                         |

#### Returns

- `str`: The metadata schema as a formatted string.

#### Detailed Explanation

This method converts the metadata stored in `MetadataSchema` into a human-readable string format, particularly focusing on the agent names and their respective outputs. This is useful for quickly reviewing the results of the concurrent workflow in a more accessible format.

### ConcurrentWorkflow._execute_agents_concurrently

Executes multiple agents concurrently with the same task.

#### Parameters

| Parameter   | Type         | Description                                               |
|-------------|--------------|-----------------------------------------------------------|
| `task`      | `str`        | The task or query to give to all agents.                  |

#### Returns

- `MetadataSchema`: The aggregated metadata and outputs from all agents.

#### Detailed Explanation

This method is responsible for managing the concurrent execution of all agents. It uses `asyncio.gather` to run multiple agents simultaneously and collects their outputs into a `MetadataSchema` object. This aggregated metadata can then be saved or returned depending on the workflow configuration.

### ConcurrentWorkflow.run

Runs the workflow for the provided task, executes agents concurrently, and saves metadata.

#### Parameters

| Parameter   | Type         | Description                                               |
|-------------|--------------|-----------------------------------------------------------|
| `task`      | `str`        | The task or query to give to all agents.                  |

#### Returns

- `Dict[str, Any]`: The final metadata as a dictionary.

#### Detailed Explanation

This is the main method that a user will call to execute the workflow. It manages the entire process from starting the agents to collecting and optionally saving the metadata. The method also provides flexibility in how the results are returned—either as a JSON dictionary or as a formatted string.

## Usage Examples

### Example 1: Basic Usage

```python
import os

from swarms import Agent, ConcurrentWorkflow, OpenAIChat

# Initialize agents
model = OpenAIChat(
    api_key=os.getenv("OPENAI_API_KEY"),
    model_name="gpt-4o-mini",
    temperature=0.1,
)


# Define custom system prompts for each social media platform
TWITTER_AGENT_SYS_PROMPT = """
You are a Twitter marketing expert specializing in real estate. Your task is to create engaging, concise tweets to promote properties, analyze trends to maximize engagement, and use appropriate hashtags and timing to reach potential buyers.
"""

INSTAGRAM_AGENT_SYS_PROMPT = """
You are an Instagram marketing expert focusing on real estate. Your task is to create visually appealing posts with engaging captions and hashtags to showcase properties, targeting specific demographics interested in real estate.
"""

FACEBOOK_AGENT_SYS_PROMPT = """
You are a Facebook marketing expert for real estate. Your task is to craft posts optimized for engagement and reach on Facebook, including using images, links, and targeted messaging to attract potential property buyers.
"""

LINKEDIN_AGENT_SYS_PROMPT = """
You are a LinkedIn marketing expert for the real estate industry. Your task is to create professional and informative posts, highlighting property features, market trends, and investment opportunities, tailored to professionals and investors.
"""

EMAIL_AGENT_SYS_PROMPT = """
You are an Email marketing expert specializing in real estate. Your task is to write compelling email campaigns to promote properties, focusing on personalization, subject lines, and effective call-to-action strategies to drive conversions.
"""

# Initialize your agents for different social media platforms
agents = [
    Agent(
        agent_name="Twitter-RealEstate-Agent",
        system_prompt=TWITTER_AGENT_SYS_PROMPT,
        llm=model,
        max_loops=1,
        dynamic_temperature_enabled=True,
        saved_state_path="twitter_realestate_agent.json",
        user_name="swarm_corp",
        retry_attempts=1,
    ),
    Agent(
        agent_name="Instagram-RealEstate-Agent",
        system_prompt=INSTAGRAM_AGENT_SYS_PROMPT,
        llm=model,
        max_loops=1,
        dynamic_temperature_enabled=True,
        saved_state_path="instagram_realestate_agent.json",
        user_name="swarm_corp",
        retry_attempts=1,
    ),
    Agent(
        agent_name="Facebook-RealEstate-Agent",
        system_prompt=FACEBOOK_AGENT_SYS_PROMPT,
        llm=model,
        max_loops=1,
        dynamic_temperature_enabled=True,
        saved_state_path="facebook_realestate_agent.json",
        user_name="swarm_corp",
        retry_attempts=1,
    ),
    Agent(
        agent_name="LinkedIn-RealEstate-Agent",
        system_prompt=LINKEDIN_AGENT_SYS_PROMPT,
        llm=model,
        max_loops=1,
        dynamic_temperature_enabled=True,
        saved_state_path="linkedin_realestate_agent.json",
        user_name="swarm_corp",
        retry_attempts=1,
    ),
    Agent(
        agent_name="Email-RealEstate-Agent",
        system_prompt=EMAIL_AGENT_SYS_PROMPT,
        llm=model,
        max_loops=1,
        dynamic_temperature_enabled=True,
        saved_state_path="email_realestate_agent.json",
        user_name="swarm_corp",
        retry_attempts=1,
    ),
]

# Initialize workflow
workflow = ConcurrentWorkflow(
    name = "Real Estate Marketing Swarm",
    agents=agents,
    metadata_output_path="metadata.json",
    description="Concurrent swarm of content generators for real estate!",
    auto_save=True,
)

# Run workflow
task = "Analyze the financial impact of a new product launch."
metadata = workflow.run(task)
print(metadata)

```

### Example 2: Custom Output Handling

```python
# Run workflow with string output
workflow = ConcurrentWorkflow(agents=agents, return_str_on=True)
metadata_str = workflow.run(task)
print(metadata_str)
```

### Example 3: Error Handling and Debugging

```python
try:
    metadata = workflow.run(task)
except ValueError as e:
    print(f"An error occurred: {e}")
```

## Tips and Best Practices

- **Agent Initialization**: Ensure that all agents are correctly initialized with their required configurations before passing them to `ConcurrentWorkflow`.
- **Metadata Management**: Use the `auto_save` flag to automatically save metadata if you plan to run multiple workflows in succession.
- **Concurrency Limits**: Adjust the number of agents based on your system's capabilities to avoid overloading resources.
- **Error Handling**: Implement try-except blocks when running workflows to catch and handle exceptions gracefully.

## References and Resources

- [Python's `asyncio` Documentation](https://docs.python.org/3/library/asyncio.html)
- [Pydantic Documentation](https://pydantic-docs.helpmanual.io/)
- [ThreadPoolExecutor in Python](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor)
- [Loguru for Logging in Python](https://loguru.readthedocs.io/en/stable/)