From d2d3dd73ff286d30510b86007496217575e9f772 Mon Sep 17 00:00:00 2001 From: Hugh <155223694+hughiwnl@users.noreply.github.com> Date: Wed, 19 Nov 2025 20:42:58 -0800 Subject: [PATCH] changed docs for agent rearrange --- docs/swarms/structs/agent_rearrange.md | 344 +++++++++++++++++-------- 1 file changed, 230 insertions(+), 114 deletions(-) diff --git a/docs/swarms/structs/agent_rearrange.md b/docs/swarms/structs/agent_rearrange.md index 9ecfcf1a..375b275e 100644 --- a/docs/swarms/structs/agent_rearrange.md +++ b/docs/swarms/structs/agent_rearrange.md @@ -1,6 +1,23 @@ # `AgentRearrange` Class -The `AgentRearrange` class represents a swarm of agents for rearranging tasks. It allows you to create a swarm of agents, add or remove agents from the swarm, and run the swarm to process tasks based on a specified flow pattern. The class now includes **sequential awareness** features that allow agents to know about the agents ahead and behind them in sequential flows. +The `AgentRearrange` class represents a swarm of agents for task rearrangement and orchestration. It enables complex workflows where multiple agents can work sequentially or concurrently based on a defined flow pattern. The class supports both sequential execution (using '->') and concurrent execution (using ',') within the same workflow, and includes **sequential awareness** features that allow agents to know about the agents ahead and behind them in sequential flows. + +## Key Features + +- Sequential and concurrent agent execution +- Custom flow patterns with arrow (->) and comma (,) syntax +- Team awareness and sequential flow information +- Human-in-the-loop integration +- Memory system support +- Batch and concurrent processing capabilities +- Comprehensive error handling and logging + +## Flow Syntax + +- Use '->' to define sequential execution: `"agent1 -> agent2 -> agent3"` +- Use ',' to define concurrent execution: `"agent1, agent2 -> agent3"` +- Combine both: `"agent1 -> agent2, agent3 -> agent4"` +- Use 'H' for human-in-the-loop: `"agent1 -> H -> agent2"` ## Attributes ---------- @@ -14,7 +31,7 @@ The `AgentRearrange` class represents a swarm of agents for rearranging tasks. I | `flow` | `str` | Flow pattern defining task execution order | | `max_loops` | `int` | Maximum number of execution loops | | `verbose` | `bool` | Whether to enable verbose logging | -| `memory_system` | `BaseVectorDatabase` | Memory system for storing agent interactions | +| `memory_system` | `Any` | Optional memory system for persistence | | `human_in_the_loop` | `bool` | Whether human intervention is enabled | | `custom_human_in_the_loop` | `Callable` | Custom function for human intervention | | `output_type` | `OutputType` | Format of output ("all", "final", "list", or "dict") | @@ -23,6 +40,7 @@ The `AgentRearrange` class represents a swarm of agents for rearranging tasks. I | `team_awareness` | `bool` | Whether to enable team awareness and sequential flow information | | `time_enabled` | `bool` | Whether to enable timestamps in conversation | | `message_id_on` | `bool` | Whether to enable message IDs in conversation | +| `conversation` | `Conversation` | Conversation history management | ## Methods ------- @@ -31,6 +49,10 @@ The `AgentRearrange` class represents a swarm of agents for rearranging tasks. I Initializes the `AgentRearrange` object with enhanced sequential awareness capabilities. +**Note:** +- The `reliability_check()` method is automatically called during initialization to validate all critical parameters (agents, max_loops, flow, output_type). +- If validation fails, a `ValueError` will be raised before the object is created. + | Parameter | Type | Description | | --- | --- | --- | | `id` | `str` (optional) | Unique identifier for the swarm. Defaults to auto-generated UUID. | @@ -74,6 +96,51 @@ Adds multiple agents to the swarm. | --- | --- | --- | | `agents` | `List[Agent]` | A list of `Agent` objects. | +### `reliability_check(self)` + +Validates the configuration parameters to ensure the system can run properly. This method is called automatically during initialization to ensure the system is properly configured before execution. + +**Raises:** + +- `ValueError`: If any of the following conditions are met: + - agents list is None or empty + - max_loops is 0 + - flow is None or empty string + - output_type is None or empty string + +**Note:** +- This method is called automatically during `__init__` to prevent runtime errors. +- You can call this method manually to validate configuration after making changes. + +### `set_custom_flow(self, flow: str)` + +Sets a custom flow pattern for agent execution. Allows dynamic modification of the execution flow after initialization. + +| Parameter | Type | Description | +| --- | --- | --- | +| `flow` | `str` | The new flow pattern to use for agent execution. Must follow the syntax: "agent1 -> agent2, agent3 -> agent4" | + +**Example:** +```python +rearrange_system.set_custom_flow("researcher -> writer, editor") +``` + +**Note:** +- The flow will be validated on the next execution. If invalid, a `ValueError` will be raised during the `run()` method. + +### `track_history(self, agent_name: str, result: str)` + +Tracks the execution history for a specific agent. Records the result of an agent's execution in the swarm history for later analysis or debugging purposes. + +| Parameter | Type | Description | +| --- | --- | --- | +| `agent_name` | `str` | The name of the agent whose result to track. | +| `result` | `str` | The result/output from the agent's execution. | + +**Note:** +- This method is typically called internally during agent execution to maintain a complete history of all agent activities. +- Can be called manually to add custom history entries. + ### `validate_flow(self)` Validates the flow pattern. @@ -125,18 +192,42 @@ flow_structure = agent_system.get_sequential_flow_structure() ### `run(self, task: str = None, img: str = None, *args, **kwargs)` -Executes the agent rearrangement task with specified compute resources. +Executes the agent rearrangement task with comprehensive logging and error handling. This is the main public method for executing tasks through the agent rearrange system. + +| Parameter | Type | Description | +| --- | --- | --- | +| `task` | `str` (optional) | The task to execute through the agent workflow. Defaults to `None`. | +| `img` | `str` (optional) | Path to input image if required by any agents. Defaults to `None`. | +| `*args` | - | Additional positional arguments passed to the internal `_run()` method. | +| `**kwargs` | - | Additional keyword arguments passed to the internal `_run()` method. | + +**Returns:** + +- The result from executing the task through the agent rearrange system. The format depends on the configured `output_type`. + +**Note:** +- This method automatically logs agent data before and after execution for telemetry and debugging purposes. +- Any exceptions are caught and handled by the `_catch_error()` method. + +### `__call__(self, task: str, *args, **kwargs)` + +Makes the class callable by executing the `run()` method. Enables the `AgentRearrange` instance to be called directly as a function. | Parameter | Type | Description | | --- | --- | --- | -| `task` | `str` (optional) | The task to execute. Defaults to `None`. | -| `img` | `str` (optional) | Path to input image if required. Defaults to `None`. | -| `*args` | - | Additional positional arguments passed to `_run()`. | -| `**kwargs` | - | Additional keyword arguments passed to `_run()`. | +| `task` | `str` | The task to execute through the agent workflow. | +| `*args` | - | Additional positional arguments passed to `run()`. | +| `**kwargs` | - | Additional keyword arguments passed to `run()`. | **Returns:** -- The result from executing the task through the cluster operations wrapper. +- The result from executing the task through the agent rearrange system. + +**Example:** +```python +rearrange_system = AgentRearrange(agents=[agent1, agent2], flow="agent1 -> agent2") +result = rearrange_system("Process this data") +``` ### `batch_run(self, tasks: List[str], img: Optional[List[str]] = None, batch_size: int = 10, *args, **kwargs)` @@ -156,23 +247,40 @@ Process multiple tasks in batches. ### `concurrent_run(self, tasks: List[str], img: Optional[List[str]] = None, max_workers: Optional[int] = None, *args, **kwargs)` -Process multiple tasks concurrently using ThreadPoolExecutor. +Process multiple tasks concurrently using ThreadPoolExecutor. This method enables true parallel processing of multiple tasks by using Python's ThreadPoolExecutor to run tasks simultaneously across multiple threads. | Parameter | Type | Description | | --- | --- | --- | -| `tasks` | `List[str]` | List of tasks to process | -| `img` | `List[str]` (optional) | Optional list of images corresponding to tasks | -| `max_workers` | `int` (optional) | Maximum number of worker threads | -| `*args` | - | Additional positional arguments | -| `**kwargs` | - | Additional keyword arguments | +| `tasks` | `List[str]` | List of tasks to process through the agent workflow | +| `img` | `List[str]` (optional) | Optional list of images corresponding to tasks. Must be the same length as tasks list. Defaults to `None`. | +| `max_workers` | `int` (optional) | Maximum number of worker threads to use. If `None`, uses the default ThreadPoolExecutor behavior. Defaults to `None`. | +| `*args` | - | Additional positional arguments passed to individual task execution | +| `**kwargs` | - | Additional keyword arguments passed to individual task execution | **Returns:** -- `List[str]`: List of results corresponding to input tasks +- `List[str]`: List of results corresponding to input tasks in the same order + +**Note:** +- This method uses ThreadPoolExecutor for true parallel execution. +- The number of concurrent executions is limited by `max_workers` parameter. +- Each task runs independently through the full agent workflow. + +### `to_dict(self) -> Dict[str, Any]` + +Converts all attributes of the class, including callables, into a dictionary. This method provides a comprehensive serialization of the `AgentRearrange` instance, converting all attributes into a dictionary format suitable for storage, logging, or transmission. + +**Returns:** + +- `Dict[str, Any]`: A dictionary representation of all class attributes. Non-serializable objects are converted to string representations or serialized using their `to_dict()` method if available. + +**Note:** +- This method is used for telemetry logging and state persistence. +- It recursively handles nested objects and provides fallback handling for objects that cannot be directly serialized. ## **Sequential Awareness Feature** -The `AgentRearrange` class now includes a powerful **sequential awareness** feature that enhances agent collaboration in sequential workflows. When agents are executed sequentially, they automatically receive information about: +The `AgentRearrange` class now includes a **sequential awareness** feature that enhances agent collaboration in sequential workflows. When agents are executed sequentially, they automatically receive information about: - **Agent ahead**: The agent that completed their task before them - **Agent behind**: The agent that will receive their output next @@ -214,21 +322,21 @@ result = workflow.run("Research and write about artificial intelligence") ## Documentation for `rearrange` Function ====================================== -The `rearrange` function is a helper function that rearranges the given list of agents based on the specified flow. +The rearrange function is a helper function that rearranges the given list of agents based on the specified flow. ## Parameters ---------- | Parameter | Type | Description | | --- | --- | --- | -| `name` | `str` (optional) | Name for the agent system. Defaults to `None`. | -| `description` | `str` (optional) | Description for the agent system. Defaults to `None`. | -| `agents` | `List[Agent]` | The list of agents to be rearranged. | -| `flow` | `str` | The flow used for rearranging the agents. | +| `name` | `str` (optional) | Name for the agent rearrange system. Defaults to `None` (uses AgentRearrange default). | +| `description` | `str` (optional) | Description of the system. Defaults to `None` (uses AgentRearrange default). | +| `agents` | `List[Agent]` | The list of agents to be included in the system. | +| `flow` | `str` | The flow pattern defining agent execution order. Uses '->' for sequential and ',' for concurrent execution. | | `task` | `str` (optional) | The task to be performed during rearrangement. Defaults to `None`. | -| `img` | `str` (optional) | Path to input image if required. Defaults to `None`. | -| `*args` | - | Additional positional arguments. | -| `**kwargs` | - | Additional keyword arguments. | +| `img` | `str` (optional) | Image input for agents that support it. Defaults to `None`. | +| `*args` | - | Additional positional arguments passed to `AgentRearrange` constructor. | +| `**kwargs` | - | Additional keyword arguments passed to `AgentRearrange` constructor. | ## Returns ------- @@ -239,10 +347,19 @@ The result of running the agent system with the specified task. ------- ```python -agents = [agent1, agent2, agent3] -flow = "agent1 -> agent2, agent3" -task = "Perform a task" -rearrange(agents, flow, task) +from swarms import Agent, rearrange + +# Create agents +agent1 = Agent(agent_name="researcher", system_prompt="Research topics") +agent2 = Agent(agent_name="writer", system_prompt="Write content") +agent3 = Agent(agent_name="reviewer", system_prompt="Review content") + +# Execute task with flow +result = rearrange( + agents=[agent1, agent2, agent3], + flow="researcher -> writer, reviewer", + task="Research and write a report" +) ``` ### Example Usage @@ -252,81 +369,38 @@ Here's an example of how to use the `AgentRearrange` class and the `rearrange` f ```python from swarms import Agent, AgentRearrange -from typing import List - -# Initialize the director agent -director = Agent( - agent_name="Accounting Director", - system_prompt="Directs the accounting tasks for the workers", - llm=Anthropic(), - max_loops=1, - dashboard=False, - streaming_on=True, - verbose=True, - stopping_token="", - state_save_file_type="json", - saved_state_path="accounting_director.json", -) - -# Initialize worker 1 -worker1 = Agent( - agent_name="Accountant 1", - system_prompt="Processes financial transactions and prepares financial statements", - llm=Anthropic(), - max_loops=1, - dashboard=False, - streaming_on=True, - verbose=True, - stopping_token="", - state_save_file_type="json", - saved_state_path="accountant1.json", -) - -# Initialize worker 2 -worker2 = Agent( - agent_name="Accountant 2", - system_prompt="Performs audits and ensures compliance with financial regulations", - llm=Anthropic(), - max_loops=1, - dashboard=False, - streaming_on=True, - verbose=True, - stopping_token="", - state_save_file_type="json", - saved_state_path="accountant2.json", -) -# Create a list of agents -agents = [director, worker1, worker2] - -# Define the flow pattern -flow = "Accounting Director -> Accountant 1 -> Accountant 2" +# Create agents +agent1 = Agent(agent_name="researcher", system_prompt="Research the topic") +agent2 = Agent(agent_name="writer", system_prompt="Write based on research") +agent3 = Agent(agent_name="reviewer", system_prompt="Review the written content") -# Using AgentRearrange class with sequential awareness -agent_system = AgentRearrange( - agents=agents, - flow=flow, +# Create sequential workflow +workflow = AgentRearrange( + agents=[agent1, agent2, agent3], + flow="researcher -> writer -> reviewer", team_awareness=True, # Enables sequential awareness - time_enabled=True, # Enable timestamps - message_id_on=True # Enable message IDs + time_enabled=True, # Enable timestamps + message_id_on=True # Enable message IDs ) # Get sequential flow information -flow_structure = agent_system.get_sequential_flow_structure() +flow_structure = workflow.get_sequential_flow_structure() print("Flow Structure:", flow_structure) # Get awareness for specific agents -worker1_awareness = agent_system.get_agent_sequential_awareness("Accountant 1") -print("Worker1 Awareness:", worker1_awareness) +writer_awareness = workflow.get_agent_sequential_awareness("writer") +print("Writer Awareness:", writer_awareness) # Run the workflow -output = agent_system.run("Process monthly financial statements") +output = workflow.run("Research and write about artificial intelligence") print(output) -``` -In this example, we first initialize three agents: `director`, `worker1`, and `worker2`. Then, we create a list of these agents and define the flow pattern `"Director -> Worker1 -> Worker2"`. +# Or use the callable interface +result = workflow("Research and write about machine learning") +``` -The new sequential awareness features provide: +In this example, we create three agents and define a sequential flow pattern. The sequential awareness features provide: - **Automatic context**: Each agent knows who came before and who comes after - **Better coordination**: Agents can reference previous work and prepare for next steps - **Flow visualization**: You can see the complete workflow structure @@ -335,61 +409,88 @@ The new sequential awareness features provide: ## Error Handling -------------- -The `AgentRearrange` class includes error handling mechanisms to validate the flow pattern. If the flow pattern is incorrectly formatted or contains duplicate agent names, a `ValueError` will be raised with an appropriate error message. +The `AgentRearrange` class includes comprehensive error handling mechanisms. During initialization, the `reliability_check()` method validates critical parameters: + +- **Agents validation**: Raises `ValueError` if agents list is None or empty +- **Max loops validation**: Raises `ValueError` if max_loops is 0 +- **Flow validation**: Raises `ValueError` if flow is None or empty +- **Output type validation**: Raises `ValueError` if output_type is None or empty + +The `validate_flow()` method checks the flow pattern format: +- Raises `ValueError` if the flow pattern doesn't include '->' to denote direction +- Raises `ValueError` if any agent in the flow is not registered ### Example: ```python -# Invalid flow pattern -invalid_flow = "Director->Worker1,Worker2->Worker3" -agent_system = AgentRearrange(agents=agents, flow=invalid_flow) -output = agent_system.run("Some task")` +# Invalid flow pattern - missing agent +invalid_flow = "agent1 -> agent2 -> agent3" +agent_system = AgentRearrange(agents=[agent1, agent2], flow=invalid_flow) +output = agent_system.run("Some task") ``` -This will raise a `ValueError` with the message `"Agent 'Worker3' is not registered."`. +This will raise a `ValueError` with the message `"Agent 'agent3' is not registered."`. + +All errors are automatically logged and, if `autosave` is enabled, the current state is saved before error reporting. ## Parallel and Sequential Processing ---------------------------------- -The `AgentRearrange` class supports both parallel and sequential processing of tasks based on the specified flow pattern. If the flow pattern includes multiple agents separated by commas (e.g., `"agent1, agent2"`), the agents will be executed in parallel, and their outputs will be concatenated. If the flow pattern includes a single agent, it will be executed sequentially with enhanced awareness. +The `AgentRearrange` class supports both parallel and sequential processing of tasks based on the specified flow pattern. The flow syntax determines execution mode: -### Parallel processing -`parallel_flow = "Worker1, Worker2 -> Director"` - -### Sequential processing with awareness -`sequential_flow = "Worker1 -> Worker2 -> Director"` +### Parallel Processing +When multiple agents are separated by commas, they execute concurrently: +```python +parallel_flow = "agent1, agent2 -> agent3" +``` +In this example, `agent1` and `agent2` will be executed in parallel using `run_agents_concurrently()`, and their outputs will be collected and passed to `agent3`. -In the `parallel_flow` example, `Worker1` and `Worker2` will be executed in parallel, and their outputs will be concatenated and passed to `Director`. +### Sequential Processing with Awareness +When agents are connected by arrows, they execute sequentially: +```python +sequential_flow = "agent1 -> agent2 -> agent3" +``` +In this example, `agent1` runs first, then `agent2` receives awareness that `agent1` came before and `agent3` comes after, and finally `agent3` receives awareness that `agent2` came before. -In the `sequential_flow` example, `Worker1` will be executed first, then `Worker2` will receive awareness that `Worker1` came before and `Director` comes after, and finally `Director` will receive awareness that `Worker2` came before. +### Combined Processing +You can combine both patterns: +```python +combined_flow = "agent1 -> agent2, agent3 -> agent4" +``` +This executes `agent1` first, then `agent2` and `agent3` run concurrently, and finally `agent4` receives both outputs. ## Logging and Monitoring ------- -The `AgentRearrange` class includes comprehensive logging capabilities using the `loguru` library. The new sequential awareness features add enhanced logging: +The `AgentRearrange` class includes comprehensive logging capabilities using the `loguru` library. Logs are stored in a dedicated "rearrange" folder. The sequential awareness features add enhanced logging: -```bash -2023-05-08 10:30:15.456 | INFO | agent_rearrange:__init__:34 - Adding agent Director to the swarm. -2023-05-08 10:30:15.457 | INFO | agent_rearrange:__init__:34 - Adding agent Worker1 to the swarm. -2023-05-08 10:30:15.457 | INFO | agent_rearrange:__init__:34 - Adding agent Worker2 to the swarm. -2023-05-08 10:30:15.458 | INFO | agent_rearrange:run:118 - Running agents in parallel: ['Worker1', 'Worker2'] -2023-05-08 10:30:15.459 | INFO | agent_rearrange:run:121 - Running agents sequentially: ['Director'] -2023-05-08 10:30:15.460 | INFO | agent_rearrange:run:125 - Added sequential awareness for Worker2: Sequential awareness: Agent ahead: Worker1 | Agent behind: Director -``` +- Flow validation messages +- Agent execution start/completion +- Sequential awareness information injection +- Concurrent execution coordination +- Error handling and recovery + +All agent data is automatically logged via telemetry before and after execution (if `autosave=True`), providing comprehensive monitoring and debugging capabilities. ## Additional Parameters --------------------- -The `AgentRearrange` class now accepts additional parameters for enhanced functionality: +The `AgentRearrange` class accepts comprehensive parameters for enhanced functionality: ```python agent_system = AgentRearrange( agents=agents, flow=flow, + max_loops=1, # Maximum execution loops team_awareness=True, # Enable sequential awareness time_enabled=True, # Enable conversation timestamps message_id_on=True, # Enable message IDs - verbose=True # Enable detailed logging + verbose=True, # Enable detailed logging + autosave=True, # Automatically save execution data + output_type="all", # Output format: "all", "final", "list", or "dict" + memory_system=None, # Optional memory system for persistence + human_in_the_loop=False, # Enable human interaction points + rules=None # Custom system rules and constraints ) ``` @@ -398,6 +499,19 @@ agent_system = AgentRearrange( The `AgentRearrange` class and the `rearrange` function can be customized and extended to suit specific use cases. The new sequential awareness features provide a foundation for building more sophisticated agent coordination systems. +## Internal Methods + +The following methods are used internally but may be useful to understand the system architecture: + +- `_run(task, img, custom_tasks, *args, **kwargs)`: Core execution method that orchestrates the workflow. The `custom_tasks` parameter (Dict[str, str]) allows overriding the main task for specific agents in the flow, enabling per-agent task customization. +- `_run_sequential_workflow()`: Handles sequential agent execution with awareness +- `_run_concurrent_workflow()`: Handles parallel agent execution using `run_agents_concurrently()` +- `_get_sequential_awareness()`: Generates awareness information for agents based on their position in the flow +- `_get_sequential_flow_info()`: Generates overall flow structure information showing the complete workflow +- `_catch_error()`: Comprehensive error handling with logging and state saving. Automatically called when exceptions occur. +- `_serialize_callable()`: Helper method for serializing callable attributes in `to_dict()` +- `_serialize_attr()`: Helper method for serializing individual attributes, handling non-serializable objects + ## Limitations ----------- @@ -405,6 +519,8 @@ It's important to note that the `AgentRearrange` class and the `rearrange` funct The sequential awareness feature works best with agents that can understand and utilize context about their position in the workflow. +Flow patterns must include at least one '->' to denote direction. Agents referenced in the flow must be registered in the agents dictionary. + ## Conclusion ----------