[IMPROVEMENT][AgentLoader with markdown, csv, and other capaibilties] [fix examples] [swarms.utils.agent_loader -> MarkdownAgentLoader] [util][swarm_id] [Improve SequentialWorkflowPrompting]
The `AgentRearrange` class represents a swarm of agents for rearranging tasks. It allows you to create a swarm of agents, add or remove agents from the swarm, and run the swarm to process tasks based on a specified flow pattern.
The `AgentRearrange` class represents a swarm of agents for rearranging tasks. It allows you to create a swarm of agents, add or remove agents from the swarm, and run the swarm to process tasks based on a specified flow pattern. The class now includes **sequential awareness** features that allow agents to know about the agents ahead and behind them in sequential flows.
## Attributes
----------
@ -17,26 +17,38 @@ The `AgentRearrange` class represents a swarm of agents for rearranging tasks. I
| `memory_system` | `BaseVectorDatabase` | Memory system for storing agent interactions |
| `human_in_the_loop` | `bool` | Whether human intervention is enabled |
| `custom_human_in_the_loop` | `Callable` | Custom function for human intervention |
| `return_json` | `bool` | Whether to return output in JSON format |
| `output_type` | `OutputType` | Format of output ("all", "final", "list", or "dict") |
| `docs` | `List[str]` | List of document paths to add to agent prompts |
| `doc_folder` | `str` | Folder path containing documents to add to agent prompts |
| `swarm_history` | `dict` | History of agent interactions |
| `autosave` | `bool` | Whether to automatically save agent data |
| `rules` | `str` | Custom rules to add to the conversation |
| `team_awareness` | `bool` | Whether to enable team awareness and sequential flow information |
| `time_enabled` | `bool` | Whether to enable timestamps in conversation |
| `message_id_on` | `bool` | Whether to enable message IDs in conversation |
Process multiple tasks concurrently using ThreadPoolExecutor.
| Parameter | Type | Description |
| --- | --- | --- |
| `tasks` | `List[str]` | List of tasks to process |
| `img` | `List[str]` | Optional list of images corresponding to tasks |
| `max_workers` | `int` | Maximum number of worker threads |
| `device` | `str` | Computing device to use |
| `device_id` | `int` | Specific device ID if applicable |
| `all_cores` | `bool` | Whether to use all CPU cores |
| `all_gpus` | `bool` | Whether to use all available GPUs |
| `img` | `List[str]` (optional) | Optional list of images corresponding to tasks |
| `max_workers` | `int` (optional) | Maximum number of worker threads |
| `*args` | - | Additional positional arguments |
| `**kwargs` | - | Additional keyword arguments |
**Returns:**
- `List[str]`: List of results corresponding to input tasks
## **Sequential Awareness Feature**
The `AgentRearrange` class now includes a powerful **sequential awareness** feature that enhances agent collaboration in sequential workflows. When agents are executed sequentially, they automatically receive information about:
- **Agent ahead**: The agent that completed their task before them
- **Agent behind**: The agent that will receive their output next
This feature is automatically enabled when using sequential flows and provides agents with context about their position in the workflow, improving coordination and task understanding.
### How It Works
1. **Automatic Detection**: The system automatically detects when agents are running sequentially vs. in parallel
2. **Context Injection**: Before each sequential agent runs, awareness information is added to the conversation
3. **Enhanced Collaboration**: Agents can reference previous agents' work and prepare output for the next agent
### Example with Sequential Awareness
```python
from swarms import Agent, AgentRearrange
# Create agents
agent1 = Agent(agent_name="Researcher", system_prompt="Research the topic")
agent2 = Agent(agent_name="Writer", system_prompt="Write based on research")
agent3 = Agent(agent_name="Editor", system_prompt="Edit the written content")
In this example, we first initialize three agents: `director`, `worker1`, and `worker2`. Then, we create a list of these agents and define the flow pattern `"Director -> Worker1 -> Worker2"`.
We can use the `AgentRearrange` class by creating an instance of it with the list of agents and the flow pattern. We then call the `run` method with the initial task, and it will execute the agents in the specified order, passing the output of one agent as the input to the next agent.
Alternatively, we can use the `rearrange` function by passing the list of agents, the flow pattern, and the initial task as arguments.
Both the `AgentRearrange` class and the `rearrange` function will return the final output after processing the task through the agents according to the specified flow pattern.
The new sequential awareness features provide:
- **Automatic context**: Each agent knows who came before and who comes after
- **Better coordination**: Agents can reference previous work and prepare for next steps
- **Flow visualization**: You can see the complete workflow structure
- **Enhanced logging**: Better tracking of agent interactions
This will raise a `ValueError` with the message `"Agent 'Worker3' is not registered."`.
## Parallel and Sequential Processing
----------------------------------
The `AgentRearrange` class supports both parallel and sequential processing of tasks based on the specified flow pattern. If the flow pattern includes multiple agents separated by commas (e.g., `"agent1, agent2"`), the agents will be executed in parallel, and their outputs will be concatenated with a semicolon (`;`). If the flow pattern includes a single agent, it will be executed sequentially.
The `AgentRearrange` class supports both parallel and sequential processing of tasks based on the specified flow pattern. If the flow pattern includes multiple agents separated by commas (e.g., `"agent1, agent2"`), the agents will be executed in parallel, and their outputs will be concatenated. If the flow pattern includes a single agent, it will be executed sequentially with enhanced awareness.
In the `parallel_flow` example, `Worker1` and `Worker2` will be executed in parallel, and their outputs will be concatenated and passed to `Director`. In the `sequential_flow` example, `Worker1` will be executed first, and its output will be passed to `Worker2`, and then the output of `Worker2` will be passed to `Director`.
In the `parallel_flow` example, `Worker1` and `Worker2` will be executed in parallel, and their outputs will be concatenated and passed to `Director`.
## Logging
-------
In the `sequential_flow` example, `Worker1` will be executed first, then `Worker2` will receive awareness that `Worker1` came before and `Director` comes after, and finally `Director` will receive awareness that `Worker2` came before.
The `AgentRearrange` class includes logging capabilities using the `loguru` library. If `verbose` is set to `True` during initialization, a log file named `agent_rearrange.log` will be created, and log messages will be written to it. You can use this log file to track the execution of the agents and any potential issues or errors that may occur.
## Logging and Monitoring
-------
The `AgentRearrange` class includes comprehensive logging capabilities using the `loguru` library. The new sequential awareness features add enhanced logging:
```bash
2023-05-08 10:30:15.456 | INFO | agent_rearrange:__init__:34 - Adding agent Director to the swarm.
2023-05-08 10:30:15.457 | INFO | agent_rearrange:__init__:34 - Adding agent Worker1 to the swarm.
2023-05-08 10:30:15.457 | INFO | agent_rearrange:__init__:34 - Adding agent Worker2 to the swarm.
2023-05-08 10:30:15.458 | INFO | agent_rearrange:run:118 - Running agents in parallel: ['Worker1', 'Worker2']
2023-05-08 10:30:15.460 | INFO | agent_rearrange:run:125 - Added sequential awareness for Worker2: Sequential awareness: Agent ahead: Worker1 | Agent behind: Director
```
## Additional Parameters
---------------------
The `AgentRearrange` class also accepts additional parameters that can be passed to the `run` method using `*args` and `**kwargs`. These parameters will be forwarded to the individual agents during execution.
The `AgentRearrange` class and the `rearrange` function can be customized and extended to suit specific use cases. For example, you can create custom agents by inheriting from the `Agent` class and implementing custom logic for task processing. You can then add these custom agents to the swarm and define the flow pattern accordingly.
Additionally, you can modify the `run` method of the `AgentRearrange` class to implement custom logic for task processing and agent interaction.
The `AgentRearrange` class and the `rearrange` function can be customized and extended to suit specific use cases. The new sequential awareness features provide a foundation for building more sophisticated agent coordination systems.
## Limitations
-----------
It's important to note that the `AgentRearrange` class and the `rearrange` function rely on the individual agents to process tasks correctly. The quality of the output will depend on the capabilities and configurations of the agents used in the swarm. Additionally, the `AgentRearrange` class does not provide any mechanisms for task prioritization or load balancing among the agents.
It's important to note that the `AgentRearrange` class and the `rearrange` function rely on the individual agents to process tasks correctly. The quality of the output will depend on the capabilities and configurations of the agents used in the swarm.
The sequential awareness feature works best with agents that can understand and utilize context about their position in the workflow.
## Conclusion
----------
The `AgentRearrange` class and the `rearrange` function provide a flexible and extensible framework for orchestrating swarms of agents to process tasks based on a specified flow pattern. By combining the capabilities of individual agents, you can create complex workflows and leverage the strengths of different agents to tackle various tasks efficiently.
The `AgentRearrange` class and the `rearrange` function provide a flexible and extensible framework for orchestrating swarms of agents to process tasks based on a specified flow pattern. The new **sequential awareness** features significantly enhance agent collaboration by providing context about workflow relationships.
By combining the capabilities of individual agents with enhanced awareness of their position in the workflow, you can create more intelligent and coordinated multi-agent systems that understand not just their individual tasks, but also their role in the larger workflow.
While the current implementation offers basic functionality for agent rearrangement, there is room for future improvements and customizations to enhance the system's capabilities and cater to more specific use cases.
Whether you're working on natural language processing tasks, data analysis, or any other domain where agent-based systems can be beneficial, the enhanced `AgentRearrange` class provides a solid foundation for building sophisticated swarm-based solutions with improved coordination and context awareness.
Whether you're working on natural language processing tasks, data analysis, or any other domain where agent-based systems can be beneficial, the `AgentRearrange` class and the `rearrange` function provide a solid foundation for building and experimenting with swarm-based solutions.
A Sequential Swarm architecture processes tasks in a linear sequence. Each agent completes its task before passing the result to the next agent in the chain. This architecture ensures orderly processing and is useful when tasks have dependencies. [Learn more here in the docs:](https://docs.swarms.world/en/latest/swarms/structs/agent_rearrange/)
A Sequential Swarm architecture processes tasks in a linear sequence. Each agent completes its task before passing the result to the next agent in the chain. This architecture ensures orderly processing and is useful when tasks have dependencies. The system now includes **sequential awareness** features that allow agents to know about the agents ahead and behind them in the workflow, significantly enhancing coordination and context understanding. [Learn more here in the docs:](https://docs.swarms.world/en/latest/swarms/structs/agent_rearrange/)
**Use-Cases:**
- Workflows where each step depends on the previous one, such as assembly lines or sequential data processing.
- Scenarios requiring strict order of operations.
- **NEW**: Enhanced workflows where agents need context about their position in the sequence for better coordination.
```mermaid
graph TD
A[First Agent] --> B[Second Agent]
B --> C[Third Agent]
C --> D[Fourth Agent]
style A fill:#e1f5fe
style B fill:#f3e5f5
style C fill:#e8f5e8
style D fill:#fff3e0
A -.->|"Awareness: None (first)"| A
B -.->|"Awareness: Ahead: A, Behind: C"| B
C -.->|"Awareness: Ahead: B, Behind: D"| C
D -.->|"Awareness: Ahead: C, Behind: None (last)"| D
```
## **Sequential Awareness Feature**
The SequentialWorkflow now includes a powerful **sequential awareness** feature that automatically provides each agent with context about their position in the workflow:
### What Agents Know Automatically
- **Agent ahead**: The agent that completed their task before them
- **Agent behind**: The agent that will receive their output next
- **Workflow position**: Their step number and role in the sequence
### Benefits
1. **Better Coordination**: Agents can reference previous work and prepare output for the next step
2. **Context Understanding**: Each agent knows their role in the larger workflow
3. **Improved Quality**: Output is tailored for the next agent in the sequence
4. **Enhanced Logging**: Better tracking of agent interactions and workflow progress
The `run` method includes logging to track the execution flow and captures errors to provide detailed information in case of failures. This is crucial for debugging and ensuring smooth operation of the workflow.
The `run` method now includes enhanced logging to track the sequential awareness flow and captures detailed information about agent interactions:
- Ensure that the agents provided to the `SequentialWorkflow` are properly initialized and configured to handle the tasks they will receive.
- **Enable Team Awareness**: Set `team_awareness=True` to unlock the full potential of sequential coordination.
- **Use Descriptive Agent Names**: Clear agent names make the awareness information more useful.
- **Monitor Logs**: Enhanced logging provides insights into how agents are coordinating.
- **Iterative Improvement**: Use the awareness features to refine agent prompts and improve workflow quality.
## **Benefits of Sequential Awareness**
- The `max_loops` parameter can be used to control how many times the workflow should be executed, which is useful for iterative processes.
1. **Improved Quality**: Agents produce better output when they understand their context
2. **Better Coordination**: Reduced redundancy and improved handoffs between agents
3. **Enhanced Debugging**: Clear visibility into agent interactions and workflow progress
4. **Scalable Workflows**: Easy to add new agents while maintaining coordination
5. **Professional Workflows**: Mimics real-world team collaboration patterns
- Utilize the logging information to monitor and debug the task execution process.
The SequentialWorkflow with sequential awareness represents a significant advancement in multi-agent coordination, enabling more sophisticated and professional workflows that closely mirror human team collaboration patterns.
The `AgentLoader` is a powerful utility for creating Swarms agents from markdown files using the Claude Code sub-agent format. It supports both single and multiple markdown file loading, providing a flexible way to define and deploy agents using YAML frontmatter configuration.
The `AgentLoader` is a comprehensive utility for creating Swarms agents from various file formats including Markdown, YAML, and CSV files. It provides a unified interface for loading agents with support for concurrent processing, configuration overrides, and automatic file type detection.
## Overview
The AgentLoader enables you to:
- Load single agents from markdown files with YAML frontmatter
- Load multiple agents from directories or file lists with concurrent processing
- Parse Claude Code sub-agent YAML frontmatter configurations
- Extract system prompts from markdown content
- Utilize 100% CPU cores for high-performance batch loading
- Provide comprehensive error handling and validation
- Load agents from Markdown files
- Load agents from YAML configuration files
- Load agents from CSV files
- Automatically detect file types and use appropriate loaders
- Process multiple files concurrently for improved performance
- Override default configurations with custom parameters
- Handle various agent configurations and settings
## Installation
The AgentLoader is included with the Swarms framework:
```python
from swarms import AgentLoader, load_agent_from_markdown, load_agents_from_markdown
from swarms.structs import AgentLoader
from swarms.utils import load_agent_from_markdown, load_agents_from_markdown
```
## Markdown Format
## Supported File Formats
The AgentLoader uses the Claude Code sub-agent YAML frontmatter format:
### 1. Markdown Files (Claude Code Format)
The primary format uses YAML frontmatter with markdown content:
```markdown
---
name: your-sub-agent-name
description: Description of when this subagent should be invoked
model_name: gpt-4
temperature: 0.3
max_loops: 2
name: FinanceAdvisor
description: Expert financial advisor for investment and budgeting guidance
model_name: claude-sonnet-4-20250514
temperature: 0.7
max_loops: 1
mcp_url: http://example.com/mcp # optional
---
Your subagent's system prompt goes here. This can be multiple paragraphs
and should clearly define the subagent's role, capabilities, and approach
to solving problems.
You are an expert financial advisor with deep knowledge in:
- Investment strategies and portfolio management
- Personal budgeting and financial planning
- Risk assessment and diversification
- Tax optimization strategies
- Retirement planning
Include specific instructions, best practices, and any constraints
the subagent should follow.
Your approach:
- Provide clear, actionable financial advice
- Consider individual risk tolerance and goals
- Explain complex concepts in simple terms
- Always emphasize the importance of diversification
- Include relevant disclaimers about financial advice
When analyzing financial situations:
1. Assess current financial position
2. Identify short-term and long-term goals
3. Evaluate risk tolerance
4. Recommend appropriate strategies
5. Suggest specific action steps
```
**Schema Fields:**
- `name` (required): Your sub-agent name
- `description` (required): Description of when this subagent should be invoked
- `model_name` (optional): Name of model (defaults to random selection if not provided)
- `temperature` (optional): Float value for model temperature (0.0-2.0)
- `max_loops` (optional): Integer for maximum reasoning loops
- `mcp_url` (optional): MCP server URL if needed
| Field | Type | Required | Default | Description |
The AgentLoader utilizes 100% of CPU cores for concurrent agent loading, providing significant performance improvements when processing multiple markdown files:
```python
from swarms.utils import load_agents_from_markdown
# Automatic concurrent processing for multiple files
"""Please navigate to chat.com and engage in a detailed technical discussion with ChatGPT about the following specific aspects of future high-energy physics:
-Duringexecution:Shareyourreasoningandintermediatesteps.Whenbuildingonordifferingfromanotheragent's output, mention them by name and explain your reasoning.
Kye Gomez
1 week ago