swarms/docs/swarms/agents/consistency_agent.md

Consistency Agent Documentation

The SelfConsistencyAgent is a specialized agent designed for generating multiple independent responses to a given task and aggregating them into a single, consistent final answer. It leverages concurrent processing to enhance efficiency and employs a majority voting mechanism to ensure the reliability of the aggregated response.

Purpose

The primary objective of the SelfConsistencyAgent is to provide a robust mechanism for decision-making and problem-solving by generating diverse responses and synthesizing them into a coherent final answer. This approach is particularly useful in scenarios where consistency and reliability are critical.

Class: SelfConsistencyAgent

Initialization

• __init__: Initializes the SelfConsistencyAgent with specified parameters.

Arguments

Argument	Type	Default	Description
name	str	"Self-Consistency-Agent"	Name of the agent.
description	str	"An agent that uses self consistency to generate a final answer."	Description of the agent's purpose.
system_prompt	str	CONSISTENCY_SYSTEM_PROMPT	System prompt for the reasoning agent.
model_name	str	Required	The underlying language model to use.
num_samples	int	5	Number of independent responses to generate.
max_loops	int	1	Maximum number of reasoning loops per sample.
majority_voting_prompt	Optional[str]	majority_voting_prompt	Custom prompt for majority voting aggregation.
eval	bool	False	Enable evaluation mode for answer validation.
output_type	OutputType	"dict"	Format of the output.
random_models_on	bool	False	Enable random model selection for diversity.

Methods

• run: Generates multiple responses for the given task and aggregates them.

  • Arguments:
    • task (str): The input prompt.
    • img (Optional[str], optional): Image input for vision tasks.
    • answer (Optional[str], optional): Expected answer for validation (if eval=True).
  • Returns: Union[str, Dict[str, Any]] - The aggregated final answer.

• aggregation_agent: Aggregates a list of responses into a single final answer using majority voting.

  • Arguments:
    • responses (List[str]): The list of responses.
    • prompt (str, optional): Custom prompt for the aggregation agent.
    • model_name (str, optional): Model to use for aggregation.
  • Returns: str - The aggregated answer.

• check_responses_for_answer: Checks if a specified answer is present in any of the provided responses.

  • Arguments:
    • responses (List[str]): A list of responses to check.
    • answer (str): The answer to look for in the responses.
  • Returns: bool - True if the answer is found, False otherwise.

• batched_run: Run the agent on multiple tasks in batch.

  • Arguments:
    • tasks (List[str]): List of tasks to be processed.
  • Returns: List[Union[str, Dict[str, Any]]] - List of results for each task.

Examples

Example 1: Basic Usage

from swarms.agents.consistency_agent import SelfConsistencyAgent

# Initialize the agent
agent = SelfConsistencyAgent(
    name="Math-Reasoning-Agent",
    model_name="gpt-4o-mini",
    max_loops=1,
    num_samples=5
)

# Define a task
task = "What is the 40th prime number?"

# Run the agent
final_answer = agent.run(task)

# Print the final aggregated answer
print("Final aggregated answer:", final_answer)

Example 2: Using Custom Majority Voting Prompt

from swarms.agents.consistency_agent import SelfConsistencyAgent

# Initialize the agent with a custom majority voting prompt
agent = SelfConsistencyAgent(
    name="Reasoning-Agent",
    model_name="gpt-4o-mini",
    max_loops=1,
    num_samples=5,
    majority_voting_prompt="Please provide the most common response."
)

# Define a task
task = "Explain the theory of relativity in simple terms."

# Run the agent
final_answer = agent.run(task)

# Print the final aggregated answer
print("Final aggregated answer:", final_answer)

Example 3: Evaluation Mode

from swarms.agents.consistency_agent import SelfConsistencyAgent

# Initialize the agent with evaluation mode
agent = SelfConsistencyAgent(
    name="Validation-Agent",
    model_name="gpt-4o-mini",
    num_samples=3,
    eval=True
)

# Run with expected answer for validation
result = agent.run("What is 2 + 2?", answer="4", eval=True)
if result is not None:
    print("Validation passed:", result)
else:
    print("Validation failed - expected answer not found")

Example 4: Random Models for Diversity

from swarms.agents.consistency_agent import SelfConsistencyAgent

# Initialize the agent with random model selection
agent = SelfConsistencyAgent(
    name="Diverse-Reasoning-Agent",
    model_name="gpt-4o-mini",
    num_samples=5,
    random_models_on=True
)

# Run the agent
result = agent.run("What are the benefits of renewable energy?")
print("Diverse reasoning result:", result)

Example 5: Batch Processing

from swarms.agents.consistency_agent import SelfConsistencyAgent

# Initialize the agent
agent = SelfConsistencyAgent(
    name="Batch-Processing-Agent",
    model_name="gpt-4o-mini",
    num_samples=3
)

# Define multiple tasks
tasks = [
    "What is the capital of France?",
    "What is 15 * 23?",
    "Explain photosynthesis in simple terms."
]

# Process all tasks
results = agent.batched_run(tasks)

# Print results
for i, result in enumerate(results):
    print(f"Task {i+1} result: {result}")

Key Features

Self-Consistency Technique

The agent implements the self-consistency approach based on the research paper "Self-Consistency Improves Chain of Thought Reasoning in Language Models" by Wang et al. (2022). This technique:

1. Generates Multiple Independent Responses: Creates several reasoning paths for the same problem
2. Analyzes Consistency: Examines agreement among different reasoning approaches
3. Aggregates Results: Uses majority voting or consensus building
4. Produces Reliable Output: Delivers a final answer reflecting the most reliable consensus

Benefits

• Mitigates Random Errors: Multiple reasoning paths reduce individual path errors
• Reduces Bias: Diverse approaches minimize single-method biases
• Improves Reliability: Consensus-based results are more trustworthy
• Handles Complexity: Better performance on complex problem-solving tasks

Use Cases

• Mathematical Problem Solving: Where accuracy is critical
• Decision Making: When reliability is paramount
• Validation Tasks: When answers need verification
• Complex Reasoning: Multi-step problem solving
• Research Questions: Where multiple perspectives are valuable

Technical Details

Concurrent Execution

The agent uses ThreadPoolExecutor to generate multiple responses concurrently, improving performance while maintaining independence between reasoning paths.

Aggregation Process

The aggregation uses an AI-powered agent that:

• Identifies dominant responses
• Analyzes disparities and disagreements
• Evaluates consensus strength
• Synthesizes minority insights
• Provides comprehensive recommendations

Output Formats

The agent supports various output types:

• "dict": Dictionary format with conversation history
• "str": Simple string output
• "list": List format
• "json": JSON formatted output

Limitations

1. Computational Cost: Higher num_samples increases processing time and cost
2. Model Dependencies: Performance depends on the underlying model capabilities
3. Consensus Challenges: May struggle with tasks where multiple valid approaches exist
4. Memory Usage: Concurrent execution requires more memory resources

Best Practices

1. Sample Size: Use 3-7 samples for most tasks; increase for critical decisions
2. Model Selection: Choose models with strong reasoning capabilities
3. Evaluation Mode: Enable for tasks with known correct answers
4. Custom Prompts: Tailor majority voting prompts for specific domains
5. Batch Processing: Use batched_run for multiple related tasks

---