swarms/docs/swarms/models/mistral.md

Mistral Documentation

Table of Contents

1. Introduction
2. Overview
3. Class Definition
   • Mistral Class
   • Initialization Parameters
4. Functionality and Usage
   • Loading the Model
   • Running the Model
   • Chatting with the Agent
5. Additional Information
6. Examples
   • Example 1: Initializing Mistral
   • Example 2: Running a Task
   • Example 3: Chatting with the Agent
7. References and Resources

---

1. Introduction

Welcome to the documentation for Mistral, a powerful language model-based AI agent. Mistral leverages the capabilities of large language models to generate text-based responses to queries and tasks. This documentation provides a comprehensive guide to understanding and using the Mistral AI agent.

1.1 Purpose

Mistral is designed to assist users by generating coherent and contextually relevant text based on user inputs or tasks. It can be used for various natural language understanding and generation tasks, such as chatbots, text completion, question answering, and content generation.

1.2 Key Features

• Utilizes large pre-trained language models.
• Supports GPU acceleration for faster processing.
• Provides an easy-to-use interface for running tasks and engaging in chat-based conversations.
• Offers fine-grained control over response generation through temperature and maximum length settings.

---

2. Overview

Before diving into the details of the Mistral AI agent, let's provide an overview of its purpose and functionality.

Mistral is built on top of powerful language models, such as GPT-3. It allows you to:

• Generate text-based responses to tasks and queries.
• Control the temperature of response generation for creativity.
• Set a maximum length for generated responses.
• Engage in chat-based conversations with the AI agent.
• Utilize GPU acceleration for faster inference.

In the following sections, we will explore the class definition, its initialization parameters, and how to use Mistral effectively.

---

3. Class Definition

Mistral consists of a single class, the Mistral class. This class provides methods for initializing the agent, loading the pre-trained model, and running tasks.

3.1 Mistral Class

class Mistral:
    """
    Mistral

    model = Mistral(device="cuda", use_flash_attention=True, temperature=0.7, max_length=200)
    task = "My favorite condiment is"
    result = model.run(task)
    print(result)
    """

    def __init__(
        self,
        ai_name: str = "Node Model Agent",
        system_prompt: str = None,
        model_name: str = "mistralai/Mistral-7B-v0.1",
        device: str = "cuda",
        use_flash_attention: bool = False,
        temperature: float = 1.0,
        max_length: int = 100,
        do_sample: bool = True,
    ):
        """
        Initializes the Mistral AI agent.

        Parameters:
        - ai_name (str): The name or identifier of the AI agent. Default: "Node Model Agent".
        - system_prompt (str): A system-level prompt for context (e.g., conversation history). Default: None.
        - model_name (str): The name of the pre-trained language model to use. Default: "mistralai/Mistral-7B-v0.1".
        - device (str): The device for model inference, such as "cuda" or "cpu". Default: "cuda".
        - use_flash_attention (bool): If True, enables flash attention for faster inference. Default: False.
        - temperature (float): A value controlling the creativity of generated text. Default: 1.0.
        - max_length (int): The maximum length of generated text. Default: 100.
        - do_sample (bool): If True, uses sampling for text generation. Default: True.
        """

3.2 Initialization Parameters

• ai_name (str): The name or identifier of the AI agent. This name can be used to distinguish between different agents if multiple instances are used. The default value is "Node Model Agent".

• system_prompt (str): A system-level prompt that provides context for the AI agent. This can be useful for maintaining a conversation history or providing context for the current task. By default, it is set to None.

• model_name (str): The name of the pre-trained language model to use. The default value is "mistralai/Mistral-7B-v0.1", which points to a specific version of the Mistral model.

• device (str): The device on which the model should perform inference. You can specify "cuda" for GPU acceleration or "cpu" for CPU-only inference. The default is "cuda", assuming GPU availability.

• use_flash_attention (bool): If set to True, Mistral uses flash attention for faster inference. This is beneficial when low-latency responses are required. The default is False.

• temperature (float): The temperature parameter controls the creativity of the generated text. Higher values (e.g., 1.0) produce more random output, while lower values (e.g., 0.7) make the output more focused and deterministic. The default value is 1.0.

• max_length (int): This parameter sets the maximum length of the generated text. It helps control the length of responses. The default value is 100.

• do_sample (bool): If set to True, Mistral uses sampling during text generation. Sampling introduces randomness into the generated text. The default is True.

---

4. Functionality and Usage

Now that we've introduced the Mistral class and its parameters, let's explore how to use Mistral for various tasks.

4.1 Loading the Model

The Mistral class handles the loading of the pre-trained language model during initialization. You do not need to explicitly load the model. Simply create an instance of Mistral, and it will take care of loading the model into memory.

4.2 Running the Model

Mistral provides two methods for running the model:

4.2.1 run Method

The run method is used to generate text-based responses to a given task or input. It takes a single string parameter, task, and returns the generated text as a string.

def run(self, task: str) -> str:
    """
    Run the model on a given task.

    Parameters:
    - task (str): The task or query for which to generate a response.

    Returns:
    - str: The generated text response.
    """

Example:

from swarms.models import Mistral

model = Mistral()
task = "Translate the following English text to French: 'Hello, how are you?'"
result = model.run(task)
print(result)

4.2.2 __call__ Method

The __call__ method provides a more concise way to run the model on a given task. You can use it by simply calling the Mistral instance with a task string.

Example:

model = Mistral()
task = "Generate a summary of the latest research paper on AI ethics."
result = model(task)
print(result)

4.3 Chatting with the Agent

Mistral supports chat-based interactions with the AI agent. You can send a series of messages to the agent, and it will respond accordingly. The chat method handles these interactions.

chat Method

The chat method allows you to engage in chat-based conversations with the AI agent. You can send messages to the agent, and it will respond with text-based messages.

def chat(self, msg: str = None, streaming: bool = False) -> str:
    """
    Run a chat conversation with the agent.

    Parameters:
    - msg (str, optional): The message to send to the agent. Defaults to None.
    - streaming (bool, optional): Whether to stream the response token by token. Defaults to False.

    Returns:
    - str: The response from the agent.
    """

Example:

model = Mistral()
conversation = [
    "Tell me a joke.",
    "What's the weather like today?",
    "Translate 'apple' to Spanish.",
]
for user_message in conversation:
    response = model.chat(user_message)
    print(f"User: {user_message}")
    print(f"Agent: {response}")

---

5. Additional Information

Here are some additional tips and information for using Mistral effectively:

• Mistral uses a specific pre-trained model ("mistralai/Mistral-7B-v0.1" by default). You can explore other available models and choose one that best suits your task.

• The temperature parameter controls the randomness of generated text. Experiment with different values to achieve the desired level of creativity in responses.

• Be cautious with max_length, especially if you set it to a very high value, as it may lead to excessively long responses.

• Ensure that you have the required libraries, such as torch and transformers, installed to use Mistral successfully.

• Consider providing a system-level prompt when engaging in chat-based conversations to provide context for the agent.

---

6. Examples

In this section, we provide practical examples to illustrate how to use Mistral for various tasks.

6.1 Example 1: Initializing Mistral

In this example, we initialize the Mistral AI agent with custom settings:

from swarms.models import Mistral

model = Mistral(
    ai_name="My AI Assistant",
    device="cpu",
    temperature=0.8,
    max_length=150,
)

6.2 Example 2: Running a Task

Here, we run a text generation task using Mistral:

model = Mistral()
task = "Summarize the main findings of the recent climate change report."
result = model.run(task)
print(result)

6.3 Example 3: Chatting with the Agent

Engage in a chat-based conversation with Mistral:

model = Mistral()
conversation = [
    "Tell me a joke.",
    "What's the latest news?",
    "Translate 'cat' to French.",
]
for user_message in conversation:
    response = model.chat(user_message)
    print(f"User: {user_message}")
    print(f"Agent: {response}")

---

7. References and Resources

Here are some references and resources for further information on Mistral and related topics:

• Mistral GitHub Repository: Official Mistral repository for updates and contributions.
• Hugging Face Transformers: Documentation and models for various transformers, including Mistral's parent models.
• PyTorch Official Website: Official website for PyTorch, the deep learning framework used in Mistral.

This concludes the documentation for the Mistral AI agent. You now have a comprehensive understanding of how to use Mistral for text generation and chat-based interactions. If you have any further questions or need assistance, please refer to the provided references and resources. Happy AI modeling!