ai-mirow-projects
/
swarms
mirror of https://github.com/kyegomez/swarms
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[CLEANUP][DOCS][swams.models]

Browse Source
pull/439/head
Kye 9 months ago
parent 022b39e8f6
commit 0d694ec2cd
11 changed files with 0 additions and 838 deletions
Show Stats Download Patch File Download Diff File

2
docs/mkdocs.yml
Unescape View File

@ -130,8 +130,6 @@ nav:
- HuggingFaceLLM: "swarms/models/huggingface.md"
- Anthropic: "swarms/models/anthropic.md"
- OpenAI: "swarms/models/openai.md"
- vLLM: "swarms/models/vllm.md"
- MPT7B: "swarms/models/mpt.md"
- Mistral: "swarms/models/mistral.md"
- Mixtral: "swarms/models/mixtral.md"
- MultiModal:

148
docs/swarms/chunkers/basechunker.md
Unescape View File

@ -1,148 +0,0 @@
# BaseChunker Documentation
## Table of Contents
1. [Introduction](#introduction)
2. [Overview](#overview)
3. [Installation](#installation)
4. [Usage](#usage)
1. [BaseChunker Class](#basechunker-class)
2. [Examples](#examples)
5. [Additional Information](#additional-information)
6. [Conclusion](#conclusion)
---
## 1. Introduction <a name="introduction"></a>
The `BaseChunker` module is a tool for splitting text into smaller chunks that can be processed by a language model. It is a fundamental component in natural language processing tasks that require handling long or complex text inputs.
This documentation provides an extensive guide on using the `BaseChunker` module, explaining its purpose, parameters, and usage.
---
## 2. Overview <a name="overview"></a>
The `BaseChunker` module is designed to address the challenge of processing lengthy text inputs that exceed the maximum token limit of language models. By breaking such text into smaller, manageable chunks, it enables efficient and accurate processing.
Key features and parameters of the `BaseChunker` module include:
- `separators`: Specifies a list of `ChunkSeparator` objects used to split the text into chunks.
- `tokenizer`: Defines the tokenizer to be used for counting tokens in the text.
- `max_tokens`: Sets the maximum token limit for each chunk.
The `BaseChunker` module facilitates the chunking process and ensures that the generated chunks are within the token limit.
---
## 3. Installation <a name="installation"></a>
Before using the `BaseChunker` module, ensure you have the required dependencies installed. The module relies on `griptape` and `swarms` libraries. You can install these dependencies using pip:
```bash
pip install griptape swarms
```
---
## 4. Usage <a name="usage"></a>
In this section, we'll cover how to use the `BaseChunker` module effectively. It consists of the `BaseChunker` class and provides examples to demonstrate its usage.
### 4.1. `BaseChunker` Class <a name="basechunker-class"></a>
The `BaseChunker` class is the core component of the `BaseChunker` module. It is used to create a `BaseChunker` instance, which can split text into chunks efficiently.
#### Parameters:
- `separators` (list[ChunkSeparator]): Specifies a list of `ChunkSeparator` objects used to split the text into chunks.
- `tokenizer` (OpenAITokenizer): Defines the tokenizer to be used for counting tokens in the text.
- `max_tokens` (int): Sets the maximum token limit for each chunk.
### 4.2. Examples <a name="examples"></a>
Let's explore how to use the `BaseChunker` class with different scenarios and applications.
#### Example 1: Basic Chunking
```python
from basechunker import BaseChunker, ChunkSeparator
# Initialize the BaseChunker
chunker = BaseChunker()
# Text to be chunked
input_text = (
"This is a long text that needs to be split into smaller chunks for processing."
)
# Chunk the text
chunks = chunker.chunk(input_text)
# Print the generated chunks
for idx, chunk in enumerate(chunks, start=1):
print(f"Chunk {idx}: {chunk.value}")
```
#### Example 2: Custom Separators
```python
from basechunker import BaseChunker, ChunkSeparator
# Define custom separators
custom_separators = [ChunkSeparator(","), ChunkSeparator(";")]
# Initialize the BaseChunker with custom separators
chunker = BaseChunker(separators=custom_separators)
# Text with custom separators
input_text = "This text, separated by commas; should be split accordingly."
# Chunk the text
chunks = chunker.chunk(input_text)
# Print the generated chunks
for idx, chunk in enumerate(chunks, start=1):
print(f"Chunk {idx}: {chunk.value}")
```
#### Example 3: Adjusting Maximum Tokens
```python
from basechunker import BaseChunker
# Initialize the BaseChunker with a custom maximum token limit
chunker = BaseChunker(max_tokens=50)
# Long text input
input_text = "This is an exceptionally long text that should be broken into smaller chunks based on token count."
# Chunk the text
chunks = chunker.chunk(input_text)
# Print the generated chunks
for idx, chunk in enumerate(chunks, start=1):
print(f"Chunk {idx}: {chunk.value}")
```
### 4.3. Additional Features
The `BaseChunker` class also provides additional features:
#### Recursive Chunking
The `_chunk_recursively` method handles the recursive chunking of text, ensuring that each chunk stays within the token limit.
---
## 5. Additional Information <a name="additional-information"></a>
- **Text Chunking**: The `BaseChunker` module is a fundamental tool for text chunking, a crucial step in preprocessing text data for various natural language processing tasks.
- **Custom Separators**: You can customize the separators used to split the text, allowing flexibility in how text is chunked.
- **Token Count**: The module accurately counts tokens using the specified tokenizer, ensuring that chunks do not exceed token limits.
---
## 6. Conclusion <a name="conclusion"></a>
The `BaseChunker` module is an essential tool for text preprocessing and handling long or complex text inputs in natural language processing tasks. This documentation has provided a comprehensive guide on its usage, parameters, and examples, enabling you to efficiently manage and process text data by splitting it into manageable chunks.
By using the `BaseChunker`, you can ensure that your text data remains within token limits and is ready for further analysis and processing.
*Please check the official `BaseChunker` repository and documentation for any updates beyond the knowledge cutoff date.*

147
docs/swarms/chunkers/pdf_chunker.md
Unescape View File

@ -1,147 +0,0 @@
# PdfChunker Documentation
## Table of Contents
1. [Introduction](#introduction)
2. [Overview](#overview)
3. [Installation](#installation)
4. [Usage](#usage)
1. [PdfChunker Class](#pdfchunker-class)
2. [Examples](#examples)
5. [Additional Information](#additional-information)
6. [Conclusion](#conclusion)
---
## 1. Introduction <a name="introduction"></a>
The `PdfChunker` module is a specialized tool designed to split PDF text content into smaller, more manageable chunks. It is a valuable asset for processing PDF documents in natural language processing and text analysis tasks.
This documentation provides a comprehensive guide on how to use the `PdfChunker` module. It covers its purpose, parameters, and usage, ensuring that you can effectively process PDF text content.
---
## 2. Overview <a name="overview"></a>
The `PdfChunker` module serves a critical role in handling PDF text content, which is often lengthy and complex. Key features and parameters of the `PdfChunker` module include:
- `separators`: Specifies a list of `ChunkSeparator` objects used to split the PDF text content into chunks.
- `tokenizer`: Defines the tokenizer used for counting tokens in the text.
- `max_tokens`: Sets the maximum token limit for each chunk.
By using the `PdfChunker`, you can efficiently prepare PDF text content for further analysis and processing.
---
## 3. Installation <a name="installation"></a>
Before using the `PdfChunker` module, ensure you have the required dependencies installed. The module relies on the `swarms` library. You can install this dependency using pip:
```bash
pip install swarms
```
---
## 4. Usage <a name="usage"></a>
In this section, we'll explore how to use the `PdfChunker` module effectively. It consists of the `PdfChunker` class and provides examples to demonstrate its usage.
### 4.1. `PdfChunker` Class <a name="pdfchunker-class"></a>
The `PdfChunker` class is the core component of the `PdfChunker` module. It is used to create a `PdfChunker` instance, which can split PDF text content into manageable chunks.
#### Parameters:
- `separators` (list[ChunkSeparator]): Specifies a list of `ChunkSeparator` objects used to split the PDF text content into chunks.
- `tokenizer` (OpenAITokenizer): Defines the tokenizer used for counting tokens in the text.
- `max_tokens` (int): Sets the maximum token limit for each chunk.
### 4.2. Examples <a name="examples"></a>
Let's explore how to use the `PdfChunker` class with different scenarios and applications.
#### Example 1: Basic Chunking
```python
from swarms.chunkers.chunk_seperator import ChunkSeparator
from swarms.chunkers.pdf_chunker import PdfChunker
# Initialize the PdfChunker
pdf_chunker = PdfChunker()
# PDF text content to be chunked
pdf_text = "This is a PDF document with multiple paragraphs and sentences. It should be split into smaller chunks for analysis."
# Chunk the PDF text content
chunks = pdf_chunker.chunk(pdf_text)
# Print the generated chunks
for idx, chunk in enumerate(chunks, start=1):
print(f"Chunk {idx}:\n{chunk.value}")
```
#### Example 2: Custom Separators
```python
from swarms.chunkers.chunk_seperator import ChunkSeparator
from swarms.chunkers.pdf_chunker import PdfChunker
# Define custom separators for PDF chunking
custom_separators = [ChunkSeparator("\n\n"), ChunkSeparator(". ")]
# Initialize the PdfChunker with custom separators
pdf_chunker = PdfChunker(separators=custom_separators)
# PDF text content with custom separators
pdf_text = "This PDF document has custom paragraph separators.\n\nIt also uses period-based sentence separators. Split accordingly."
# Chunk the PDF text content
chunks = pdf_chunker.chunk(pdf_text)
# Print the generated chunks
for idx, chunk in enumerate(chunks, start=1):
print(f"Chunk {idx}:\n{chunk.value}")
```
#### Example 3: Adjusting Maximum Tokens
```python
from swarms.chunkers.pdf_chunker import PdfChunker
# Initialize the PdfChunker with a custom maximum token limit
pdf_chunker = PdfChunker(max_tokens=50)
# Lengthy PDF text content
pdf_text = "This is an exceptionally long PDF document that should be broken into smaller chunks based on token count."
# Chunk the PDF text content
chunks = pdf_chunker.chunk(pdf_text)
# Print the generated chunks
for idx, chunk in enumerate(chunks, start=1):
print(f"Chunk {idx}:\n{chunk.value}")
```
### 4.3. Additional Features
The `PdfChunker` class also provides additional features:
#### Recursive Chunking
The `_chunk_recursively` method handles the recursive chunking of PDF text content, ensuring that each chunk stays within the token limit.
---
## 5. Additional Information <a name="additional-information"></a>
- **PDF Text Chunking**: The `PdfChunker` module is a specialized tool for splitting PDF text content into manageable chunks, making it suitable for natural language processing tasks involving PDF documents.
- **Custom Separators**: You can customize separators to adapt the PDF text content chunking process to specific document structures.
- **Token Count**: The module accurately counts tokens using the specified tokenizer, ensuring that chunks do not exceed token limits.
---
## 6. Conclusion <a name="conclusion"></a>
The `PdfChunker` module is a valuable asset for processing PDF text content in various natural language processing and text analysis tasks. This documentation has provided a comprehensive guide on its usage, parameters, and examples, ensuring that you can effectively prepare PDF documents for further analysis and processing.
By using the `PdfChunker`, you can efficiently break down lengthy and complex PDF text content into manageable chunks, making it ready for in-depth analysis.
*Please check the official `PdfChunker` repository and documentation for any updates beyond the knowledge cutoff date.*

104
docs/swarms/models/bingchat.md
Unescape View File

@ -1,104 +0,0 @@
# BingChat Documentation
## Introduction
Welcome to the documentation for BingChat, a powerful chatbot and image generation tool based on OpenAI's GPT model. This documentation provides a comprehensive understanding of BingChat, its architecture, usage, and how it can be integrated into your projects.
## Overview
BingChat is designed to provide text responses and generate images based on user prompts. It utilizes the capabilities of the GPT model to generate creative and contextually relevant responses. In addition, it can create images based on text prompts, making it a versatile tool for various applications.
## Class Definition
```python
class BingChat:
def __init__(self, cookies_path: str):
```
## Usage
To use BingChat, follow these steps:
1. Initialize the BingChat instance:
```python
from swarms.models.bing_chat import BingChat
edgegpt = BingChat(cookies_path="./path/to/cookies.json")
```
2. Get a text response:
```python
response = edgegpt("Hello, my name is ChatGPT")
print(response)
```
### Example 1 - Text Response
```python
from swarms.models.bing_chat import BingChat
edgegpt = BingChat(cookies_path="./path/to/cookies.json")
response = edgegpt("Hello, my name is ChatGPT")
print(response)
```
3. Generate an image based on a text prompt:
```python
image_path = edgegpt.create_img(
"Sunset over mountains", output_dir="./output", auth_cookie="your_auth_cookie"
)
print(f"Generated image saved at {image_path}")
```
### Example 2 - Image Generation
```python
from swarms.models.bing_chat import BingChat
edgegpt = BingChat(cookies_path="./path/to/cookies.json")
image_path = edgegpt.create_img(
"Sunset over mountains", output_dir="./output", auth_cookie="your_auth_cookie"
)
print(f"Generated image saved at {image_path}")
```
4. Set the directory path for managing cookies:
```python
BingChat.set_cookie_dir_path("./cookie_directory")
```
### Example 3 - Set Cookie Directory Path
```python
BingChat.set_cookie_dir_path("./cookie_directory")
```
## How BingChat Works
BingChat works by utilizing cookies for authentication and interacting with OpenAI's GPT model. Here's how it works:
1. **Initialization**: When you create a BingChat instance, it loads the necessary cookies for authentication with BingChat.
2. **Text Response**: You can use the `__call__` method to get a text response from the GPT model based on the provided prompt. You can specify the conversation style for different response types.
3. **Image Generation**: The `create_img` method allows you to generate images based on text prompts. It requires an authentication cookie and saves the generated images to the specified output directory.
4. **Cookie Directory**: You can set the directory path for managing cookies using the `set_cookie_dir_path` method.
## Parameters
- `cookies_path`: The path to the cookies.json file necessary for authenticating with BingChat.
## Additional Information
- BingChat provides both text-based and image-based responses, making it versatile for various use cases.
- Cookies are used for authentication, so make sure to provide the correct path to the cookies.json file.
- Image generation requires an authentication cookie, and the generated images can be saved to a specified directory.
That concludes the documentation for BingChat. We hope you find this tool valuable for your text generation and image generation tasks. If you have any questions or encounter any issues, please refer to the BingChat documentation for further assistance. Enjoy working with BingChat!

157
docs/swarms/models/biogpt.md
Unescape View File

@ -1,157 +0,0 @@
# `BioGPT` Documentation
## Table of Contents
1. [Introduction](#introduction)
2. [Overview](#overview)
3. [Installation](#installation)
4. [Usage](#usage)
1. [BioGPT Class](#biogpt-class)
2. [Examples](#examples)
5. [Additional Information](#additional-information)
6. [Conclusion](#conclusion)
---
## 1. Introduction <a name="introduction"></a>
The `BioGPT` module is a domain-specific generative language model designed for the biomedical domain. It is built upon the powerful Transformer architecture and pretrained on a large corpus of biomedical literature. This documentation provides an extensive guide on using the `BioGPT` module, explaining its purpose, parameters, and usage.
---
## 2. Overview <a name="overview"></a>
The `BioGPT` module addresses the need for a language model specialized in the biomedical domain. Unlike general-purpose language models, `BioGPT` excels in generating coherent and contextually relevant text specific to biomedical terms and concepts. It has been evaluated on various biomedical natural language processing tasks and has demonstrated superior performance.
Key features and parameters of the `BioGPT` module include:
- `model_name`: Name of the pretrained model.
- `max_length`: Maximum length of generated text.
- `num_return_sequences`: Number of sequences to return.
- `do_sample`: Whether to use sampling in generation.
- `min_length`: Minimum length of generated text.
The `BioGPT` module is equipped with features for generating text, extracting features, and more.
---
## 3. Installation <a name="installation"></a>
Before using the `BioGPT` module, ensure you have the required dependencies installed, including the Transformers library and Torch. You can install these dependencies using pip:
```bash
pip install transformers
pip install torch
```
---
## 4. Usage <a name="usage"></a>
In this section, we'll cover how to use the `BioGPT` module effectively. It consists of the `BioGPT` class and provides examples to demonstrate its usage.
### 4.1. `BioGPT` Class <a name="biogpt-class"></a>
The `BioGPT` class is the core component of the `BioGPT` module. It is used to create a `BioGPT` instance, which can generate text, extract features, and more.
#### Parameters:
- `model_name` (str): Name of the pretrained model.
- `max_length` (int): Maximum length of generated text.
- `num_return_sequences` (int): Number of sequences to return.
- `do_sample` (bool): Whether or not to use sampling in generation.
- `min_length` (int): Minimum length of generated text.
### 4.2. Examples <a name="examples"></a>
Let's explore how to use the `BioGPT` class with different scenarios and applications.
#### Example 1: Generating Biomedical Text
```python
from swarms.models import BioGPT
# Initialize the BioGPT model
biogpt = BioGPT()
# Generate biomedical text
input_text = "The patient has a fever"
generated_text = biogpt(input_text)
print(generated_text)
```
#### Example 2: Extracting Features
```python
from swarms.models import BioGPT
# Initialize the BioGPT model
biogpt = BioGPT()
# Extract features from a biomedical text
input_text = "The patient has a fever"
features = biogpt.get_features(input_text)
print(features)
```
#### Example 3: Using Beam Search Decoding
```python
from swarms.models import BioGPT
# Initialize the BioGPT model
biogpt = BioGPT()
# Generate biomedical text using beam search decoding
input_text = "The patient has a fever"
generated_text = biogpt.beam_search_decoding(input_text)
print(generated_text)
```
### 4.3. Additional Features
The `BioGPT` class also provides additional features:
#### Set a New Pretrained Model
```python
biogpt.set_pretrained_model("new_pretrained_model")
```
#### Get the Model's Configuration
```python
config = biogpt.get_config()
print(config)
```
#### Save and Load the Model
```python
# Save the model and tokenizer to a directory
biogpt.save_model("saved_model")
# Load a model and tokenizer from a directory
biogpt.load_from_path("saved_model")
```
#### Print the Model's Architecture
```python
biogpt.print_model()
```
---
## 5. Additional Information <a name="additional-information"></a>
- **Biomedical Text Generation**: The `BioGPT` module is designed specifically for generating biomedical text, making it a valuable tool for various biomedical natural language processing tasks.
- **Feature Extraction**: It also provides the capability to extract features from biomedical text.
- **Beam Search Decoding**: Beam search decoding is available for generating text with improved quality.
- **Customization**: You can set a new pretrained model and save/load models for customization.
---
## 6. Conclusion <a name="conclusion"></a>
The `BioGPT` module is a powerful and specialized tool for generating and working with biomedical text. This documentation has provided a comprehensive guide on its usage, parameters, and examples, enabling you to effectively leverage it for various biomedical natural language processing tasks.
By using `BioGPT`, you can enhance your biomedical text generation and analysis tasks with contextually relevant and coherent text.
*Please check the official `BioGPT` repository and documentation for any updates beyond the knowledge cutoff date.*

82
docs/swarms/models/elevenlabs.md
Unescape View File

@ -1,82 +0,0 @@
# ElevenLabsText2SpeechTool Documentation
## Table of Contents
1. [Introduction](#introduction)
2. [Class Overview](#class-overview)
- [Attributes](#attributes)
3. [Installation](#installation)
4. [Usage](#usage)
- [Initialization](#initialization)
- [Converting Text to Speech](#converting-text-to-speech)
- [Playing and Streaming Speech](#playing-and-streaming-speech)
5. [Exception Handling](#exception-handling)
6. [Advanced Usage](#advanced-usage)
7. [Contributing](#contributing)
8. [References](#references)
## 1. Introduction <a name="introduction"></a>
The `ElevenLabsText2SpeechTool` is a Python class designed to simplify the process of converting text to speech using the Eleven Labs Text2Speech API. This tool is a wrapper around the API and provides a convenient interface for generating speech from text. It supports multiple languages, making it suitable for a wide range of applications, including voice assistants, audio content generation, and more.
## 2. Class Overview <a name="class-overview"></a>
### Attributes <a name="attributes"></a>
- `model` (Union[ElevenLabsModel, str]): The model to use for text to speech. Defaults to `ElevenLabsModel.MULTI_LINGUAL`.
- `name` (str): The name of the tool. Defaults to `"eleven_labs_text2speech"`.
- `description` (str): A brief description of the tool. Defaults to a detailed explanation of its functionality.
## 3. Installation <a name="installation"></a>
To use the `ElevenLabsText2SpeechTool`, you need to install the required dependencies and have access to the Eleven Labs Text2Speech API. Follow these steps:
1. Install the `elevenlabs` library:
```
pip install elevenlabs
```
2. Install the `swarms` library
`pip install swarms`
3. Set up your API key by following the instructions at [Eleven Labs Documentation](https://docs.elevenlabs.io/welcome/introduction).
## 4. Usage <a name="usage"></a>
### Initialization <a name="initialization"></a>
To get started, create an instance of the `ElevenLabsText2SpeechTool`. You can customize the `model` attribute if needed.
```python
from swarms.models import ElevenLabsText2SpeechTool
stt = ElevenLabsText2SpeechTool(model=ElevenLabsModel.MONO_LINGUAL)
```
### Converting Text to Speech <a name="converting-text-to-speech"></a>
You can use the `run` method to convert text to speech. It returns the path to the generated speech file.
```python
speech_file = stt.run("Hello, this is a test.")
```
### Playing and Streaming Speech <a name="playing-and-streaming-speech"></a>
- Use the `play` method to play the generated speech file.
```python
stt.play(speech_file)
```
- Use the `stream_speech` method to stream the text as speech. It plays the speech in real-time.
```python
stt.stream_speech("Hello world!")
```
## 5. Exception Handling <a name="exception-handling"></a>
The `ElevenLabsText2SpeechTool` handles exceptions gracefully. If an error occurs during the conversion process, it raises a `RuntimeError` with an informative error message.
## 6. Advanced Usage <a name="advanced-usage"></a>
- You can implement custom error handling and logging to further enhance the functionality of this tool.
- For advanced users, extending the class to support additional features or customization is possible.
## 7. Contributing <a name="contributing"></a>
Contributions to this tool are welcome. Feel free to open issues, submit pull requests, or provide feedback to improve its functionality and documentation.
## 8. References <a name="references"></a>
- [Eleven Labs Text2Speech API Documentation](https://docs.elevenlabs.io/welcome/introduction)
This documentation provides a comprehensive guide to using the `ElevenLabsText2SpeechTool`. It covers installation, basic usage, advanced features, and contribution guidelines. Refer to the [References](#references) section for additional resources.

90
docs/swarms/models/zephyr.md
Unescape View File

@ -1,90 +0,0 @@
# `Zephyr` Documentation
## Introduction
Welcome to the documentation for Zephyr, a language model by Hugging Face designed for text generation tasks. Zephyr is capable of generating text in response to prompts and is highly customizable using various parameters. This document will provide you with a detailed understanding of Zephyr, its purpose, and how to effectively use it in your projects.
## Overview
Zephyr is a text generation model that can be used to generate human-like text based on a given prompt. It utilizes the power of transformers and fine-tuning to create coherent and contextually relevant text. Users can control the generated text's characteristics through parameters such as `temperature`, `top_k`, `top_p`, and `max_new_tokens`.
## Class Definition
```python
class Zephyr:
def __init__(
self,
max_new_tokens: int = 300,
temperature: float = 0.5,
top_k: float = 50,
top_p: float = 0.95,
):
"""
Initialize the Zephyr model.
Args:
max_new_tokens (int): The maximum number of tokens in the generated text.
temperature (float): The temperature parameter, controlling the randomness of the output.
top_k (float): The top-k parameter, limiting the vocabulary used in generation.
top_p (float): The top-p parameter, controlling the diversity of the output.
"""
```
## Parameters
- `max_new_tokens` (int): The maximum number of tokens in the generated text.
- `temperature` (float): The temperature parameter, controlling the randomness of the output.
- `top_k` (float): The top-k parameter, limiting the vocabulary used in generation.
- `top_p` (float): The top-p parameter, controlling the diversity of the output.
## Usage
To use the Zephyr model, follow these steps:
1. Initialize the Zephyr model with your desired parameters:
```python
from swarms.models import Zephyr
model = Zephyr(max_new_tokens=300, temperature=0.7, top_k=50, top_p=0.95)
```
2. Generate text by providing a prompt:
```python
output = model("Generate a funny joke about cats")
print(output)
```
### Example 1 - Generating a Joke
```python
model = Zephyr(max_new_tokens=100)
output = model("Tell me a joke about programmers")
print(output)
```
### Example 2 - Writing Poetry
```python
model = Zephyr(temperature=0.2, top_k=30)
output = model("Write a short poem about the moon")
print(output)
```
### Example 3 - Asking for Advice
```python
model = Zephyr(temperature=0.8, top_p=0.9)
output = model("Give me advice on starting a healthy lifestyle")
print(output)
```
## Additional Information
- Zephyr is based on the Hugging Face Transformers library and uses the "HuggingFaceH4/zephyr-7b-alpha" model.
- The generated text can vary based on the values of `temperature`, `top_k`, and `top_p`. Experiment with these parameters to achieve the desired output.
- The `max_new_tokens` parameter can be adjusted to control the length of the generated text.
- You can integrate Zephyr into chat applications, creative writing projects, or any task that involves generating human-like text.
That concludes the documentation for Zephyr. We hope you find this model useful for your text generation needs! If you have any questions or encounter any issues, please refer to the Hugging Face Transformers documentation for further assistance. Happy text generation!

105
docs/swarms/models/zeroscope.md
Unescape View File

@ -1,105 +0,0 @@
# Module Name: ZeroscopeTTV
## Introduction
The ZeroscopeTTV module is a versatile zero-shot video generation model designed to create videos based on textual descriptions. This comprehensive documentation will provide you with an in-depth understanding of the ZeroscopeTTV module, its architecture, purpose, arguments, and detailed usage examples.
## Purpose
The ZeroscopeTTV module serves as a powerful tool for generating videos from text descriptions. Whether you need to create video content for various applications, visualize textual data, or explore the capabilities of ZeroscopeTTV, this module offers a flexible and efficient solution. With its easy-to-use interface, you can quickly generate videos based on your textual input.
## Architecture
The ZeroscopeTTV module is built on top of the Diffusers library, leveraging the power of diffusion models for video generation. It allows you to specify various parameters such as model name, data type, chunk size, dimensions, and more to customize the video generation process. The model performs multiple inference steps and utilizes a diffusion pipeline to generate high-quality videos.
## Class Definition
### `ZeroscopeTTV(model_name: str = "cerspense/zeroscope_v2_576w", torch_dtype=torch.float16, chunk_size: int = 1, dim: int = 1, num_inference_steps: int = 40, height: int = 320, width: int = 576, num_frames: int = 36)`
#### Parameters
- `model_name` (str, optional): The name of the pre-trained model to use. Default is "cerspense/zeroscope_v2_576w".
- `torch_dtype` (torch.dtype, optional): The torch data type to use for computations. Default is torch.float16.
- `chunk_size` (int, optional): The size of chunks for forward chunking. Default is 1.
- `dim` (int, optional): The dimension along which the input is split for forward chunking. Default is 1.
- `num_inference_steps` (int, optional): The number of inference steps to perform. Default is 40.
- `height` (int, optional): The height of the video frames. Default is 320.
- `width` (int, optional): The width of the video frames. Default is 576.
- `num_frames` (int, optional): The number of frames in the video. Default is 36.
## Functionality and Usage
The ZeroscopeTTV module offers a straightforward interface for video generation. It accepts a textual task or description as input and returns the path to the generated video.
### `run(task: str = None, *args, **kwargs) -> str`
#### Parameters
- `task` (str, optional): The input task or description for video generation.
#### Returns
- `str`: The path to the generated video.
## Usage Examples
### Example 1: Basic Usage
```python
from swarms.models import ZeroscopeTTV
# Initialize the ZeroscopeTTV model
zeroscope = ZeroscopeTTV()
# Generate a video based on a textual description
task = "A bird flying in the sky."
video_path = zeroscope.run(task)
print(f"Generated video path: {video_path}")
```
### Example 2: Custom Model and Parameters
You can specify a custom pre-trained model and adjust various parameters for video generation.
```python
custom_model_name = "your_custom_model_path"
custom_dtype = torch.float32
custom_chunk_size = 2
custom_dim = 2
custom_num_inference_steps = 50
custom_height = 480
custom_width = 720
custom_num_frames = 48
custom_zeroscope = ZeroscopeTTV(
model_name=custom_model_name,
torch_dtype=custom_dtype,
chunk_size=custom_chunk_size,
dim=custom_dim,
num_inference_steps=custom_num_inference_steps,
height=custom_height,
width=custom_width,
num_frames=custom_num_frames,
)
task = "A car driving on the road."
video_path = custom_zeroscope.run(task)
print(f"Generated video path: {video_path}")
```
### Example 3: Exporting Video Frames
You can also export individual video frames if needed.
```python
from swarms.models import export_to_video
# Generate video frames
video_frames = zeroscope.run("A boat sailing on the water.")
# Export video frames to a video file
video_path = export_to_video(video_frames)
print(f"Generated video path: {video_path}")
```
## Additional Information and Tips
- Ensure that the input textual task or description is clear and descriptive to achieve the desired video output.
- Experiment with different parameter settings to control video resolution, frame count, and inference steps.
- Use the `export_to_video` function to export individual video frames as needed.
- Monitor the progress and output paths to access the generated videos.
## Conclusion
The ZeroscopeTTV module is a powerful solution for zero-shot video generation based on textual descriptions. Whether you are creating videos for storytelling, data visualization, or other applications, ZeroscopeTTV offers a versatile and efficient way to bring your text to life. With a flexible interface and customizable parameters, it empowers you to generate high-quality videos with ease.
If you encounter any issues or have questions about using ZeroscopeTTV, please refer to the Diffusers library documentation or reach out to their support team for further assistance. Enjoy creating videos with ZeroscopeTTV!

1
docs/swarms/tokenizers/majorityvoting.md
Unescape View File

@ -1 +0,0 @@
content='# Swarms Structs Documentation\n\n## Module: MajorityVoting\n\n### Overview\n\nThe `MajorityVoting` module in the Swarms Structs library represents a majority voting system for agents. This module allows multiple agents to provide responses to a given task, and the system aggregates these responses to determine the majority vote. The purpose of this module is to leverage the collective intelligence of multiple agents to arrive at consensual decisions or answers.\n\n### Class Definition\n\n```python\nclass MajorityVoting:\n """\n Class representing a majority voting system for agents.\n\n Args:\n agents (list): A list of agents to be used in the majority voting system.\n output_parser (function, optional): A function used to parse the output of the agents.\n If not provided, the default majority voting function is used.\n autosave (bool, optional): A boolean indicating whether to autosave the conversation to a file.\n verbose (bool, optional): A boolean indicating whether to enable verbose logging.\n Examples:\n >>> from swarms.structs.agent import Agent\n >>> from swarms.structs.majority_voting import MajorityVoting\n >>> agents = [\n ... Agent("GPT-3"),\n ... Agent("Codex"),\n ... Agent("Tabnine"),\n ... ]\n >>> majority_voting = MajorityVoting(agents)\n >>> majority_voting.run("What is the capital of France?")\n \'Paris\'\n """\n\n def __init__(\n self,\n agents: List[Agent],\n output_parser: Optional[Callable] = majority_voting,\n autosave: bool = False,\n verbose: bool = False,\n *args,\n **kwargs,\n ):\n # Constructor code here\n```\n\n### Functionality and Usage\n\nThe `MajorityVoting` class is used to create a majority voting system for a given set of agents. The key features and usage of this class are as follows:\n\n- **Initialization**: The class is initialized with a list of agents, an optional output parser function, and flags for autosave and verbose logging.\n- **Run Method**: The `run` method is used to execute the majority voting system. It takes a task as input, routes the task to each agent concurrently, aggregates agent responses, and performs a majority vote.\n\n#### Examples\n\n1. **Basic Usage**:\n\n```python\nfrom swarms.structs.agent import Agent\nfrom swarms.structs.majority_voting import MajorityVoting\n\nagents = [\n Agent("GPT-3"),\n Agent("Codex"),\n Agent("Tabnine"),\n]\nmajority_voting = MajorityVoting(agents)\nresult = majority_voting.run("What is the capital of France?")\nprint(result) # Output: \'Paris\'\n```\n\n2. **Custom Output Parser**:\n\n```python\ndef custom_parser(responses):\n # Custom parsing logic here\n return most_common_response\n\nmajority_voting = MajorityVoting(agents, output_parser=custom_parser)\nresult = majority_voting.run("What is the capital of France?")\n```\n\n3. **Autosave Conversation**:\n\n```python\nmajority_voting = MajorityVoting(agents, autosave=True)\nresult = majority_voting.run("What is the capital of France?")\n```\n\n### Additional Information\n\n- **Threaded Execution**: The `MajorityVoting` class uses a ThreadPoolExecutor for concurrent execution of agents\' tasks.\n- **Logging**: Verbose logging can be enabled to track agent responses and system operations.\n- **Output Parsing**: Custom output parsers can be provided to handle different response aggregation strategies.\n\n### References\n\nFor more information on multi-agent systems and consensus algorithms, refer to the following resources:\n\n1. [Multi-Agent Systems: A Modern Approach to Distributed Artificial Intelligence](https://www.cambridge.org/highereducation/books/multi-agent-systems/7A7C7D3E4E0B6A53C4C4C24E7E5F1B58)\n2. [Consensus Algorithms in Multi-Agent Systems](https://link.springer.com/chapter/10.1007/978-3-319-02813-0_22)\n\n---\n\nThis detailed documentation provides a comprehensive guide for understanding and utilizing the `MajorityVoting` module in the Swarms Structs library. It covers class definition, functionality, usage examples, additional information, and external references to enhance the user\'s knowledge and application of the module.' response_metadata={'token_usage': {'completion_tokens': 918, 'prompt_tokens': 2388, 'total_tokens': 3306}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': 'fp_c2295e73ad', 'finish_reason': 'stop', 'logprobs': None}

1
docs/swarms/tokenizers/stackoverflowswarm.md
View File

File diff suppressed because one or more lines are too long

1
docs/swarms/tokenizers/taskqueuebase.md
Unescape View File

@ -1 +0,0 @@
content='# Module/Class Name: TaskQueueBase\n\n## Overview\nThe `TaskQueueBase` class is a helper class that provides a standard way to create an Abstract Base Class (ABC) using inheritance. It includes methods for adding tasks to a queue, getting the next task from the queue, marking tasks as completed, and resetting tasks if they were not completed successfully.\n\n## Class Definition\n```python\nclass TaskQueueBase(ABC):\n def __init__(self):\n self.lock = threading.Lock()\n\n @synchronized_queue\n @abstractmethod\n def add(self, task: Task) -> bool:\n """Adds a task to the queue.\n\n Args:\n task (Task): The task to be added to the queue.\n\n Returns:\n bool: True if the task was successfully added, False otherwise.\n """\n\n @synchronized_queue\n @abstractmethod\n def get(self, agent: Agent) -> Task:\n """Gets the next task from the queue.\n\n Args:\n agent (Agent): The agent requesting the task.\n\n Returns:\n Task: The next task from the queue.\n """\n\n @synchronized_queue\n @abstractmethod\n def complete_task(self, task_id: str):\n """Sets the task as completed.\n\n Args:\n task_id (str): The ID of the task to be marked as completed.\n """\n\n @synchronized_queue\n @abstractmethod\n def reset(self, task_id: str):\n """Resets the task if the agent failed to complete it.\n\n Args:\n task_id (str): The ID of the task to be reset.\n """\n```\n\n## Functionality and Usage\nThe `TaskQueueBase` class is designed to be used as a base class for creating task queue implementations. It provides a set of abstract methods that must be implemented by subclasses to define the specific behavior of the task queue.\n\n### Adding a Task\nTo add a task to the queue, you need to implement the `add` method in a subclass of `TaskQueueBase`. This method takes a `Task` object as input and returns a boolean value indicating whether the task was successfully added to the queue.\n\n```python\nclass MyTaskQueue(TaskQueueBase):\n def add(self, task: Task) -> bool:\n # Custom implementation to add task to the queue\n pass\n```\n\n### Getting the Next Task\nThe `get` method is used to retrieve the next task from the queue. This method should be implemented in a subclass of `TaskQueueBase` to define how tasks are retrieved from the queue based on the requesting agent.\n\n```python\nclass MyTaskQueue(TaskQueueBase):\n def get(self, agent: Agent) -> Task:\n # Custom implementation to get the next task from the queue\n pass\n```\n\n### Completing and Resetting Tasks\nThe `complete_task` method is used to mark a task as completed, while the `reset` method is used to reset a task if the agent failed to complete it. Subclasses of `TaskQueueBase` should provide implementations for these methods based on the specific requirements of the task queue.\n\n```python\nclass MyTaskQueue(TaskQueueBase):\n def complete_task(self, task_id: str):\n # Custom implementation to mark task as completed\n pass\n\n def reset(self, task_id: str):\n # Custom implementation to reset task\n pass\n```\n\n## Additional Information\n- It is important to ensure thread safety when implementing the methods of `TaskQueueBase` due to the presence of the `threading.Lock` object in the class.\n- Subclasses of `TaskQueueBase` should provide detailed implementations for each abstract method to define the behavior of the task queue.\n- Consider using the `synchronized_queue` decorator to ensure that the queue operations are thread-safe.\n\n## References\n- Python threading documentation: [Python Threading](https://docs.python.org/3/library/threading.html)\n- Abstract Base Classes in Python: [ABCs in Python](https://docs.python.org/3/library/abc.html)' response_metadata={'token_usage': {'completion_tokens': 835, 'prompt_tokens': 1734, 'total_tokens': 2569}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': 'fp_c2295e73ad', 'finish_reason': 'stop', 'logprobs': None}
Write Preview
Loading…
Cancel
Save