docs for Zephyr, Vilt

docs/swarms/models/vilt.md

@@ -0,0 +1,83 @@
+# Vilt Documentation
+
+## Introduction
+
+Welcome to the documentation for Vilt, a Vision-and-Language Transformer (ViLT) model fine-tuned on the VQAv2 dataset. Vilt is a powerful model capable of answering questions about images. This documentation will provide a comprehensive understanding of Vilt, its architecture, usage, and how it can be integrated into your projects.
+
+## Overview
+
+Vilt is based on the Vision-and-Language Transformer (ViLT) architecture, designed for tasks that involve understanding both text and images. It has been fine-tuned on the VQAv2 dataset, making it adept at answering questions about images. This model is particularly useful for tasks where textual and visual information needs to be combined to provide meaningful answers.
+
+## Class Definition
+
+```python
+class Vilt:
+    def __init__(self):
+        """
+        Initialize the Vilt model.
+        """
+```
+
+## Usage
+
+To use the Vilt model, follow these steps:
+
+1. Initialize the Vilt model:
+
+```python
+from swarms.models import Vilt
+model = Vilt()
+```
+
+2. Call the model with a text question and an image URL:
+
+```python
+output = model("What is this image?", "http://images.cocodataset.org/val2017/000000039769.jpg")
+```
+
+### Example 1 - Image Questioning
+
+```python
+model = Vilt()
+output = model("What are the objects in this image?", "http://images.cocodataset.org/val2017/000000039769.jpg")
+print(output)
+```
+
+### Example 2 - Image Analysis
+
+```python
+model = Vilt()
+output = model("Describe the scene in this image.", "http://images.cocodataset.org/val2017/000000039769.jpg")
+print(output)
+```
+
+### Example 3 - Visual Knowledge Retrieval
+
+```python
+model = Vilt()
+output = model("Tell me more about the landmark in this image.", "http://images.cocodataset.org/val2017/000000039769.jpg")
+print(output)
+```
+
+## How Vilt Works
+
+Vilt operates by combining text and image information to generate meaningful answers to questions about the provided image. Here's how it works:
+
+1. **Initialization**: When you create a Vilt instance, it initializes the processor and the model. The processor is responsible for handling the image and text input, while the model is the fine-tuned ViLT model.
+
+2. **Processing Input**: When you call the Vilt model with a text question and an image URL, it downloads the image and processes it along with the text question. This processing step involves tokenization and encoding of the input.
+
+3. **Forward Pass**: The encoded input is then passed through the ViLT model. It calculates the logits, and the answer with the highest probability is selected.
+
+4. **Output**: The predicted answer is returned as the output of the model.
+
+## Parameters
+
+Vilt does not require any specific parameters during initialization. It is pre-configured to work with the "dandelin/vilt-b32-finetuned-vqa" model.
+
+## Additional Information
+
+- Vilt is fine-tuned on the VQAv2 dataset, making it proficient at answering questions about a wide range of images.
+- You can use Vilt for various applications, including image question-answering, image analysis, and visual knowledge retrieval.
+
+That concludes the documentation for Vilt. We hope you find this model useful for your vision-and-language tasks. If you have any questions or encounter any issues, please refer to the Hugging Face Transformers documentation for further assistance. Enjoy working with Vilt!

docs/swarms/models/zephyr.md

@@ -0,0 +1,89 @@
+# Zephyr Documentation
+
+## Introduction
+
+Welcome to the documentation for Zephyr, a language model by Hugging Face designed for text generation tasks. Zephyr is capable of generating text in response to prompts and is highly customizable using various parameters. This document will provide you with a detailed understanding of Zephyr, its purpose, and how to effectively use it in your projects.
+
+## Overview
+
+Zephyr is a text generation model that can be used to generate human-like text based on a given prompt. It utilizes the power of transformers and fine-tuning to create coherent and contextually relevant text. Users can control the generated text's characteristics through parameters such as `temperature`, `top_k`, `top_p`, and `max_new_tokens`.
+
+## Class Definition
+
+```python
+class Zephyr:
+    def __init__(
+        self,
+        max_new_tokens: int = 300,
+        temperature: float = 0.5,
+        top_k: float = 50,
+        top_p: float = 0.95,
+    ):
+        """
+        Initialize the Zephyr model.
+
+        Args:
+            max_new_tokens (int): The maximum number of tokens in the generated text.
+            temperature (float): The temperature parameter, controlling the randomness of the output.
+            top_k (float): The top-k parameter, limiting the vocabulary used in generation.
+            top_p (float): The top-p parameter, controlling the diversity of the output.
+        """
+```
+
+## Parameters
+
+- `max_new_tokens` (int): The maximum number of tokens in the generated text.
+- `temperature` (float): The temperature parameter, controlling the randomness of the output.
+- `top_k` (float): The top-k parameter, limiting the vocabulary used in generation.
+- `top_p` (float): The top-p parameter, controlling the diversity of the output.
+
+## Usage
+
+To use the Zephyr model, follow these steps:
+
+1. Initialize the Zephyr model with your desired parameters:
+
+```python
+from swarms.models import Zephyr
+model = Zephyr(max_new_tokens=300, temperature=0.7, top_k=50, top_p=0.95)
+```
+
+2. Generate text by providing a prompt:
+
+```python
+output = model("Generate a funny joke about cats")
+print(output)
+```
+
+### Example 1 - Generating a Joke
+
+```python
+model = Zephyr(max_new_tokens=100)
+output = model("Tell me a joke about programmers")
+print(output)
+```
+
+### Example 2 - Writing Poetry
+
+```python
+model = Zephyr(temperature=0.2, top_k=30)
+output = model("Write a short poem about the moon")
+print(output)
+```
+
+### Example 3 - Asking for Advice
+
+```python
+model = Zephyr(temperature=0.8, top_p=0.9)
+output = model("Give me advice on starting a healthy lifestyle")
+print(output)
+```
+
+## Additional Information
+
+- Zephyr is based on the Hugging Face Transformers library and uses the "HuggingFaceH4/zephyr-7b-alpha" model.
+- The generated text can vary based on the values of `temperature`, `top_k`, and `top_p`. Experiment with these parameters to achieve the desired output.
+- The `max_new_tokens` parameter can be adjusted to control the length of the generated text.
+- You can integrate Zephyr into chat applications, creative writing projects, or any task that involves generating human-like text.
+
+That concludes the documentation for Zephyr. We hope you find this model useful for your text generation needs! If you have any questions or encounter any issues, please refer to the Hugging Face Transformers documentation for further assistance. Happy text generation!

mkdocs.yml

@@ -95,6 +95,8 @@ nav:
     - Anthropic: "swarms/models/anthropic.md"
     - OpenAI: "swarms/models/openai.md"
     - Fuyu: "swarms/models/fuyu.md"
+    - Zephyr: "swarms/models/zephyr.md"
+    - Vilt: "swarms/models/vilt.md"
   - swarms.structs:
     - Overview: "swarms/structs/overview.md"
     - Workflow: "swarms/structs/workflow.md"