exxa

Former-commit-id: 62618fca8b

docs/examples/omni_agent.md

@@ -0,0 +1,71 @@
+# **Title: A Comprehensive Guide to Setting Up OmniWorker: Your Passport to Multimodal Tasks**
+
+**Introduction**
+- Introduction to OmniWorker
+- Explanation of its use-cases and importance in multimodal tasks
+- Mention of prerequisites: Git, Python 3.x, Terminal or Command Prompt access
+
+**Chapter 1: Cloning the Necessary Repository**
+- Explanation of Git and its use in version control
+- Step-by-step guide on how to clone the OmniWorker repository
+  ```bash
+  !git clone https://github.com/kyegomez/swarms
+  ```
+
+**Chapter 2: Navigating to the Cloned Directory**
+- Explanation of directory navigation in the terminal
+  ```bash
+  %cd /swarms
+  ```
+
+**Chapter 3: Installing the Required Dependencies**
+- Explanation of Python dependencies and the purpose of `requirements.txt` file
+- Step-by-step installation of dependencies
+  ```bash
+  !pip install -r requirements.txt
+  ```
+
+**Chapter 4: Installing Additional Dependencies**
+- Discussion on the additional dependencies and their roles in OmniWorker
+  ```bash
+  !pip install git+https://github.com/IDEA-Research/GroundingDINO.git
+  !pip install git+https://github.com/facebookresearch/segment-anything.git
+  !pip install faiss-gpu
+  !pip install langchain-experimental
+  ```
+
+**Chapter 5: Setting Up Your OpenAI API Key**
+- Explanation of OpenAI API and its key
+- Guide on how to obtain and set up the OpenAI API key
+  ```bash
+  !export OPENAI_API_KEY="your-api-key"
+  ```
+
+**Chapter 6: Running the OmniModal Agent Script**
+- Discussion on the OmniModal Agent script and its functionality
+- Guide on how to run the script
+  ```bash
+  !python3 omnimodal_agent.py
+  ```
+
+**Chapter 7: Importing the Necessary Modules**
+- Discussion on Python modules and their importance
+- Step-by-step guide on importing necessary modules for OmniWorker
+  ```python
+  from langchain.llms import OpenAIChat
+  from swarms.agents import OmniModalAgent
+  ```
+
+**Chapter 8: Creating and Running OmniModalAgent Instance**
+- Explanation of OmniModalAgent instance and its role
+- Guide on how to create and run OmniModalAgent instance
+  ```python
+  llm = OpenAIChat()
+  agent = OmniModalAgent(llm)
+  agent.run("Create a video of a swarm of fish")
+  ```
+
+**Conclusion**
+- Recap of the steps taken to set up OmniWorker
+- Encouragement to explore more functionalities and apply OmniWorker to various multimodal tasks
+

omni_exa.py

@@ -0,0 +1,11 @@
+from exa import Inference
+from swarms.agents import OmniModalAgent
+
+llm = Inference(
+    model_id="mistralai/Mistral-7B-v0.1",
+    quantize=True
+)
+
+agent = OmniModalAgent(llm)
+
+agent.run("Create a video of a swarm of fish")