Zack Bradshaw
2 years ago
committed by
GitHub
commit
e291d60c58
@ -0,0 +1,3 @@
|
||||
.env
|
||||
__pycache__
|
||||
.venv
|
||||
@ -0,0 +1,2 @@
|
||||
*.png filter=lfs diff=lfs merge=lfs -text
|
||||
*.gif filter=lfs diff=lfs merge=lfs -text
|
||||
@ -0,0 +1 @@
|
||||
Subproject commit 0fe322de89d224b877e6bdb510cdd0982cbb58ae
|
||||
@ -0,0 +1 @@
|
||||
Subproject commit 6a94ba0582981be05a0e7f18adcf885ac7edb5b0
|
||||
@ -0,0 +1 @@
|
||||
Subproject commit a21fb2ce34ef144daf7387c21f82a1927e50b16e
|
||||
@ -0,0 +1 @@
|
||||
Subproject commit b98b95af4d4962cfe53f1bcd8db6f55396d011dd
|
||||
@ -0,0 +1,75 @@
|
||||
# Swarms Documentation
|
||||
|
||||
## Overview
|
||||
The Swarm module includes the implementation of two classes, `WorkerNode` and `BossNode`, which respectively represent a worker agent and a boss agent. A worker agent is responsible for completing given tasks, while a boss agent is responsible for creating and managing tasks for the worker agent(s).
|
||||
|
||||
## Key Classes
|
||||
|
||||
### WorkerNode
|
||||
```python
|
||||
class WorkerNode:
|
||||
```
|
||||
|
||||
The WorkerNode class represents an autonomous worker agent that can perform a range of tasks.
|
||||
|
||||
__Methods__:
|
||||
|
||||
- `create_agent(ai_name: str, ai_role: str, human_in_the_loop: bool, search_kwargs: dict) -> None`:
|
||||
|
||||
This method creates a new autonomous agent that can complete tasks. The agent utilizes several tools such as search engines, a file writer/reader, and a multi-modal visual tool.
|
||||
The agent's configuration is customizable through the method parameters.
|
||||
|
||||
```python
|
||||
# Example usage
|
||||
worker_node = WorkerNode(llm, tools, vectorstore)
|
||||
worker_node.create_agent('test_agent', 'test_role', False, {})
|
||||
```
|
||||
|
||||
- `run_agent(prompt: str) -> None`:
|
||||
|
||||
This method runs the agent on a given task, defined by the `prompt` parameter.
|
||||
|
||||
```python
|
||||
# Example usage
|
||||
worker_node = WorkerNode(llm, tools, vectorstore)
|
||||
worker_node.create_agent('test_agent', 'test_role', False, {})
|
||||
worker_node.run_agent('Calculate the square root of 144.')
|
||||
```
|
||||
|
||||
### BossNode
|
||||
```python
|
||||
class BossNode:
|
||||
```
|
||||
|
||||
The BossNode class represents a manager agent that can create tasks and control the execution of these tasks.
|
||||
|
||||
__Methods__:
|
||||
|
||||
- `create_task(objective: str) -> dict`:
|
||||
|
||||
This method creates a new task based on the provided `objective`. The created task is a dictionary with the objective as its value.
|
||||
|
||||
```python
|
||||
# Example usage
|
||||
boss_node = BossNode(llm, vectorstore, task_execution_chain, False, 3)
|
||||
task = boss_node.create_task('Find the square root of 144.')
|
||||
```
|
||||
|
||||
- `execute_task(task: dict) -> None`:
|
||||
|
||||
This method triggers the execution of a given task.
|
||||
|
||||
```python
|
||||
# Example usage
|
||||
boss_node = BossNode(llm, vectorstore, task_execution_chain, False, 3)
|
||||
task = boss_node.create_task('Find the square root of 144.')
|
||||
boss_node.execute_task(task)
|
||||
```
|
||||
|
||||
### Note
|
||||
|
||||
Before creating the WorkerNode and BossNode, make sure to initialize the lower level model (llm), tools, and vectorstore which are used as parameters in the constructors of the two classes.
|
||||
|
||||
In addition, the WorkerNode class uses the MultiModalVisualAgentTool which is a custom tool that enables the worker agent to run multi-modal visual tasks. Ensure that this tool is correctly initialized before running the WorkerNode.
|
||||
|
||||
This documentation provides an overview of the main functionalities of the Swarm module. For additional details and advanced functionalities, please review the source code and the accompanying comments.
|
||||
@ -0,0 +1,214 @@
|
||||
## Swarming Architectures
|
||||
|
||||
Here are three examples of swarming architectures that could be applied in this context.
|
||||
|
||||
1. **Hierarchical Swarms**: In this architecture, a 'lead' agent coordinates the efforts of other agents, distributing tasks based on each agent's unique strengths. The lead agent might be equipped with additional functionality or decision-making capabilities to effectively manage the swarm.
|
||||
|
||||
2. **Collaborative Swarms**: Here, each agent in the swarm works in parallel, potentially on different aspects of a task. They then collectively determine the best output, often through a voting or consensus mechanism.
|
||||
|
||||
3. **Competitive Swarms**: In this setup, multiple agents work on the same task independently. The output from the agent which produces the highest confidence or quality result is then selected. This can often lead to more robust outputs, as the competition drives each agent to perform at its best.
|
||||
|
||||
4. **Multi-Agent Debate**: Here, multiple agents debate a topic. The output from the agent which produces the highest confidence or quality result is then selected. This can lead to more robust outputs, as the competition drives each agent to perform it's best.
|
||||
|
||||
|
||||
# Ideas
|
||||
|
||||
A swarm, particularly in the context of distributed computing, refers to a large number of coordinated agents or nodes that work together to solve a problem. The specific requirements of a swarm might vary depending on the task at hand, but some of the general requirements include:
|
||||
|
||||
1. **Distributed Nature**: The swarm should consist of multiple individual units or nodes, each capable of functioning independently.
|
||||
|
||||
2. **Coordination**: The nodes in the swarm need to coordinate with each other to ensure they're working together effectively. This might involve communication between nodes, or it could be achieved through a central orchestrator.
|
||||
|
||||
3. **Scalability**: A well-designed swarm system should be able to scale up or down as needed, adding or removing nodes based on the task load.
|
||||
|
||||
4. **Resilience**: If a node in the swarm fails, it shouldn't bring down the entire system. Instead, other nodes should be able to pick up the slack.
|
||||
|
||||
5. **Load Balancing**: Tasks should be distributed evenly across the nodes in the swarm to avoid overloading any single node.
|
||||
|
||||
6. **Interoperability**: Each node should be able to interact with others, regardless of differences in underlying hardware or software.
|
||||
|
||||
Integrating these requirements with Large Language Models (LLMs) can be done as follows:
|
||||
|
||||
1. **Distributed Nature**: Each LLM agent can be considered as a node in the swarm. These agents can be distributed across multiple servers or even geographically dispersed data centers.
|
||||
|
||||
2. **Coordination**: An orchestrator can manage the LLM agents, assigning tasks, coordinating responses, and ensuring effective collaboration between agents.
|
||||
|
||||
3. **Scalability**: As the demand for processing power increases or decreases, the number of LLM agents can be adjusted accordingly.
|
||||
|
||||
4. **Resilience**: If an LLM agent goes offline or fails, the orchestrator can assign its tasks to other agents, ensuring the swarm continues functioning smoothly.
|
||||
|
||||
5. **Load Balancing**: The orchestrator can also handle load balancing, ensuring tasks are evenly distributed amongst the LLM agents.
|
||||
|
||||
6. **Interoperability**: By standardizing the input and output formats of the LLM agents, they can effectively communicate and collaborate, regardless of the specific model or configuration of each agent.
|
||||
|
||||
In terms of architecture, the swarm might look something like this:
|
||||
|
||||
```
|
||||
(Orchestrator)
|
||||
/ \
|
||||
Tools + Vector DB -- (LLM Agent)---(Communication Layer) (Communication Layer)---(LLM Agent)-- Tools + Vector DB
|
||||
/ | | \
|
||||
(Task Assignment) (Task Completion) (Task Assignment) (Task Completion)
|
||||
```
|
||||
|
||||
Each LLM agent communicates with the orchestrator through a dedicated communication layer. The orchestrator assigns tasks to each LLM agent, which the agents then complete and return. This setup allows for a high degree of flexibility, scalability, and robustness.
|
||||
|
||||
|
||||
## Communication Layer
|
||||
|
||||
Communication layers play a critical role in distributed systems, enabling interaction between different nodes (agents) and the orchestrator. Here are three potential communication layers for a distributed system, including their strengths and weaknesses:
|
||||
|
||||
1. **Message Queuing Systems (like RabbitMQ, Kafka)**:
|
||||
|
||||
- Strengths: They are highly scalable, reliable, and designed for high-throughput systems. They also ensure delivery of messages and can persist them if necessary. Furthermore, they support various messaging patterns like publish/subscribe, which can be highly beneficial in a distributed system. They also have robust community support.
|
||||
|
||||
- Weaknesses: They can add complexity to the system, including maintenance of the message broker. Moreover, they require careful configuration to perform optimally, and handling failures can sometimes be challenging.
|
||||
|
||||
2. **RESTful APIs**:
|
||||
|
||||
- Strengths: REST is widely adopted, and most programming languages have libraries to easily create RESTful APIs. They leverage standard HTTP(S) protocols and methods and are straightforward to use. Also, they can be stateless, meaning each request contains all the necessary information, enabling scalability.
|
||||
|
||||
- Weaknesses: For real-time applications, REST may not be the best fit due to its synchronous nature. Additionally, handling a large number of API requests can put a strain on the system, causing slowdowns or timeouts.
|
||||
|
||||
3. **gRPC (Google Remote Procedure Call)**:
|
||||
|
||||
- Strengths: gRPC uses Protocol Buffers as its interface definition language, leading to smaller payloads and faster serialization/deserialization compared to JSON (commonly used in RESTful APIs). It supports bidirectional streaming and can use HTTP/2 features, making it excellent for real-time applications.
|
||||
|
||||
- Weaknesses: gRPC is more complex to set up compared to REST. Protocol Buffers' binary format can be more challenging to debug than JSON. It's also not as widely adopted as REST, so tooling and support might be limited in some environments.
|
||||
|
||||
In the context of swarm LLMs, one could consider an **Omni-Vector Embedding Database** for communication. This database could store and manage the high-dimensional vectors produced by each LLM agent.
|
||||
|
||||
- Strengths: This approach would allow for similarity-based lookup and matching of LLM-generated vectors, which can be particularly useful for tasks that involve finding similar outputs or recognizing patterns.
|
||||
|
||||
- Weaknesses: An Omni-Vector Embedding Database might add complexity to the system in terms of setup and maintenance. It might also require significant computational resources, depending on the volume of data being handled and the complexity of the vectors. The handling and transmission of high-dimensional vectors could also pose challenges in terms of network load.
|
||||
|
||||
|
||||
|
||||
|
||||
# Technical Analysis Document: Particle Swarm of AI Agents using Ocean Database
|
||||
|
||||
## Overview
|
||||
|
||||
The goal is to create a particle swarm of AI agents using the OpenAI API for the agents and the Ocean database as the communication space, where the embeddings act as particles. The swarm will work collectively to perform tasks and optimize their behavior based on the interaction with the Ocean database.
|
||||
|
||||
## Algorithmic Overview
|
||||
|
||||
1. Initialize the AI agents and the Ocean database.
|
||||
2. Assign tasks to the AI agents.
|
||||
3. AI agents use the OpenAI API to perform tasks and generate embeddings.
|
||||
4. AI agents store their embeddings in the Ocean database.
|
||||
5. AI agents query the Ocean database for relevant embeddings.
|
||||
6. AI agents update their positions based on the retrieved embeddings.
|
||||
7. Evaluate the performance of the swarm and update the agents' behavior accordingly.
|
||||
8. Repeat steps 3-7 until a stopping criterion is met.
|
||||
|
||||
## Python Implementation Logic
|
||||
|
||||
1. **Initialize the AI agents and the Ocean database.**
|
||||
|
||||
```python
|
||||
import openai
|
||||
import oceandb
|
||||
from oceandb.utils.embedding_functions import ImageBindEmbeddingFunction
|
||||
|
||||
# Initialize Ocean database
|
||||
client = oceandb.Client()
|
||||
text_embedding_function = ImageBindEmbeddingFunction(modality="text")
|
||||
collection = client.create_collection("all-my-documents", embedding_function=text_embedding_function)
|
||||
|
||||
# Initialize AI agents
|
||||
agents = initialize_agents(...)
|
||||
```
|
||||
|
||||
2. **Assign tasks to the AI agents.**
|
||||
|
||||
```python
|
||||
tasks = assign_tasks_to_agents(agents, ...)
|
||||
```
|
||||
|
||||
3. **AI agents use the OpenAI API to perform tasks and generate embeddings.**
|
||||
|
||||
```python
|
||||
def agent_perform_task(agent, task):
|
||||
# Perform the task using the OpenAI API
|
||||
result = perform_task_with_openai_api(agent, task)
|
||||
# Generate the embedding
|
||||
embedding = generate_embedding(result)
|
||||
return embedding
|
||||
|
||||
embeddings = [agent_perform_task(agent, task) for agent, task in zip(agents, tasks)]
|
||||
```
|
||||
|
||||
4. **AI agents store their embeddings in the Ocean database.**
|
||||
|
||||
```python
|
||||
def store_embeddings_in_database(embeddings, collection):
|
||||
for i, embedding in enumerate(embeddings):
|
||||
document_id = f"agent_{i}"
|
||||
collection.add(documents=[embedding], ids=[document_id])
|
||||
|
||||
store_embeddings_in_database(embeddings, collection)
|
||||
```
|
||||
|
||||
5. **AI agents query the Ocean database for relevant embeddings.**
|
||||
|
||||
```python
|
||||
def query_database_for_embeddings(agent, collection, n_results=1):
|
||||
query_result = collection.query(query_texts=[agent], n_results=n_results)
|
||||
return query_result
|
||||
|
||||
queried_embeddings = [query_database_for_embeddings(agent, collection) for agent in agents]
|
||||
```
|
||||
|
||||
6. **AI agents update their positions based on the retrieved embeddings.**
|
||||
|
||||
```python
|
||||
def update_agent_positions(agents, queried_embeddings):
|
||||
for agent, embedding in zip(agents, queried_embeddings):
|
||||
agent.update_position(embedding)
|
||||
|
||||
update_agent_positions(agents, queried_embeddings)
|
||||
```
|
||||
|
||||
7. **Evaluate the performance of the swarm and update the agents' behavior accordingly.**
|
||||
|
||||
```python
|
||||
def evaluate_swarm_performance(agents, ...):
|
||||
# Evaluate the performance of the swarm
|
||||
performance = compute_performance_metric(agents, ...)
|
||||
return performance
|
||||
|
||||
def update_agent_behavior(agents, performance):
|
||||
# Update agents' behavior based on swarm performance
|
||||
for agent in agents:
|
||||
agent.adjust_behavior(performance)
|
||||
|
||||
performance = evaluate_swarm_performance(agents, ...)
|
||||
update_agent_behavior(agents, performance)
|
||||
```
|
||||
|
||||
8. **Repeat steps 3-7 until a stopping criterion is met.**
|
||||
|
||||
```python
|
||||
while not stopping_criterion_met():
|
||||
# Perform tasks and generate embeddings
|
||||
embeddings = [agent_perform_task(agent, task) for agent, task in zip(agents, tasks)]
|
||||
|
||||
# Store embeddings in the Ocean database
|
||||
store_embeddings_in_database(embeddings, collection)
|
||||
|
||||
# Query the Ocean database for relevant embeddings
|
||||
queried_embeddings = [query_database_for_embeddings(agent, collection) for agent in agents]
|
||||
|
||||
# Update AI agent positions based on the retrieved embeddings
|
||||
update_agent_positions(agents, queried_embeddings)
|
||||
|
||||
# Evaluate the performance of the swarm and update the agents' behavior accordingly
|
||||
performance = evaluate_swarm_performance(agents, ...)
|
||||
update_agent_behavior(agents, performance)
|
||||
```
|
||||
|
||||
This code demonstrates the complete loop to repeat steps 3-7 until a stopping criterion is met. You will need to define the `stopping_criterion_met()` function, which could be based on a predefined number of iterations, a target performance level, or any other condition that indicates that the swarm has reached a desired state.
|
||||
|
||||
|
||||
|
||||
@ -0,0 +1,13 @@
|
||||
Today, we stand at the verge of a revolution in artificial intelligence and machine learning. Individual models have accomplished incredible feats, achieving unprecedented levels of understanding and generating incredibly human-like text. But this is just the beginning.
|
||||
|
||||
In the future, we should expect more. These models, which we've seen perform so admirably in isolation, should be able to work together, as a team, a swarm. However, this kind of collaborative intelligence doesn't exist today. That's because the technology to seamlessly integrate these models and foster true inter-model collaboration has been missing, until now.
|
||||
|
||||
In attempting to create this swarm, we face numerous challenges, such as developing the necessary infrastructure, ensuring seamless integration between the agents, and overcoming the practical limitations of our current computing capabilities. These are daunting tasks, and many have shied away from them because of the sheer complexity of the problem. But, if we can overcome these challenges, the rewards will be unimaginable, all digital activities will be automated.
|
||||
|
||||
We envision a future where swarms of Language Learning Model (LLM) agents revolutionize fields like customer support, content creation, and research. Imagine an AI system that could work cohesively, understand complex problems, and deliver multi-faceted solutions. We estimate this could lead to a 100-fold improvement in AI effectiveness, and up to a trillion-dollar impact on the global economy.
|
||||
|
||||
The secret to achieving this lies in our open-source approach and the power of the collective. By embracing open-source, we are enabling hundreds of thousands of minds worldwide to contribute to this vision, each bringing unique insights and solutions. Our bug bounty program and automated testing environments will act as catalysts, motivating and rewarding contributors while ensuring the robustness and reliability of our technology.
|
||||
|
||||
At Agora, we believe in the transformative potential of this technology, and we are committed to making it a reality. Our world-class team of researchers, engineers, and AI enthusiasts are singularly focused on this mission. With a proven track record of success, and the tenacity to tackle the most complex problems, we are best positioned to lead this charge.
|
||||
|
||||
We invite you to join us on this exciting journey. Let's come together to create swarms, advance humanity, and redefine what is possible with artificial intelligence. Our future is in our hands. Let's shape it together.
|
||||
@ -0,0 +1,149 @@
|
||||
# Bounty Program
|
||||
|
||||
Our bounty program is an exciting opportunity for contributors to help us build the future of Swarms. By participating, you can earn rewards while contributing to a project that aims to revolutionize digital activity.
|
||||
|
||||
Here's how it works:
|
||||
|
||||
1. **Check out our Roadmap**: We've shared our roadmap detailing our short and long-term goals. These are the areas where we're seeking contributions.
|
||||
|
||||
2. **Pick a Task**: Choose a task from the roadmap that aligns with your skills and interests. If you're unsure, you can reach out to our team for guidance.
|
||||
|
||||
3. **Get to Work**: Once you've chosen a task, start working on it. Remember, quality is key. We're looking for contributions that truly make a difference.
|
||||
|
||||
4. **Submit your Contribution**: Once your work is complete, submit it for review. We'll evaluate your contribution based on its quality, relevance, and the value it brings to Swarms.
|
||||
|
||||
5. **Earn Rewards**: If your contribution is approved, you'll earn a bounty. The amount of the bounty depends on the complexity of the task, the quality of your work, and the value it brings to Swarms.
|
||||
|
||||
## The Three Phases of Our Bounty Program
|
||||
|
||||
### Phase 1: Building the Foundation
|
||||
In the first phase, our focus is on building the basic infrastructure of Swarms. This includes developing key components like the Swarms class, integrating essential tools, and establishing task completion and evaluation logic. We'll also start developing our testing and evaluation framework during this phase. If you're interested in foundational work and have a knack for building robust, scalable systems, this phase is for you.
|
||||
|
||||
### Phase 2: Enhancing the System
|
||||
In the second phase, we'll focus on enhancing Swarms by integrating more advanced features, improving the system's efficiency, and refining our testing and evaluation framework. This phase involves more complex tasks, so if you enjoy tackling challenging problems and contributing to the development of innovative features, this is the phase for you.
|
||||
|
||||
### Phase 3: Towards Super-Intelligence
|
||||
The third phase of our bounty program is the most exciting - this is where we aim to achieve super-intelligence. In this phase, we'll be working on improving the swarm's capabilities, expanding its skills, and fine-tuning the system based on real-world testing and feedback. If you're excited about the future of AI and want to contribute to a project that could potentially transform the digital world, this is the phase for you.
|
||||
|
||||
Remember, our roadmap is a guide, and we encourage you to bring your own ideas and creativity to the table. We believe that every contribution, no matter how small, can make a difference. So join us on this exciting journey and help us create the future of Swarms.
|
||||
|
||||
**To participate in our bounty program, visit the [Swarms Bounty Program Page](https://swarms.ai/bounty).** Let's build the future together!
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Bounties for Roadmap Items
|
||||
|
||||
To accelerate the development of Swarms and to encourage more contributors to join our journey towards automating every digital activity in existence, we are announcing a Bounty Program for specific roadmap items. Each bounty will be rewarded based on the complexity and importance of the task. Below are the items available for bounty:
|
||||
|
||||
1. **Multi-Agent Debate Integration**: $2000
|
||||
2. **Meta Prompting Integration**: $1500
|
||||
3. **Swarms Class**: $1500
|
||||
4. **Integration of Additional Tools**: $1000
|
||||
5. **Task Completion and Evaluation Logic**: $2000
|
||||
6. **Ocean Integration**: $2500
|
||||
7. **Improved Communication**: $2000
|
||||
8. **Testing and Evaluation**: $1500
|
||||
9. **Worker Swarm Class**: $2000
|
||||
10. **Documentation**: $500
|
||||
|
||||
For each bounty task, there will be a strict evaluation process to ensure the quality of the contribution. This process includes a thorough review of the code and extensive testing to ensure it meets our standards.
|
||||
|
||||
# 3-Phase Testing Framework
|
||||
|
||||
To ensure the quality and efficiency of the Swarm, we will introduce a 3-phase testing framework which will also serve as our evaluation criteria for each of the bounty tasks.
|
||||
|
||||
## Phase 1: Unit Testing
|
||||
In this phase, individual modules will be tested to ensure that they work correctly in isolation. Unit tests will be designed for all functions and methods, with an emphasis on edge cases.
|
||||
|
||||
## Phase 2: Integration Testing
|
||||
After passing unit tests, we will test the integration of different modules to ensure they work correctly together. This phase will also test the interoperability of the Swarm with external systems and libraries.
|
||||
|
||||
## Phase 3: Benchmarking & Stress Testing
|
||||
In the final phase, we will perform benchmarking and stress tests. We'll push the limits of the Swarm under extreme conditions to ensure it performs well in real-world scenarios. This phase will measure the performance, speed, and scalability of the Swarm under high load conditions.
|
||||
|
||||
By following this 3-phase testing framework, we aim to develop a reliable, high-performing, and scalable Swarm that can automate all digital activities.
|
||||
|
||||
# Reverse Engineering to Reach Phase 3
|
||||
|
||||
To reach the Phase 3 level, we need to reverse engineer the tasks we need to complete. Here's an example of what this might look like:
|
||||
|
||||
1. **Set Clear Expectations**: Define what success looks like for each task. Be clear about the outputs and outcomes we expect. This will guide our testing and development efforts.
|
||||
|
||||
2. **Develop Testing Scenarios**: Create a comprehensive list of testing scenarios that cover both common and edge cases. This will help us ensure that our Swarm can handle a wide range of situations.
|
||||
|
||||
3. **Write Test Cases**: For each scenario, write detailed test cases that outline the exact steps to be followed, the inputs to be used, and the expected outputs.
|
||||
|
||||
4. **Execute the Tests**: Run the test cases on our Swarm, making note of any issues or bugs that arise.
|
||||
|
||||
5. **Iterate and Improve**: Based on the results of our tests, iterate and improve our Swarm. This may involve fixing bugs, optimizing code, or redesigning parts of our system.
|
||||
|
||||
6. **Repeat**: Repeat this process until our Swarm meets our expectations and passes all test cases.
|
||||
|
||||
By following these steps, we will systematically build, test, and improve our Swarm until it reaches the Phase 3 level. This methodical approach will help us ensure that we create a reliable, high-performing, and scalable Swarm that can truly automate all digital activities.
|
||||
|
||||
Let's shape the future of digital automation together!
|
||||
|
||||
|
||||
--------------------
|
||||
# Super-Intelligence Roadmap
|
||||
|
||||
Creating a Super-Intelligent Swarm involves three main phases, where each phase has multiple sub-stages, each of which will require rigorous testing and evaluation to ensure progress towards super-intelligence.
|
||||
|
||||
## Phase 1: Narrow Intelligence
|
||||
|
||||
In this phase, the goal is to achieve high performance in specific tasks. These tasks will be predefined and the swarm will be trained and tested on these tasks.
|
||||
|
||||
1. **Single Task Mastery**: Focus on mastering one task at a time. This can range from simple tasks like image recognition to complex tasks like natural language processing.
|
||||
|
||||
2. **Task Switching**: Train the swarm to switch between different tasks effectively. This includes being able to stop one task and start another one without any loss in performance.
|
||||
|
||||
3. **Multi-tasking**: The swarm should be capable of performing multiple tasks simultaneously without any degradation in performance.
|
||||
|
||||
## Phase 2: General Intelligence
|
||||
|
||||
In this phase, the swarm will be trained to handle a variety of tasks that were not part of the original training set.
|
||||
|
||||
1. **Transfer Learning**: The swarm should be able to transfer knowledge learned in one context to another context. This means being able to apply knowledge learned in one task to a different but related task.
|
||||
|
||||
2. **Adaptive Learning**: The swarm should be capable of adapting its learning strategies based on the task at hand. This includes being able to adjust its learning rate, exploration vs exploitation balance, etc.
|
||||
|
||||
3. **Self-Learning**: The swarm should be able to learn new tasks on its own without any external guidance. This includes being able to understand the task requirements, find relevant information, learn the task, and evaluate its performance.
|
||||
|
||||
## Phase 3: Super Intelligence
|
||||
|
||||
In this phase, the swarm will surpass human-level performance in most economically valuable work. This involves the swarm being able to solve complex real-world problems, make accurate predictions, and generate innovative solutions.
|
||||
|
||||
1. **Complex Problem Solving**: The swarm should be able to solve complex real-world problems. This includes being able to understand the problem, identify relevant information, generate solutions, evaluate the solutions, and implement the best solution.
|
||||
|
||||
2. **Predictive Abilities**: The swarm should be able to make accurate predictions about future events based on past data. This includes being able to understand the data, identify relevant patterns, make accurate predictions, and evaluate the accuracy of its predictions.
|
||||
|
||||
3. **Innovation**: The swarm should be able to generate innovative solutions to problems. This includes being able to think creatively, generate novel ideas, evaluate the ideas, and implement the best idea.
|
||||
|
||||
4. **Self-improvement**: The swarm should be capable of improving its own capabilities. This includes being able to identify areas of weakness, find ways to improve, and implement the improvements.
|
||||
|
||||
5. **Understanding**: The swarm should be able to understand complex concepts, make inferences, and draw conclusions. This includes being able to understand natural language, reason logically, and make sound judgments.
|
||||
|
||||
Each of these stages will require extensive testing and evaluation to ensure progress towards super-intelligence.
|
||||
|
||||
# Reverse-Engineering Super-Intelligence
|
||||
|
||||
To reach the Phase 3 level of super-intelligence, we need to reverse engineer the tasks that need to be completed. Here's an outline of what this might look like:
|
||||
|
||||
1. **Setting Success Metrics**: For each stage, define clear success metrics. These metrics should be quantitative and measurable, and they should align with the objectives of the stage.
|
||||
|
||||
2. **Identifying Prerequisites**: Determine what needs to be in place before each stage can begin. This could include certain capabilities, resources, or technologies.
|
||||
|
||||
3. **Developing Training Programs**: For each stage, develop a comprehensive training program. This should include a variety of tasks that will challenge the swarm and push it to
|
||||
|
||||
develop the necessary capabilities.
|
||||
|
||||
4. **Creating Testing Protocols**: Develop rigorous testing protocols for each stage. These protocols should test all aspects of the swarm's performance and they should be designed to push the swarm to its limits.
|
||||
|
||||
5. **Iterating and Improving**: Based on the results of the tests, iterate and improve the swarm. This could involve adjusting the training program, modifying the swarm's architecture, or tweaking its learning algorithms.
|
||||
|
||||
6. **Moving to the Next Stage**: Once the swarm has met the success metrics for a stage, it can move on to the next stage. This process continues until the swarm has reached the level of super-intelligence.
|
||||
|
||||
This process will require a significant amount of time, resources, and effort. However, by following this structured approach, we can systematically guide the swarm towards super-intelligence.
|
||||
|
||||
@ -0,0 +1,91 @@
|
||||
Jeff Bezos, the founder of Amazon.com, is known for his customer-centric approach and long-term strategic thinking. Leveraging his methodology, here are five ways you could monetize the Swarms framework:
|
||||
|
||||
1. **Platform as a Service (PaaS):** Create a cloud-based platform that allows users to build, run, and manage applications without the complexity of maintaining the infrastructure. You could charge users a subscription fee for access to the platform and provide different pricing tiers based on usage levels. This could be an attractive solution for businesses that do not have the capacity to build or maintain their own swarm intelligence solutions.
|
||||
|
||||
2. **Professional Services:** Offer consultancy and implementation services to businesses looking to utilize the Swarm technology. This could include assisting with integration into existing systems, offering custom development services, or helping customers to build specific solutions using the framework.
|
||||
|
||||
3. **Education and Training:** Create a certification program for developers or companies looking to become proficient with the Swarms framework. This could be sold as standalone courses, or bundled with other services.
|
||||
|
||||
4. **Managed Services:** Some companies may prefer to outsource the management of their Swarm-based systems. A managed services solution could take care of all the technical aspects, from hosting the solution to ensuring it runs smoothly, allowing the customer to focus on their core business.
|
||||
|
||||
5. **Data Analysis and Insights:** Swarm intelligence can generate valuable data and insights. By anonymizing and aggregating this data, you could provide industry reports, trend analysis, and other valuable insights to businesses.
|
||||
|
||||
As for the type of platform, Swarms can be offered as a cloud-based solution given its scalability and flexibility. This would also allow you to apply a SaaS/PaaS type monetization model, which provides recurring revenue.
|
||||
|
||||
Potential customers could range from small to large enterprises in various sectors such as logistics, eCommerce, finance, and technology, who are interested in leveraging artificial intelligence and machine learning for complex problem solving, optimization, and decision-making.
|
||||
|
||||
**Product Brief Monetization Strategy:**
|
||||
|
||||
Product Name: Swarms.AI Platform
|
||||
|
||||
Product Description: A cloud-based AI and ML platform harnessing the power of swarm intelligence.
|
||||
|
||||
1. **Platform as a Service (PaaS):** Offer tiered subscription plans (Basic, Premium, Enterprise) to accommodate different usage levels and business sizes.
|
||||
|
||||
2. **Professional Services:** Offer consultancy and custom development services to tailor the Swarms solution to the specific needs of the business.
|
||||
|
||||
3. **Education and Training:** Launch an online Swarms.AI Academy with courses and certifications for developers and businesses.
|
||||
|
||||
4. **Managed Services:** Provide a premium, fully-managed service offering that includes hosting, maintenance, and 24/7 support.
|
||||
|
||||
5. **Data Analysis and Insights:** Offer industry reports and customized insights generated from aggregated and anonymized Swarm data.
|
||||
|
||||
Potential Customers: Enterprises in sectors such as logistics, eCommerce, finance, and technology. This can be sold globally, provided there's an internet connection.
|
||||
|
||||
Marketing Channels: Online marketing (SEO, Content Marketing, Social Media), Partnerships with tech companies, Direct Sales to Enterprises.
|
||||
|
||||
This strategy is designed to provide multiple revenue streams, while ensuring the Swarms.AI platform is accessible and useful to a range of potential customers.
|
||||
|
||||
1. **AI Solution as a Service:** By offering the Swarms framework as a service, businesses can access and utilize the power of multiple LLM agents without the need to maintain the infrastructure themselves. Subscription can be tiered based on usage and additional features.
|
||||
|
||||
2. **Integration and Custom Development:** Offer integration services to businesses wanting to incorporate the Swarms framework into their existing systems. Also, you could provide custom development for businesses with specific needs not met by the standard framework.
|
||||
|
||||
3. **Training and Certification:** Develop an educational platform offering courses, webinars, and certifications on using the Swarms framework. This can serve both developers seeking to broaden their skills and businesses aiming to train their in-house teams.
|
||||
|
||||
4. **Managed Swarms Solutions:** For businesses that prefer to outsource their AI needs, provide a complete solution which includes the development, maintenance, and continuous improvement of swarms-based applications.
|
||||
|
||||
5. **Data Analytics Services:** Leveraging the aggregated insights from the AI swarms, you could offer data analytics services. Businesses can use these insights to make informed decisions and predictions.
|
||||
|
||||
**Type of Platform:**
|
||||
|
||||
Cloud-based platform or Software as a Service (SaaS) will be a suitable model. It offers accessibility, scalability, and ease of updates.
|
||||
|
||||
**Target Customers:**
|
||||
|
||||
The technology can be beneficial for businesses across sectors like eCommerce, technology, logistics, finance, healthcare, and education, among others.
|
||||
|
||||
**Product Brief Monetization Strategy:**
|
||||
|
||||
Product Name: Swarms.AI
|
||||
|
||||
1. **AI Solution as a Service:** Offer different tiered subscriptions (Standard, Premium, and Enterprise) each with varying levels of usage and features.
|
||||
|
||||
2. **Integration and Custom Development:** Offer custom development and integration services, priced based on the scope and complexity of the project.
|
||||
|
||||
3. **Training and Certification:** Launch the Swarms.AI Academy with courses and certifications, available for a fee.
|
||||
|
||||
4. **Managed Swarms Solutions:** Offer fully managed solutions tailored to business needs, priced based on scope and service level agreements.
|
||||
|
||||
5. **Data Analytics Services:** Provide insightful reports and data analyses, which can be purchased on a one-off basis or through a subscription.
|
||||
|
||||
By offering a variety of services and payment models, Swarms.AI will be able to cater to a diverse range of business needs, from small start-ups to large enterprises. Marketing channels would include digital marketing, partnerships with technology companies, presence in tech events, and direct sales to targeted industries.
|
||||
|
||||
|
||||
|
||||
# Roadmap
|
||||
|
||||
* Create a landing page for swarms apac.ai/product/swarms
|
||||
|
||||
* Create Hosted Swarms API for anybody to just use without need for mega gpu infra, charge usage based pricing. Prerequisites for success => Swarms has to be extremely reliable + we need world class documentation and many daily users => how do we get many daily users? We provide a seamless and fluid experience, how do we create a seamless and fluid experience? We write good code that is modular, provides feedback to the user in times of distress, and ultimately accomplishes the user's tasks.
|
||||
|
||||
* Hosted consumer and enterprise subscription as a service on The Domain, where users can interact with 1000s of APIs and ingest 1000s of different data streams.
|
||||
|
||||
* Hosted dedicated capacity deals with mega enterprises on automating many operations with Swarms for monthly subscription 300,000+$
|
||||
|
||||
* Partnerships with enterprises, massive contracts with performance based fee
|
||||
|
||||
* Have discord bot and or slack bot with users personal data, charge subscription + browser extension
|
||||
|
||||
* each user gets a dedicated ocean instance of all their data so the swarm can query it as needed.
|
||||
|
||||
*
|
||||
@ -0,0 +1,30 @@
|
||||
Thank you for contributing to Swarms!
|
||||
|
||||
Replace this comment with:
|
||||
- Description: a description of the change,
|
||||
- Issue: the issue # it fixes (if applicable),
|
||||
- Dependencies: any dependencies required for this change,
|
||||
- Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
|
||||
- Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!
|
||||
|
||||
Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.
|
||||
|
||||
See contribution guidelines for more information on how to write/run tests, lint, etc:
|
||||
https://github.com/kyegomez/swarms/blob/master/CONTRIBUTING.md
|
||||
|
||||
If you're adding a new integration, please include:
|
||||
1. a test for the integration, preferably unit tests that do not rely on network access,
|
||||
2. an example notebook showing its use.
|
||||
|
||||
|
||||
Maintainer responsibilities:
|
||||
- General / Misc / if you don't know who to tag: kye@apac.ai
|
||||
- DataLoaders / VectorStores / Retrievers: kye@apac.ai
|
||||
- swarms.models: kye@apac.ai
|
||||
- swarms.memory: kye@apac.ai
|
||||
- swarms.structures: kye@apac.ai
|
||||
|
||||
If no one reviews your PR within a few days, feel free to email Kye at kye@apac.ai
|
||||
|
||||
See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
|
||||
-->
|
||||
@ -0,0 +1,61 @@
|
||||
|
||||
import os
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Dict, List
|
||||
|
||||
from fastapi.templating import Jinja2Templates
|
||||
|
||||
from swarms.agents.workers.agents import AgentManager
|
||||
from swarms.utils.utils import BaseHandler, FileHandler, FileType, StaticUploader, CsvToDataframe
|
||||
|
||||
from swarms.tools.main import BaseToolSet, ExitConversation, RequestsGet, CodeEditor, Terminal
|
||||
|
||||
from env import settings
|
||||
|
||||
|
||||
BASE_DIR = Path(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
os.chdir(BASE_DIR / os.getenv["PLAYGROUND_DIR"])
|
||||
|
||||
|
||||
toolsets: List[BaseToolSet] = [
|
||||
Terminal(),
|
||||
CodeEditor(),
|
||||
RequestsGet(),
|
||||
ExitConversation(),
|
||||
]
|
||||
handlers: Dict[FileType, BaseHandler] = {FileType.DATAFRAME: CsvToDataframe()}
|
||||
|
||||
if os.getenv["USE_GPU"]:
|
||||
import torch
|
||||
|
||||
from swarms.tools.main import ImageCaptioning
|
||||
from swarms.tools.main import (
|
||||
ImageEditing,
|
||||
InstructPix2Pix,
|
||||
Text2Image,
|
||||
VisualQuestionAnswering,
|
||||
)
|
||||
|
||||
if torch.cuda.is_available():
|
||||
toolsets.extend(
|
||||
[
|
||||
Text2Image("cuda"),
|
||||
ImageEditing("cuda"),
|
||||
InstructPix2Pix("cuda"),
|
||||
VisualQuestionAnswering("cuda"),
|
||||
]
|
||||
)
|
||||
handlers[FileType.IMAGE] = ImageCaptioning("cuda")
|
||||
|
||||
agent_manager = AgentManager.create(toolsets=toolsets)
|
||||
|
||||
file_handler = FileHandler(handlers=handlers, path=BASE_DIR)
|
||||
|
||||
templates = Jinja2Templates(directory=BASE_DIR / "api" / "templates")
|
||||
|
||||
uploader = StaticUploader.from_settings(
|
||||
settings, path=BASE_DIR / "static", endpoint="static"
|
||||
)
|
||||
|
||||
reload_dirs = [BASE_DIR / "swarms", BASE_DIR / "api"]
|
||||
@ -0,0 +1,130 @@
|
||||
import os
|
||||
import re
|
||||
from multiprocessing import Process
|
||||
from tempfile import NamedTemporaryFile
|
||||
|
||||
from typing import List, TypedDict
|
||||
import uvicorn
|
||||
from fastapi import FastAPI, Request, UploadFile
|
||||
from fastapi.responses import HTMLResponse
|
||||
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from pydantic import BaseModel
|
||||
|
||||
from api.container import agent_manager, file_handler, reload_dirs, templates, uploader
|
||||
from api.worker import get_task_result, start_worker, task_execute
|
||||
|
||||
|
||||
app = FastAPI()
|
||||
|
||||
app.mount("/static", StaticFiles(directory=uploader.path), name="static")
|
||||
|
||||
|
||||
class ExecuteRequest(BaseModel):
|
||||
session: str
|
||||
prompt: str
|
||||
files: List[str]
|
||||
|
||||
|
||||
class ExecuteResponse(TypedDict):
|
||||
answer: str
|
||||
files: List[str]
|
||||
|
||||
|
||||
@app.get("/", response_class=HTMLResponse)
|
||||
async def index(request: Request):
|
||||
return templates.TemplateResponse("index.html", {"request": request})
|
||||
|
||||
|
||||
@app.get("/dashboard", response_class=HTMLResponse)
|
||||
async def dashboard(request: Request):
|
||||
return templates.TemplateResponse("dashboard.html", {"request": request})
|
||||
|
||||
|
||||
@app.post("/upload")
|
||||
async def create_upload_file(files: List[UploadFile]):
|
||||
urls = []
|
||||
for file in files:
|
||||
extension = "." + file.filename.split(".")[-1]
|
||||
with NamedTemporaryFile(suffix=extension) as tmp_file:
|
||||
tmp_file.write(file.file.read())
|
||||
tmp_file.flush()
|
||||
urls.append(uploader.upload(tmp_file.name))
|
||||
return {"urls": urls}
|
||||
|
||||
|
||||
@app.post("/api/execute")
|
||||
async def execute(request: ExecuteRequest) -> ExecuteResponse:
|
||||
query = request.prompt
|
||||
files = request.files
|
||||
session = request.session
|
||||
|
||||
executor = agent_manager.create_executor(session)
|
||||
|
||||
promptedQuery = "\n".join([file_handler.handle(file) for file in files])
|
||||
promptedQuery += query
|
||||
|
||||
try:
|
||||
res = executor({"input": promptedQuery})
|
||||
except Exception as e:
|
||||
return {"answer": str(e), "files": []}
|
||||
|
||||
files = re.findall(r"\[file://\S*\]", res["output"])
|
||||
files = [file[1:-1].split("file://")[1] for file in files]
|
||||
|
||||
return {
|
||||
"answer": res["output"],
|
||||
"files": [uploader.upload(file) for file in files],
|
||||
}
|
||||
|
||||
|
||||
@app.post("/api/execute/async")
|
||||
async def execute_async(request: ExecuteRequest):
|
||||
query = request.prompt
|
||||
files = request.files
|
||||
session = request.session
|
||||
|
||||
promptedQuery = "\n".join([file_handler.handle(file) for file in files])
|
||||
promptedQuery += query
|
||||
|
||||
execution = task_execute.delay(session, promptedQuery)
|
||||
return {"id": execution.id}
|
||||
|
||||
|
||||
@app.get("/api/execute/async/{execution_id}")
|
||||
async def execute_async(execution_id: str):
|
||||
execution = get_task_result(execution_id)
|
||||
|
||||
result = {}
|
||||
if execution.status == "SUCCESS" and execution.result:
|
||||
output = execution.result.get("output", "")
|
||||
files = re.findall(r"\[file://\S*\]", output)
|
||||
files = [file[1:-1].split("file://")[1] for file in files]
|
||||
result = {
|
||||
"answer": output,
|
||||
"files": [uploader.upload(file) for file in files],
|
||||
}
|
||||
|
||||
return {
|
||||
"status": execution.status,
|
||||
"info": execution.info,
|
||||
"result": result,
|
||||
}
|
||||
|
||||
|
||||
def serve():
|
||||
p = Process(target=start_worker, args=[])
|
||||
p.start()
|
||||
uvicorn.run("api.main:app", host="0.0.0.0", port=os.getenv["EVAL_PORT"])
|
||||
|
||||
|
||||
def dev():
|
||||
p = Process(target=start_worker, args=[])
|
||||
p.start()
|
||||
uvicorn.run(
|
||||
"api.main:app",
|
||||
host="0.0.0.0",
|
||||
port=os.getenv["EVAL_PORT"],
|
||||
reload=True,
|
||||
reload_dirs=reload_dirs,
|
||||
)
|
||||
@ -0,0 +1,46 @@
|
||||
import os
|
||||
from celery import Celery
|
||||
from celery.result import AsyncResult
|
||||
|
||||
from api.container import agent_manager
|
||||
# from env import settings
|
||||
|
||||
celery_broker = os.environ["CELERY_BROKER_URL"]
|
||||
|
||||
|
||||
celery_app = Celery(__name__)
|
||||
celery_app.conf.broker_url = celery_broker
|
||||
celery_app.conf.result_backend = celery_broker
|
||||
celery_app.conf.update(
|
||||
task_track_started=True,
|
||||
task_serializer="json",
|
||||
accept_content=["json"], # Ignore other content
|
||||
result_serializer="json",
|
||||
enable_utc=True,
|
||||
)
|
||||
|
||||
|
||||
@celery_app.task(name="task_execute", bind=True)
|
||||
def task_execute(self, session: str, prompt: str):
|
||||
executor = agent_manager.create_executor(session, self)
|
||||
response = executor({"input": prompt})
|
||||
result = {"output": response["output"]}
|
||||
|
||||
previous = AsyncResult(self.request.id)
|
||||
if previous and previous.info:
|
||||
result.update(previous.info)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def get_task_result(task_id):
|
||||
return AsyncResult(task_id)
|
||||
|
||||
|
||||
def start_worker():
|
||||
celery_app.worker_main(
|
||||
[
|
||||
"worker",
|
||||
"--loglevel=INFO",
|
||||
]
|
||||
)
|
||||
@ -0,0 +1,363 @@
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from functools import partial
|
||||
from pathlib import Path
|
||||
from threading import Lock
|
||||
import warnings
|
||||
|
||||
from swarms.modelui.modules.block_requests import OpenMonkeyPatch, RequestBlocker
|
||||
from swarms.modelui.modules.logging_colors import logger
|
||||
|
||||
os.environ['GRADIO_ANALYTICS_ENABLED'] = 'False'
|
||||
os.environ['BITSANDBYTES_NOWELCOME'] = '1'
|
||||
warnings.filterwarnings('ignore', category=UserWarning, message='TypedStorage is deprecated')
|
||||
warnings.filterwarnings('ignore', category=UserWarning, message='Using the update method is deprecated')
|
||||
warnings.filterwarnings('ignore', category=UserWarning, message='Field "model_name" has conflict')
|
||||
|
||||
with RequestBlocker():
|
||||
import gradio as gr
|
||||
|
||||
import matplotlib
|
||||
|
||||
matplotlib.use('Agg') # This fixes LaTeX rendering on some systems
|
||||
|
||||
import swarms.modelui.modules.extensions as extensions_module
|
||||
from swarms.modelui.modules import (
|
||||
chat,
|
||||
shared,
|
||||
training,
|
||||
ui,
|
||||
ui_chat,
|
||||
ui_default,
|
||||
ui_file_saving,
|
||||
ui_model_menu,
|
||||
ui_notebook,
|
||||
ui_parameters,
|
||||
ui_session,
|
||||
utils
|
||||
)
|
||||
from swarms.modelui.modules.extensions import apply_extensions
|
||||
from swarms.modelui.modules.LoRA import add_lora_to_model
|
||||
from swarms.modelui.modules.models import load_model
|
||||
from swarms.modelui.modules.models_settings import (
|
||||
get_fallback_settings,
|
||||
get_model_metadata,
|
||||
update_model_parameters
|
||||
)
|
||||
from swarms.modelui.modules.utils import gradio
|
||||
|
||||
import yaml
|
||||
import gradio as gr
|
||||
from swarms.tools.tools_controller import MTQuestionAnswerer, load_valid_tools
|
||||
from swarms.tools.singletool import STQuestionAnswerer
|
||||
from langchain.schema import AgentFinish
|
||||
import os
|
||||
import requests
|
||||
|
||||
from swarms.modelui.server import create_interface
|
||||
from tool_server import run_tool_server
|
||||
from threading import Thread
|
||||
from multiprocessing import Process
|
||||
import time
|
||||
|
||||
tool_server_flag = False
|
||||
def start_tool_server():
|
||||
# server = Thread(target=run_tool_server)
|
||||
server = Process(target=run_tool_server)
|
||||
server.start()
|
||||
global tool_server_flag
|
||||
tool_server_flag = True
|
||||
|
||||
|
||||
available_models = ["ChatGPT", "GPT-3.5"]
|
||||
DEFAULTMODEL = "ChatGPT" # "GPT-3.5"
|
||||
|
||||
tools_mappings = {
|
||||
"klarna": "https://www.klarna.com/",
|
||||
"weather": "http://127.0.0.1:8079/tools/weather/",
|
||||
# "database": "http://127.0.0.1:8079/tools/database/",
|
||||
# "db_diag": "http://127.0.0.1:8079/tools/db_diag/",
|
||||
"chemical-prop": "http://127.0.0.1:8079/tools/chemical-prop/",
|
||||
"douban-film": "http://127.0.0.1:8079/tools/douban-film/",
|
||||
"wikipedia": "http://127.0.0.1:8079/tools/wikipedia/",
|
||||
# "wikidata": "http://127.0.0.1:8079/tools/kg/wikidata/",
|
||||
"wolframalpha": "http://127.0.0.1:8079/tools/wolframalpha/",
|
||||
"bing_search": "http://127.0.0.1:8079/tools/bing_search/",
|
||||
"office-ppt": "http://127.0.0.1:8079/tools/office-ppt/",
|
||||
"stock": "http://127.0.0.1:8079/tools/stock/",
|
||||
"bing_map": "http://127.0.0.1:8079/tools/map.bing_map/",
|
||||
# "baidu_map": "http://127.0.0.1:8079/tools/map/baidu_map/",
|
||||
"zillow": "http://127.0.0.1:8079/tools/zillow/",
|
||||
"airbnb": "http://127.0.0.1:8079/tools/airbnb/",
|
||||
"job_search": "http://127.0.0.1:8079/tools/job_search/",
|
||||
# "baidu-translation": "http://127.0.0.1:8079/tools/translation/baidu-translation/",
|
||||
# "nllb-translation": "http://127.0.0.1:8079/tools/translation/nllb-translation/",
|
||||
"tutorial": "http://127.0.0.1:8079/tools/tutorial/",
|
||||
"file_operation": "http://127.0.0.1:8079/tools/file_operation/",
|
||||
"meta_analysis": "http://127.0.0.1:8079/tools/meta_analysis/",
|
||||
"code_interpreter": "http://127.0.0.1:8079/tools/code_interpreter/",
|
||||
"arxiv": "http://127.0.0.1:8079/tools/arxiv/",
|
||||
"google_places": "http://127.0.0.1:8079/tools/google_places/",
|
||||
"google_serper": "http://127.0.0.1:8079/tools/google_serper/",
|
||||
"google_scholar": "http://127.0.0.1:8079/tools/google_scholar/",
|
||||
"python": "http://127.0.0.1:8079/tools/python/",
|
||||
"sceneXplain": "http://127.0.0.1:8079/tools/sceneXplain/",
|
||||
"shell": "http://127.0.0.1:8079/tools/shell/",
|
||||
"image_generation": "http://127.0.0.1:8079/tools/image_generation/",
|
||||
"hugging_tools": "http://127.0.0.1:8079/tools/hugging_tools/",
|
||||
"gradio_tools": "http://127.0.0.1:8079/tools/gradio_tools/",
|
||||
"travel": "http://127.0.0.1:8079/tools/travel",
|
||||
"walmart": "http://127.0.0.1:8079/tools/walmart",
|
||||
}
|
||||
|
||||
valid_tools_info = []
|
||||
all_tools_list = []
|
||||
|
||||
gr.close_all()
|
||||
|
||||
MAX_TURNS = 30
|
||||
MAX_BOXES = MAX_TURNS * 2
|
||||
|
||||
return_msg = []
|
||||
chat_history = ""
|
||||
|
||||
MAX_SLEEP_TIME = 40
|
||||
def load_tools():
|
||||
global valid_tools_info
|
||||
global all_tools_list
|
||||
try:
|
||||
valid_tools_info = load_valid_tools(tools_mappings)
|
||||
except BaseException as e:
|
||||
print(repr(e))
|
||||
all_tools_list = sorted(list(valid_tools_info.keys()))
|
||||
return gr.update(choices=all_tools_list)
|
||||
|
||||
def set_environ(OPENAI_API_KEY: str,
|
||||
WOLFRAMALPH_APP_ID: str = "",
|
||||
WEATHER_API_KEYS: str = "",
|
||||
BING_SUBSCRIPT_KEY: str = "",
|
||||
ALPHA_VANTAGE_KEY: str = "",
|
||||
BING_MAP_KEY: str = "",
|
||||
BAIDU_TRANSLATE_KEY: str = "",
|
||||
RAPIDAPI_KEY: str = "",
|
||||
SERPER_API_KEY: str = "",
|
||||
GPLACES_API_KEY: str = "",
|
||||
SCENEX_API_KEY: str = "",
|
||||
STEAMSHIP_API_KEY: str = "",
|
||||
HUGGINGFACE_API_KEY: str = "",
|
||||
AMADEUS_ID: str = "",
|
||||
AMADEUS_KEY: str = "",):
|
||||
os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY
|
||||
os.environ["WOLFRAMALPH_APP_ID"] = WOLFRAMALPH_APP_ID
|
||||
os.environ["WEATHER_API_KEYS"] = WEATHER_API_KEYS
|
||||
os.environ["BING_SUBSCRIPT_KEY"] = BING_SUBSCRIPT_KEY
|
||||
os.environ["ALPHA_VANTAGE_KEY"] = ALPHA_VANTAGE_KEY
|
||||
os.environ["BING_MAP_KEY"] = BING_MAP_KEY
|
||||
os.environ["BAIDU_TRANSLATE_KEY"] = BAIDU_TRANSLATE_KEY
|
||||
os.environ["RAPIDAPI_KEY"] = RAPIDAPI_KEY
|
||||
os.environ["SERPER_API_KEY"] = SERPER_API_KEY
|
||||
os.environ["GPLACES_API_KEY"] = GPLACES_API_KEY
|
||||
os.environ["SCENEX_API_KEY"] = SCENEX_API_KEY
|
||||
os.environ["STEAMSHIP_API_KEY"] = STEAMSHIP_API_KEY
|
||||
os.environ["HUGGINGFACE_API_KEY"] = HUGGINGFACE_API_KEY
|
||||
os.environ["AMADEUS_ID"] = AMADEUS_ID
|
||||
os.environ["AMADEUS_KEY"] = AMADEUS_KEY
|
||||
if not tool_server_flag:
|
||||
start_tool_server()
|
||||
time.sleep(MAX_SLEEP_TIME)
|
||||
return gr.update(value="OK!")
|
||||
|
||||
def show_avatar_imgs(tools_chosen):
|
||||
if len(tools_chosen) == 0:
|
||||
tools_chosen = list(valid_tools_info.keys())
|
||||
img_template = '<a href="{}" style="float: left"> <img style="margin:5px" src="{}.png" width="24" height="24" alt="avatar" /> {} </a>'
|
||||
imgs = [valid_tools_info[tool]['avatar'] for tool in tools_chosen if valid_tools_info[tool]['avatar'] != None]
|
||||
imgs = ' '.join([img_template.format(img, img, tool) for img, tool in zip(imgs, tools_chosen)])
|
||||
return [gr.update(value='<span class="">' + imgs + '</span>', visible=True), gr.update(visible=True)]
|
||||
|
||||
def answer_by_tools(question, tools_chosen, model_chosen):
|
||||
global return_msg
|
||||
return_msg += [(question, None), (None, '...')]
|
||||
yield [gr.update(visible=True, value=return_msg), gr.update(), gr.update()]
|
||||
OPENAI_API_KEY = os.environ.get('OPENAI_API_KEY', '')
|
||||
|
||||
if len(tools_chosen) == 0: # if there is no tools chosen, we use all todo (TODO: What if the pool is too large.)
|
||||
tools_chosen = list(valid_tools_info.keys())
|
||||
|
||||
if len(tools_chosen) == 1:
|
||||
answerer = STQuestionAnswerer(OPENAI_API_KEY.strip(), stream_output=True, llm=model_chosen)
|
||||
agent_executor = answerer.load_tools(tools_chosen[0], valid_tools_info[tools_chosen[0]],
|
||||
prompt_type="react-with-tool-description", return_intermediate_steps=True)
|
||||
else:
|
||||
answerer = MTQuestionAnswerer(OPENAI_API_KEY.strip(),
|
||||
load_valid_tools({k: tools_mappings[k] for k in tools_chosen}),
|
||||
stream_output=True, llm=model_chosen)
|
||||
|
||||
agent_executor = answerer.build_runner()
|
||||
|
||||
global chat_history
|
||||
chat_history += "Question: " + question + "\n"
|
||||
question = chat_history
|
||||
for inter in agent_executor(question):
|
||||
if isinstance(inter, AgentFinish): continue
|
||||
result_str = []
|
||||
return_msg.pop()
|
||||
if isinstance(inter, dict):
|
||||
result_str.append("<font color=red>Answer:</font> {}".format(inter['output']))
|
||||
chat_history += "Answer:" + inter['output'] + "\n"
|
||||
result_str.append("...")
|
||||
else:
|
||||
try:
|
||||
not_observation = inter[0].log
|
||||
except:
|
||||
print(inter[0])
|
||||
not_observation = inter[0]
|
||||
if not not_observation.startswith('Thought:'):
|
||||
not_observation = "Thought: " + not_observation
|
||||
chat_history += not_observation
|
||||
not_observation = not_observation.replace('Thought:', '<font color=green>Thought: </font>')
|
||||
not_observation = not_observation.replace('Action:', '<font color=purple>Action: </font>')
|
||||
not_observation = not_observation.replace('Action Input:', '<font color=purple>Action Input: </font>')
|
||||
result_str.append("{}".format(not_observation))
|
||||
result_str.append("<font color=blue>Action output:</font>\n{}".format(inter[1]))
|
||||
chat_history += "\nAction output:" + inter[1] + "\n"
|
||||
result_str.append("...")
|
||||
return_msg += [(None, result) for result in result_str]
|
||||
yield [gr.update(visible=True, value=return_msg), gr.update(), gr.update()]
|
||||
return_msg.pop()
|
||||
if return_msg[-1][1].startswith("<font color=red>Answer:</font> "):
|
||||
return_msg[-1] = (return_msg[-1][0], return_msg[-1][1].replace("<font color=red>Answer:</font> ",
|
||||
"<font color=green>Final Answer:</font> "))
|
||||
yield [gr.update(visible=True, value=return_msg), gr.update(visible=True), gr.update(visible=False)]
|
||||
|
||||
|
||||
def retrieve(tools_search):
|
||||
if tools_search == "":
|
||||
return gr.update(choices=all_tools_list)
|
||||
else:
|
||||
url = "http://127.0.0.1:8079/retrieve"
|
||||
param = {
|
||||
"query": tools_search
|
||||
}
|
||||
response = requests.post(url, json=param)
|
||||
result = response.json()
|
||||
retrieved_tools = result["tools"]
|
||||
return gr.update(choices=retrieved_tools)
|
||||
|
||||
|
||||
def clear_retrieve():
|
||||
return [gr.update(value=""), gr.update(choices=all_tools_list)]
|
||||
|
||||
|
||||
def clear_history():
|
||||
global return_msg
|
||||
global chat_history
|
||||
return_msg = []
|
||||
chat_history = ""
|
||||
yield gr.update(visible=True, value=return_msg)
|
||||
|
||||
title = 'Swarm Models'
|
||||
|
||||
# css/js strings
|
||||
css = ui.css
|
||||
js = ui.js
|
||||
css += apply_extensions('css')
|
||||
js += apply_extensions('js')
|
||||
|
||||
# with gr.Blocks(css=css, analytics_enabled=False, title=title, theme=ui.theme) as demo:
|
||||
with gr.Blocks() as demo:
|
||||
with gr.Row():
|
||||
with gr.Column(scale=14):
|
||||
gr.Markdown("")
|
||||
with gr.Column(scale=1):
|
||||
gr.Image(show_label=False, show_download_button=False, value="images/swarmslogobanner.png")
|
||||
|
||||
with gr.Tab("Models"):
|
||||
create_interface()
|
||||
|
||||
with gr.Tab("Key setting"):
|
||||
OPENAI_API_KEY = gr.Textbox(label="OpenAI API KEY:", placeholder="sk-...", type="text")
|
||||
WOLFRAMALPH_APP_ID = gr.Textbox(label="Wolframalpha app id:", placeholder="Key to use wlframalpha", type="text")
|
||||
WEATHER_API_KEYS = gr.Textbox(label="Weather api key:", placeholder="Key to use weather api", type="text")
|
||||
BING_SUBSCRIPT_KEY = gr.Textbox(label="Bing subscript key:", placeholder="Key to use bing search", type="text")
|
||||
ALPHA_VANTAGE_KEY = gr.Textbox(label="Stock api key:", placeholder="Key to use stock api", type="text")
|
||||
BING_MAP_KEY = gr.Textbox(label="Bing map key:", placeholder="Key to use bing map", type="text")
|
||||
BAIDU_TRANSLATE_KEY = gr.Textbox(label="Baidu translation key:", placeholder="Key to use baidu translation", type="text")
|
||||
RAPIDAPI_KEY = gr.Textbox(label="Rapidapi key:", placeholder="Key to use zillow, airbnb and job search", type="text")
|
||||
SERPER_API_KEY = gr.Textbox(label="Serper key:", placeholder="Key to use google serper and google scholar", type="text")
|
||||
GPLACES_API_KEY = gr.Textbox(label="Google places key:", placeholder="Key to use google places", type="text")
|
||||
SCENEX_API_KEY = gr.Textbox(label="Scenex api key:", placeholder="Key to use sceneXplain", type="text")
|
||||
STEAMSHIP_API_KEY = gr.Textbox(label="Steamship api key:", placeholder="Key to use image generation", type="text")
|
||||
HUGGINGFACE_API_KEY = gr.Textbox(label="Huggingface api key:", placeholder="Key to use models in huggingface hub", type="text")
|
||||
AMADEUS_ID = gr.Textbox(label="Amadeus id:", placeholder="Id to use Amadeus", type="text")
|
||||
AMADEUS_KEY = gr.Textbox(label="Amadeus key:", placeholder="Key to use Amadeus", type="text")
|
||||
key_set_btn = gr.Button(value="Set keys!")
|
||||
|
||||
|
||||
with gr.Tab("Chat with Tool"):
|
||||
with gr.Row():
|
||||
with gr.Column(scale=4):
|
||||
with gr.Row():
|
||||
with gr.Column(scale=0.85):
|
||||
txt = gr.Textbox(show_label=False, placeholder="Question here. Use Shift+Enter to add new line.",
|
||||
lines=1).style(container=False)
|
||||
with gr.Column(scale=0.15, min_width=0):
|
||||
buttonChat = gr.Button("Chat")
|
||||
|
||||
chatbot = gr.Chatbot(show_label=False, visible=True).style(height=600)
|
||||
buttonClear = gr.Button("Clear History")
|
||||
buttonStop = gr.Button("Stop", visible=False)
|
||||
|
||||
with gr.Column(scale=1):
|
||||
model_chosen = gr.Dropdown(
|
||||
list(available_models), value=DEFAULTMODEL, multiselect=False, label="Model provided",
|
||||
info="Choose the model to solve your question, Default means ChatGPT."
|
||||
)
|
||||
with gr.Row():
|
||||
tools_search = gr.Textbox(
|
||||
lines=1,
|
||||
label="Tools Search",
|
||||
placeholder="Please input some text to search tools.",
|
||||
)
|
||||
buttonSearch = gr.Button("Reset search condition")
|
||||
tools_chosen = gr.CheckboxGroup(
|
||||
choices=all_tools_list,
|
||||
value=["chemical-prop"],
|
||||
label="Tools provided",
|
||||
info="Choose the tools to solve your question.",
|
||||
)
|
||||
|
||||
key_set_btn.click(fn=set_environ, inputs=[
|
||||
OPENAI_API_KEY,
|
||||
WOLFRAMALPH_APP_ID,
|
||||
WEATHER_API_KEYS,
|
||||
BING_SUBSCRIPT_KEY,
|
||||
ALPHA_VANTAGE_KEY,
|
||||
BING_MAP_KEY,
|
||||
BAIDU_TRANSLATE_KEY,
|
||||
RAPIDAPI_KEY,
|
||||
SERPER_API_KEY,
|
||||
GPLACES_API_KEY,
|
||||
SCENEX_API_KEY,
|
||||
STEAMSHIP_API_KEY,
|
||||
HUGGINGFACE_API_KEY,
|
||||
AMADEUS_ID,
|
||||
AMADEUS_KEY,
|
||||
], outputs=key_set_btn)
|
||||
key_set_btn.click(fn=load_tools, outputs=tools_chosen)
|
||||
|
||||
tools_search.change(retrieve, tools_search, tools_chosen)
|
||||
buttonSearch.click(clear_retrieve, [], [tools_search, tools_chosen])
|
||||
|
||||
txt.submit(lambda: [gr.update(value=''), gr.update(visible=False), gr.update(visible=True)], [],
|
||||
[txt, buttonClear, buttonStop])
|
||||
inference_event = txt.submit(answer_by_tools, [txt, tools_chosen, model_chosen], [chatbot, buttonClear, buttonStop])
|
||||
buttonChat.click(answer_by_tools, [txt, tools_chosen, model_chosen], [chatbot, buttonClear, buttonStop])
|
||||
buttonStop.click(lambda: [gr.update(visible=True), gr.update(visible=False)], [], [buttonClear, buttonStop],
|
||||
cancels=[inference_event])
|
||||
buttonClear.click(clear_history, [], chatbot)
|
||||
|
||||
# demo.queue().launch(share=False, inbrowser=True, server_name="127.0.0.1", server_port=7001)
|
||||
demo.queue().launch()
|
||||
@ -0,0 +1,8 @@
|
||||
from swarms.models import Dalle3
|
||||
|
||||
dalle3 = Dalle3(
|
||||
openai_api_key=""
|
||||
)
|
||||
task = "A painting of a dog"
|
||||
image_url = dalle3(task)
|
||||
print(image_url)
|
||||
@ -0,0 +1,14 @@
|
||||
version: '3'
|
||||
services:
|
||||
web_demo:
|
||||
build: .
|
||||
command: python host_local_tools.py
|
||||
volumes:
|
||||
- .:/app
|
||||
ports:
|
||||
- "5000:5000"
|
||||
host_local_tools:
|
||||
build: .
|
||||
command: python web_demo.py
|
||||
volumes:
|
||||
- .:/app
|
||||
|
Before Width: | Height: | Size: 200 KiB After Width: | Height: | Size: 131 B |
|
Before Width: | Height: | Size: 235 KiB After Width: | Height: | Size: 131 B |
|
Before Width: | Height: | Size: 148 KiB After Width: | Height: | Size: 131 B |
|
Before Width: | Height: | Size: 170 KiB After Width: | Height: | Size: 131 B |
|
Before Width: | Height: | Size: 878 KiB After Width: | Height: | Size: 131 B |
@ -0,0 +1,17 @@
|
||||
import os
|
||||
from swarms.models import OpenAIChat
|
||||
from swarms.structs.flow import Flow, stop_when_repeats
|
||||
from dotenv import load_dotenv
|
||||
|
||||
load_dotenv()
|
||||
|
||||
# Initialize the OpenAIChat model
|
||||
openai_api_key = os.getenv("OPENAI_API_KEY")
|
||||
llm = OpenAIChat(openai_api_key=openai_api_key)
|
||||
|
||||
# Initialize the Flow
|
||||
flow = Flow(llm=llm, max_loops=3, stopping_condition=stop_when_repeats)
|
||||
|
||||
# Run the Flow with a task
|
||||
response = flow.run("")
|
||||
print(response)
|
||||
@ -0,0 +1,12 @@
|
||||
from swarms.models.gpt4v import GPT4Vision
|
||||
|
||||
|
||||
gpt4vision = GPT4Vision(openai_api_key="")
|
||||
|
||||
img = "https://upload.wikimedia.org/wikipedia/commons/thumb/0/0d/VFPt_Solenoid_correct2.svg/640px-VFPt_Solenoid_correct2.svg.png"
|
||||
|
||||
task = "What is this image"
|
||||
|
||||
answer = gpt4vision.run(task, img)
|
||||
|
||||
print(answer)
|
||||
|
Before Width: | Height: | Size: 1.0 MiB After Width: | Height: | Size: 132 B |
|
Before Width: | Height: | Size: 251 KiB After Width: | Height: | Size: 131 B |
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 130 B |
|
Before Width: | Height: | Size: 1.1 MiB After Width: | Height: | Size: 132 B |
|
Before Width: | Height: | Size: 235 KiB After Width: | Height: | Size: 131 B |
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 130 B |
|
Before Width: | Height: | Size: 402 KiB After Width: | Height: | Size: 131 B |
|
Before Width: | Height: | Size: 283 KiB After Width: | Height: | Size: 131 B |
@ -0,0 +1,4 @@
|
||||
{
|
||||
"internal": [],
|
||||
"visible": []
|
||||
}
|
||||
@ -0,0 +1,173 @@
|
||||
from swarms.tools.serve import ToolServer
|
||||
|
||||
def run_tool_server():
|
||||
# def load_weather_tool():
|
||||
# WEATHER_API_KEYS = os.environ.get('WEATHER_API_KEYS', None)
|
||||
# if not WEATHER_API_KEYS:
|
||||
# raise RuntimeError("WEATHER_API_KEYS not provided, please register one from https://www.weatherapi.com/ and add it to environment variables.")
|
||||
# server.load_tool("weather", {"subscription_key": WEATHER_API_KEYS})
|
||||
|
||||
# def load_database_tool():
|
||||
# server.load_tool("database")
|
||||
|
||||
# def load_db_diag_tool():
|
||||
# server.load_tool("db_diag")
|
||||
|
||||
def load_chemical_prop_tool():
|
||||
server.load_tool("chemical-prop")
|
||||
|
||||
def load_douban_tool():
|
||||
server.load_tool("douban-film")
|
||||
|
||||
def load_wikipedia_tool():
|
||||
server.load_tool("wikipedia")
|
||||
|
||||
# def load_wikidata_tool():
|
||||
# server.load_tool("wikidata")
|
||||
|
||||
def load_travel_tool():
|
||||
server.load_tool("travel")
|
||||
|
||||
# def load_wolframalpha_tool():
|
||||
# WOLFRAMALPH_APP_ID = os.environ.get("WOLFRAMALPH_APP_ID", None)
|
||||
# if not WOLFRAMALPH_APP_ID:
|
||||
# raise RuntimeError("WOLFRAMALPH_APP_ID not provided, please register one from https://products.wolframalpha.com/api/ and add it to environment variables.")
|
||||
# server.load_tool("wolframalpha", {"subscription_key": WOLFRAMALPH_APP_ID})
|
||||
|
||||
# def load_bing_search_tool():
|
||||
# BING_SUBSCRIPT_KEY = os.environ.get('BING_SUBSCRIPT_KEY', None)
|
||||
# if not BING_SUBSCRIPT_KEY:
|
||||
# raise RuntimeError("Bing search key not provided, please register one from https://www.microsoft.com/en-us/bing/apis/bing-web-search-api and add it to environment variables.")
|
||||
# server.load_tool("bing_search", {"subscription_key": BING_SUBSCRIPT_KEY})
|
||||
|
||||
# def load_office_ppt_tool():
|
||||
# server.load_tool("office-ppt")
|
||||
|
||||
# def load_alpha_vantage_tool():
|
||||
# ALPHA_VANTAGE_KEY = os.environ.get('ALPHA_VANTAGE_KEY', None)
|
||||
# if not ALPHA_VANTAGE_KEY:
|
||||
# raise RuntimeError("Stock key not provided, please register one from https://www.alphavantage.co/support/#api-key and add it to environment variables.")
|
||||
# server.load_tool("stock", {"subscription_key": ALPHA_VANTAGE_KEY})
|
||||
|
||||
# def load_map_tool():
|
||||
# BING_MAP_KEY = os.environ.get('BING_MAP_KEY', None)
|
||||
# if not BING_MAP_KEY:
|
||||
# raise RuntimeError("Bing map key not provided, please register one from https://www.bingmapsportal.com/ and add it to environment variables.")
|
||||
# server.load_tool("bing_map", {"subscription_key": BING_MAP_KEY})
|
||||
|
||||
# baidu map tool
|
||||
# BAIDU_SECRET_KEY = os.environ.get('BAIDU_SECRET_KEY', None)
|
||||
# BAIDU_MAP_KEY = os.environ.get('BAIDU_MAP_KEY', None)
|
||||
# if not BAIDU_SECRET_KEY or not BAIDU_MAP_KEY:
|
||||
# raise RuntimeError("Baidu map key not provided, please register one from https://lbsyun.baidu.com/apiconsole/key and add it to environment variables.")
|
||||
# server.load_tool("baidu_map", {"subscription_key": BAIDU_MAP_KEY, "baidu_secret_key": BAIDU_SECRET_KEY})
|
||||
|
||||
# def load_rapidapi_tool():
|
||||
# RAPIDAPI_KEY = os.environ.get('RAPIDAPI_KEY', None)
|
||||
# if not RAPIDAPI_KEY:
|
||||
# raise RuntimeError("RAPIDAPI_KEY not provided, please register one from https://rapidapi.com/ and add it to environment variables.")
|
||||
# server.load_tool("zillow", {"subscription_key": RAPIDAPI_KEY})
|
||||
# server.load_tool("airbnb", {"subscription_key": RAPIDAPI_KEY})
|
||||
# server.load_tool("job_search", {"subscription_key": RAPIDAPI_KEY})
|
||||
|
||||
# def load_nllb_translation_tool():
|
||||
# server.load_tool("nllb-translation")
|
||||
|
||||
# def load_baidu_translation_tool():
|
||||
# server.load_tool("baidu-translation")
|
||||
|
||||
def load_tutorial_tool():
|
||||
server.load_tool("tutorial")
|
||||
|
||||
def load_file_operation_tool():
|
||||
server.load_tool("file_operation")
|
||||
|
||||
def load_meta_analysis_tool():
|
||||
server.load_tool("meta_analysis")
|
||||
|
||||
def load_code_interpreter_tool():
|
||||
server.load_tool("code_interpreter")
|
||||
|
||||
def load_arxiv_tool():
|
||||
server.load_tool("arxiv")
|
||||
|
||||
# def load_google_places_tool():
|
||||
# GPLACES_API_KEY = os.environ.get('GPLACES_API_KEY', '')
|
||||
# if not GPLACES_API_KEY:
|
||||
# raise RuntimeError("GPLACES_API_KEY not provided, please register one from https://developers.google.com/maps/documentation/elevation/get-api-key and add it to environment variables.")
|
||||
# server.load_tool("google_places", {"subscription_key": GPLACES_API_KEY})
|
||||
|
||||
# def load_google_serper_tool():
|
||||
# SERPER_API_KEY = os.environ.get('SERPER_API_KEY', None)
|
||||
# if not SERPER_API_KEY:
|
||||
# raise RuntimeError("SERPER_API_KEY not provided, please register one from https://serper.dev and add it to environment variables.")
|
||||
# server.load_tool("google_serper", {"subscription_key": SERPER_API_KEY})
|
||||
# server.load_tool("google_scholar", {"subscription_key": SERPER_API_KEY})
|
||||
# server.load_tool("walmart", {"subscription_key": SERPER_API_KEY})
|
||||
|
||||
def load_python_tool():
|
||||
server.load_tool("python")
|
||||
|
||||
# def load_sceneXplain_tool():
|
||||
# SCENEX_API_KEY = os.environ.get('SCENEX_API_KEY', None)
|
||||
# if not SCENEX_API_KEY:
|
||||
# raise RuntimeError("SCENEX_API_KEY is not provided. Please sign up for a free account at https://scenex.jina.ai/, create a new API key, and add it to environment variables.")
|
||||
# server.load_tool("sceneXplain", {"subscription_key": SCENEX_API_KEY})
|
||||
|
||||
def load_shell_tool():
|
||||
server.load_tool("shell")
|
||||
|
||||
# def load_image_generation_tool():
|
||||
# STEAMSHIP_API_KEY = os.environ.get('STEAMSHIP_API_KEY', None)
|
||||
# if not STEAMSHIP_API_KEY:
|
||||
# raise RuntimeError("STEAMSHIP_API_KEY is not provided. Please sign up for a free account at https://steamship.com/account/api, create a new API key, and add it to environment variables.")
|
||||
# server.load_tool("image_generation")
|
||||
|
||||
# def load_hugging_tools():
|
||||
# HUGGINGFACE_API_KEY = os.environ.get('HUGGINGFACE_API_KEY', None)
|
||||
# if not HUGGINGFACE_API_KEY:
|
||||
# raise RuntimeError("Huggingface api key (access tokens) not provided, please register one from https://huggingface.co/ and add it to environment variables.")
|
||||
# server.load_tool("hugging_tools")
|
||||
|
||||
def load_gradio_tools():
|
||||
server.load_tool("gradio_tools")
|
||||
|
||||
server = ToolServer()
|
||||
print(server.list_tools())
|
||||
|
||||
# tool_choice = input("Enter 'ALL' to load all tools, or enter the specific tools you want to load (comma-separated): ")
|
||||
|
||||
# load_weather_tool()
|
||||
# load_database_tool()
|
||||
# load_db_diag_tool()
|
||||
load_chemical_prop_tool()
|
||||
load_douban_tool()
|
||||
load_wikipedia_tool()
|
||||
# load_wikidata_tool()
|
||||
# load_wolframalpha_tool()
|
||||
# load_bing_search_tool()
|
||||
# load_office_ppt_tool()
|
||||
# load_alpha_vantage_tool()
|
||||
# load_map_tool()
|
||||
# load_rapidapi_tool()
|
||||
# load_nllb_translation_tool()
|
||||
# load_baidu_translation_tool()
|
||||
load_tutorial_tool()
|
||||
load_file_operation_tool()
|
||||
load_meta_analysis_tool()
|
||||
load_code_interpreter_tool()
|
||||
load_arxiv_tool()
|
||||
# load_google_places_tool()
|
||||
# load_google_serper_tool()
|
||||
load_python_tool()
|
||||
# load_sceneXplain_tool()
|
||||
load_shell_tool()
|
||||
# load_image_generation_tool()
|
||||
# load_hugging_tools()
|
||||
load_gradio_tools()
|
||||
load_travel_tool()
|
||||
|
||||
server.serve()
|
||||
|
||||
if __name__ == "__main__":
|
||||
run_tool_server()
|
||||
@ -0,0 +1,28 @@
|
||||
from swarms.tools.tools_controller import load_valid_tools, MTQuestionAnswerer
|
||||
import jsonlines
|
||||
# Choose the tools that you need
|
||||
tools_mappings = {
|
||||
#"klarna": "https://www.klarna.com/",
|
||||
#"chemical-prop": "http://127.0.0.1:8079/tools/chemical-prop/",
|
||||
"wolframalpha": "http://127.0.0.1:8079/tools/wolframalpha/",
|
||||
#"meta_analysis": "http://127.0.0.1:8079/tools/meta_analysis/",
|
||||
#"map": "http://127.0.0.1:8079/tools/map/",
|
||||
#"douban": "http://127.0.0.1:8079/tools/douban-film/",
|
||||
#"weather": "http://127.0.0.1:8079/tools/weather/",
|
||||
"office-ppt": "http://127.0.0.1:8079/tools/office-ppt/",
|
||||
"wikipedia": "http://127.0.0.1:8079/tools/wikipedia/",
|
||||
#"nllb-translation": "http://127.0.0.1:8079/tools/nllb-translation/",
|
||||
"file_operation": "http://127.0.0.1:8079/tools/file_operation/",
|
||||
"bing_search": "http://127.0.0.1:8079/tools/bing_search/",
|
||||
}
|
||||
|
||||
tools = load_valid_tools(tools_mappings)
|
||||
qa = MTQuestionAnswerer(all_tools=tools)
|
||||
|
||||
agent = qa.build_runner()
|
||||
|
||||
|
||||
agent(["Who's the main actress of Titanic? What did she do apart from this film? Help me make slides with this information."])
|
||||
#agent(['I want to go to Berkeley for one-week vacation. Please help me recommend some tourisms, restaurants, as well as the recent weather conditions for the place.'])
|
||||
#agent(["How many benzene rings are there in 9H-Carbazole-3-carboxaldehyde? and what is sin(x)*exp(x)'s plot, what is it integrated from 0 to 1? "])
|
||||
|
||||
@ -0,0 +1,18 @@
|
||||
export OPENAI_API_KEY='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'
|
||||
export WOLFRAMALPH_APP_ID=''
|
||||
export WEATHER_API_KEYS=''
|
||||
export BING_SUBSCRIPT_KEY=''
|
||||
export ALPHA_VANTAGE_KEY=''
|
||||
export BING_MAP_KEY=''
|
||||
export BAIDU_TRANSLATE_KEY=''
|
||||
export BAIDU_SECRET_KEY=''
|
||||
export HUGGINGFACE_API_KEY=''
|
||||
export BAIDU_MAP_KEY=''
|
||||
export RAPIDAPI_KEY=''
|
||||
export SERPER_API_KEY=''
|
||||
export GPLACES_API_KEY=''
|
||||
export SCENEX_API_KEY=''
|
||||
export STEAMSHIP_API_KEY=''
|
||||
export AMADEUS_ID=''
|
||||
export AMADEUS_KEY=''
|
||||
|
||||
@ -0,0 +1,42 @@
|
||||
from swarms.tools.singletool import load_single_tools, STQuestionAnswerer
|
||||
|
||||
# Langchain
|
||||
tool_name, tool_url = 'weather', "http://127.0.0.1:8079/tools/weather/"
|
||||
tool_name, tool_config = load_single_tools(tool_name, tool_url)
|
||||
print(tool_name, tool_config)
|
||||
stqa = STQuestionAnswerer()
|
||||
|
||||
agent = stqa.load_tools(tool_name, tool_config, prompt_type="react-with-tool-description")
|
||||
agent("write a weather report for SF today")
|
||||
|
||||
# BabyAGI
|
||||
# tool_name, tool_url = 'weather', "http://127.0.0.1:8079/tools/weather/"
|
||||
# tool_name, tool_config = load_single_tools(tool_name, tool_url)
|
||||
# print(tool_name, tool_config)
|
||||
# stqa = STQuestionAnswerer()
|
||||
|
||||
# agent = stqa.load_tools(tool_name, tool_config, prompt_type="babyagi")
|
||||
# agent("write a weather report for SF today")
|
||||
|
||||
# Auto-GPT
|
||||
# tool_name, tool_url = 'weather', "http://127.0.0.1:8079/tools/weather/"
|
||||
# tool_name, tool_config = load_single_tools(tool_name, tool_url)
|
||||
# print(tool_name, tool_config)
|
||||
# stqa = STQuestionAnswerer()
|
||||
|
||||
# agent = stqa.load_tools(tool_name, tool_config, prompt_type="autogpt")
|
||||
# agent.run(["write a weather report for SF today"])
|
||||
|
||||
|
||||
"""
|
||||
from swarms.agent.singletool import load_single_tools, STQuestionAnswerer
|
||||
|
||||
tool_name, tool_url = 'wikipedia', "http://127.0.0.1:8079/tools/wikipedia/"
|
||||
tool_name, tool_config = load_single_tools(tool_name, tool_url)
|
||||
print(tool_name, tool_config)
|
||||
stqa = STQuestionAnswerer()
|
||||
|
||||
agent = stqa.load_tools(tool_name, tool_config, prompt_type="babyagi")
|
||||
# agent = stqa.load_tools(tool_name, tool_config, prompt_type="react-with-tool-description")# prompt_type="babyagi")
|
||||
agent("Where is Yaoming Born?")
|
||||
"""
|
||||
@ -0,0 +1,13 @@
|
||||
from swarms.agent.tools_controller import load_valid_tools, MTQuestionAnswerer
|
||||
tools_mappings = {
|
||||
'weather': "http://127.0.0.1:8079/tools/weather/",
|
||||
'file_operation': "http://127.0.0.1:8079/tools/file_operation/",
|
||||
}
|
||||
|
||||
tools = load_valid_tools(tools_mappings)
|
||||
|
||||
qa = MTQuestionAnswerer(openai_api_key='', all_tools=tools)
|
||||
|
||||
agent = qa.build_runner()
|
||||
|
||||
agent("what is the weather in Beijing?")
|
||||
@ -0,0 +1,184 @@
|
||||
import gradio as gr
|
||||
from swarms.tools.tools_controller import MTQuestionAnswerer, load_valid_tools
|
||||
from swarms.tools.singletool import STQuestionAnswerer
|
||||
from langchain.schema import AgentFinish
|
||||
import os
|
||||
import requests
|
||||
|
||||
available_models = ["ChatGPT", "GPT-3.5"]
|
||||
DEFAULTMODEL = "ChatGPT" # "GPT-3.5"
|
||||
|
||||
tools_mappings = {
|
||||
"klarna": "https://www.klarna.com/",
|
||||
"weather": "http://127.0.0.1:8079/tools/weather/",
|
||||
# "database": "http://127.0.0.1:8079/tools/database/",
|
||||
# "db_diag": "http://127.0.0.1:8079/tools/db_diag/",
|
||||
"chemical-prop": "http://127.0.0.1:8079/tools/chemical-prop/",
|
||||
"douban-film": "http://127.0.0.1:8079/tools/douban-film/",
|
||||
"wikipedia": "http://127.0.0.1:8079/tools/wikipedia/",
|
||||
# "wikidata": "http://127.0.0.1:8079/tools/wikidata/",
|
||||
"wolframalpha": "http://127.0.0.1:8079/tools/wolframalpha/",
|
||||
"bing_search": "http://127.0.0.1:8079/tools/bing_search/",
|
||||
"office-ppt": "http://127.0.0.1:8079/tools/office-ppt/",
|
||||
"stock": "http://127.0.0.1:8079/tools/stock/",
|
||||
"bing_map": "http://127.0.0.1:8079/tools/bing_map/",
|
||||
# "baidu_map": "http://127.0.0.1:8079/tools/baidu_map/",
|
||||
"zillow": "http://127.0.0.1:8079/tools/zillow/",
|
||||
"airbnb": "http://127.0.0.1:8079/tools/airbnb/",
|
||||
"job_search": "http://127.0.0.1:8079/tools/job_search/",
|
||||
# "baidu-translation": "http://127.0.0.1:8079/tools/baidu-translation/",
|
||||
# "nllb-translation": "http://127.0.0.1:8079/tools/nllb-translation/",
|
||||
"tutorial": "http://127.0.0.1:8079/tools/tutorial/",
|
||||
"file_operation": "http://127.0.0.1:8079/tools/file_operation/",
|
||||
"meta_analysis": "http://127.0.0.1:8079/tools/meta_analysis/",
|
||||
"code_interpreter": "http://127.0.0.1:8079/tools/code_interpreter/",
|
||||
"arxiv": "http://127.0.0.1:8079/tools/arxiv/",
|
||||
"google_places": "http://127.0.0.1:8079/tools/google_places/",
|
||||
"google_serper": "http://127.0.0.1:8079/tools/google_serper/",
|
||||
"google_scholar": "http://127.0.0.1:8079/tools/google_scholar/",
|
||||
"python": "http://127.0.0.1:8079/tools/python/",
|
||||
"sceneXplain": "http://127.0.0.1:8079/tools/sceneXplain/",
|
||||
"shell": "http://127.0.0.1:8079/tools/shell/",
|
||||
"image_generation": "http://127.0.0.1:8079/tools/image_generation/",
|
||||
"hugging_tools": "http://127.0.0.1:8079/tools/hugging_tools/",
|
||||
"gradio_tools": "http://127.0.0.1:8079/tools/gradio_tools/",
|
||||
}
|
||||
|
||||
valid_tools_info = load_valid_tools(tools_mappings)
|
||||
print(valid_tools_info)
|
||||
all_tools_list = sorted(list(valid_tools_info.keys()))
|
||||
|
||||
gr.close_all()
|
||||
|
||||
MAX_TURNS = 30
|
||||
MAX_BOXES = MAX_TURNS * 2
|
||||
|
||||
return_msg = []
|
||||
chat_history = ""
|
||||
|
||||
def show_avatar_imgs(tools_chosen):
|
||||
if len(tools_chosen) == 0:
|
||||
tools_chosen = list(valid_tools_info.keys())
|
||||
img_template = '<a href="{}" style="float: left"> <img style="margin:5px" src="{}.png" width="24" height="24" alt="avatar" /> {} </a>'
|
||||
imgs = [valid_tools_info[tool]['avatar'] for tool in tools_chosen if valid_tools_info[tool]['avatar'] != None]
|
||||
imgs = ' '.join([img_template.format(img, img, tool ) for img, tool in zip(imgs, tools_chosen) ])
|
||||
return [gr.update(value='<span class="">'+imgs+'</span>', visible=True), gr.update(visible=True)]
|
||||
|
||||
def answer_by_tools(question, tools_chosen, model_chosen):
|
||||
global return_msg
|
||||
return_msg += [(question, None), (None, '...')]
|
||||
yield [gr.update(visible=True, value=return_msg), gr.update(), gr.update()]
|
||||
OPENAI_API_KEY = os.environ.get('OPENAI_API_KEY', '')
|
||||
|
||||
if len(tools_chosen) == 0: # if there is no tools chosen, we use all todo (TODO: What if the pool is too large.)
|
||||
tools_chosen = list(valid_tools_info.keys())
|
||||
|
||||
if len(tools_chosen) == 1:
|
||||
answerer = STQuestionAnswerer(OPENAI_API_KEY.strip(), stream_output=True, llm=model_chosen)
|
||||
agent_executor = answerer.load_tools(tools_chosen[0], valid_tools_info[tools_chosen[0]], prompt_type="react-with-tool-description", return_intermediate_steps=True)
|
||||
else:
|
||||
answerer = MTQuestionAnswerer(OPENAI_API_KEY.strip(), load_valid_tools({k: tools_mappings[k] for k in tools_chosen}), stream_output=True, llm=model_chosen)
|
||||
|
||||
agent_executor = answerer.build_runner()
|
||||
|
||||
global chat_history
|
||||
chat_history += "Question: " + question + "\n"
|
||||
question = chat_history
|
||||
for inter in agent_executor(question):
|
||||
if isinstance(inter, AgentFinish): continue
|
||||
result_str = []
|
||||
return_msg.pop()
|
||||
if isinstance(inter, dict):
|
||||
result_str.append("<font color=red>Answer:</font> {}".format(inter['output']))
|
||||
chat_history += "Answer:" + inter['output'] + "\n"
|
||||
result_str.append("...")
|
||||
else:
|
||||
not_observation = inter[0].log
|
||||
if not not_observation.startswith('Thought:'):
|
||||
not_observation = "Thought: " + not_observation
|
||||
chat_history += not_observation
|
||||
not_observation = not_observation.replace('Thought:', '<font color=green>Thought: </font>')
|
||||
not_observation = not_observation.replace('Action:', '<font color=purple>Action: </font>')
|
||||
not_observation = not_observation.replace('Action Input:', '<font color=purple>Action Input: </font>')
|
||||
result_str.append("{}".format(not_observation))
|
||||
result_str.append("<font color=blue>Action output:</font>\n{}".format(inter[1]))
|
||||
chat_history += "\nAction output:" + inter[1] + "\n"
|
||||
result_str.append("...")
|
||||
return_msg += [(None, result) for result in result_str]
|
||||
yield [gr.update(visible=True, value=return_msg), gr.update(), gr.update()]
|
||||
return_msg.pop()
|
||||
if return_msg[-1][1].startswith("<font color=red>Answer:</font> "):
|
||||
return_msg[-1] = (return_msg[-1][0], return_msg[-1][1].replace("<font color=red>Answer:</font> ", "<font color=green>Final Answer:</font> "))
|
||||
yield [gr.update(visible=True, value=return_msg), gr.update(visible=True), gr.update(visible=False)]
|
||||
|
||||
def retrieve(tools_search):
|
||||
if tools_search == "":
|
||||
return gr.update(choices=all_tools_list)
|
||||
else:
|
||||
url = "http://127.0.0.1:8079/retrieve"
|
||||
param = {
|
||||
"query": tools_search
|
||||
}
|
||||
response = requests.post(url, json=param)
|
||||
result = response.json()
|
||||
retrieved_tools = result["tools"]
|
||||
return gr.update(choices=retrieved_tools)
|
||||
|
||||
def clear_retrieve():
|
||||
return [gr.update(value=""), gr.update(choices=all_tools_list)]
|
||||
|
||||
def clear_history():
|
||||
global return_msg
|
||||
global chat_history
|
||||
return_msg = []
|
||||
chat_history = ""
|
||||
yield gr.update(visible=True, value=return_msg)
|
||||
|
||||
with gr.Blocks() as demo:
|
||||
with gr.Row():
|
||||
with gr.Column(scale=14):
|
||||
gr.Markdown("<h1 align='left'> Swarm Tools </h1>")
|
||||
with gr.Column(scale=1):
|
||||
gr.Image('../../images/swarmslogobanner.png', show_download_button=False, show_label=False )
|
||||
# gr.Markdown('<img src="../../images/swarmslogobanner.png" alt="swarms">')
|
||||
with gr.Row():
|
||||
with gr.Column(scale=4):
|
||||
with gr.Row():
|
||||
with gr.Column(scale=0.85):
|
||||
txt = gr.Textbox(show_label=False, placeholder="Question here. Use Shift+Enter to add new line.", lines=1).style(container=False)
|
||||
with gr.Column(scale=0.15, min_width=0):
|
||||
buttonChat = gr.Button("Chat")
|
||||
|
||||
chatbot = gr.Chatbot(show_label=False, visible=True).style(height=600)
|
||||
buttonClear = gr.Button("Clear History")
|
||||
buttonStop = gr.Button("Stop", visible=False)
|
||||
|
||||
with gr.Column(scale=1):
|
||||
model_chosen = gr.Dropdown(
|
||||
list(available_models), value=DEFAULTMODEL, multiselect=False, label="Model provided",
|
||||
info="Choose the model to solve your question, Default means ChatGPT."
|
||||
)
|
||||
with gr.Row():
|
||||
tools_search = gr.Textbox(
|
||||
lines=1,
|
||||
label="Tools Search",
|
||||
placeholder="Please input some text to search tools.",
|
||||
)
|
||||
buttonSearch = gr.Button("Reset search condition")
|
||||
tools_chosen = gr.CheckboxGroup(
|
||||
choices=all_tools_list,
|
||||
value=["chemical-prop"],
|
||||
label="Tools provided",
|
||||
info="Choose the tools to solve your question.",
|
||||
)
|
||||
|
||||
tools_search.change(retrieve, tools_search, tools_chosen)
|
||||
buttonSearch.click(clear_retrieve, [], [tools_search, tools_chosen])
|
||||
|
||||
txt.submit(lambda : [gr.update(value=''), gr.update(visible=False), gr.update(visible=True)], [], [txt, buttonClear, buttonStop])
|
||||
inference_event = txt.submit(answer_by_tools, [txt, tools_chosen, model_chosen], [chatbot, buttonClear, buttonStop])
|
||||
buttonChat.click(answer_by_tools, [txt, tools_chosen, model_chosen], [chatbot, buttonClear, buttonStop])
|
||||
buttonStop.click(lambda : [gr.update(visible=True), gr.update(visible=False)], [], [buttonClear, buttonStop], cancels=[inference_event])
|
||||
buttonClear.click(clear_history, [], chatbot)
|
||||
|
||||
demo.queue().launch(share=False, inbrowser=True, server_name="127.0.0.1", server_port=7001)
|
||||
@ -0,0 +1 @@
|
||||
images/swarms_demo.mp4 filter=lfs diff=lfs merge=lfs -text
|
||||
@ -0,0 +1,299 @@
|
||||
import gradio as gr
|
||||
from swarms.tools.tools_controller import MTQuestionAnswerer, load_valid_tools
|
||||
from swarms.tools.singletool import STQuestionAnswerer
|
||||
from langchain.schema import AgentFinish
|
||||
import os
|
||||
import requests
|
||||
|
||||
from tool_server import run_tool_server
|
||||
from threading import Thread
|
||||
from multiprocessing import Process
|
||||
import time
|
||||
|
||||
tool_server_flag = False
|
||||
def start_tool_server():
|
||||
# server = Thread(target=run_tool_server)
|
||||
server = Process(target=run_tool_server)
|
||||
server.start()
|
||||
global tool_server_flag
|
||||
tool_server_flag = True
|
||||
|
||||
|
||||
available_models = ["ChatGPT", "GPT-3.5"]
|
||||
DEFAULTMODEL = "ChatGPT" # "GPT-3.5"
|
||||
|
||||
tools_mappings = {
|
||||
"klarna": "https://www.klarna.com/",
|
||||
"weather": "http://127.0.0.1:8079/tools/weather/",
|
||||
# "database": "http://127.0.0.1:8079/tools/database/",
|
||||
# "db_diag": "http://127.0.0.1:8079/tools/db_diag/",
|
||||
"chemical-prop": "http://127.0.0.1:8079/tools/chemical-prop/",
|
||||
"douban-film": "http://127.0.0.1:8079/tools/douban-film/",
|
||||
"wikipedia": "http://127.0.0.1:8079/tools/wikipedia/",
|
||||
# "wikidata": "http://127.0.0.1:8079/tools/kg/wikidata/",
|
||||
"wolframalpha": "http://127.0.0.1:8079/tools/wolframalpha/",
|
||||
"bing_search": "http://127.0.0.1:8079/tools/bing_search/",
|
||||
"office-ppt": "http://127.0.0.1:8079/tools/office-ppt/",
|
||||
"stock": "http://127.0.0.1:8079/tools/stock/",
|
||||
"bing_map": "http://127.0.0.1:8079/tools/map.bing_map/",
|
||||
# "baidu_map": "http://127.0.0.1:8079/tools/map/baidu_map/",
|
||||
"zillow": "http://127.0.0.1:8079/tools/zillow/",
|
||||
"airbnb": "http://127.0.0.1:8079/tools/airbnb/",
|
||||
"job_search": "http://127.0.0.1:8079/tools/job_search/",
|
||||
# "baidu-translation": "http://127.0.0.1:8079/tools/translation/baidu-translation/",
|
||||
# "nllb-translation": "http://127.0.0.1:8079/tools/translation/nllb-translation/",
|
||||
"tutorial": "http://127.0.0.1:8079/tools/tutorial/",
|
||||
"file_operation": "http://127.0.0.1:8079/tools/file_operation/",
|
||||
"meta_analysis": "http://127.0.0.1:8079/tools/meta_analysis/",
|
||||
"code_interpreter": "http://127.0.0.1:8079/tools/code_interpreter/",
|
||||
"arxiv": "http://127.0.0.1:8079/tools/arxiv/",
|
||||
"google_places": "http://127.0.0.1:8079/tools/google_places/",
|
||||
"google_serper": "http://127.0.0.1:8079/tools/google_serper/",
|
||||
"google_scholar": "http://127.0.0.1:8079/tools/google_scholar/",
|
||||
"python": "http://127.0.0.1:8079/tools/python/",
|
||||
"sceneXplain": "http://127.0.0.1:8079/tools/sceneXplain/",
|
||||
"shell": "http://127.0.0.1:8079/tools/shell/",
|
||||
"image_generation": "http://127.0.0.1:8079/tools/image_generation/",
|
||||
"hugging_tools": "http://127.0.0.1:8079/tools/hugging_tools/",
|
||||
"gradio_tools": "http://127.0.0.1:8079/tools/gradio_tools/",
|
||||
"travel": "http://127.0.0.1:8079/tools/travel",
|
||||
"walmart": "http://127.0.0.1:8079/tools/walmart",
|
||||
}
|
||||
|
||||
valid_tools_info = []
|
||||
all_tools_list = []
|
||||
|
||||
gr.close_all()
|
||||
|
||||
MAX_TURNS = 30
|
||||
MAX_BOXES = MAX_TURNS * 2
|
||||
|
||||
return_msg = []
|
||||
chat_history = ""
|
||||
|
||||
MAX_SLEEP_TIME = 40
|
||||
def load_tools():
|
||||
global valid_tools_info
|
||||
global all_tools_list
|
||||
try:
|
||||
valid_tools_info = load_valid_tools(tools_mappings)
|
||||
except BaseException as e:
|
||||
print(repr(e))
|
||||
all_tools_list = sorted(list(valid_tools_info.keys()))
|
||||
return gr.update(choices=all_tools_list)
|
||||
|
||||
def set_environ(OPENAI_API_KEY: str,
|
||||
WOLFRAMALPH_APP_ID: str = "",
|
||||
WEATHER_API_KEYS: str = "",
|
||||
BING_SUBSCRIPT_KEY: str = "",
|
||||
ALPHA_VANTAGE_KEY: str = "",
|
||||
BING_MAP_KEY: str = "",
|
||||
BAIDU_TRANSLATE_KEY: str = "",
|
||||
RAPIDAPI_KEY: str = "",
|
||||
SERPER_API_KEY: str = "",
|
||||
GPLACES_API_KEY: str = "",
|
||||
SCENEX_API_KEY: str = "",
|
||||
STEAMSHIP_API_KEY: str = "",
|
||||
HUGGINGFACE_API_KEY: str = "",
|
||||
AMADEUS_ID: str = "",
|
||||
AMADEUS_KEY: str = "",):
|
||||
os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY
|
||||
os.environ["WOLFRAMALPH_APP_ID"] = WOLFRAMALPH_APP_ID
|
||||
os.environ["WEATHER_API_KEYS"] = WEATHER_API_KEYS
|
||||
os.environ["BING_SUBSCRIPT_KEY"] = BING_SUBSCRIPT_KEY
|
||||
os.environ["ALPHA_VANTAGE_KEY"] = ALPHA_VANTAGE_KEY
|
||||
os.environ["BING_MAP_KEY"] = BING_MAP_KEY
|
||||
os.environ["BAIDU_TRANSLATE_KEY"] = BAIDU_TRANSLATE_KEY
|
||||
os.environ["RAPIDAPI_KEY"] = RAPIDAPI_KEY
|
||||
os.environ["SERPER_API_KEY"] = SERPER_API_KEY
|
||||
os.environ["GPLACES_API_KEY"] = GPLACES_API_KEY
|
||||
os.environ["SCENEX_API_KEY"] = SCENEX_API_KEY
|
||||
os.environ["STEAMSHIP_API_KEY"] = STEAMSHIP_API_KEY
|
||||
os.environ["HUGGINGFACE_API_KEY"] = HUGGINGFACE_API_KEY
|
||||
os.environ["AMADEUS_ID"] = AMADEUS_ID
|
||||
os.environ["AMADEUS_KEY"] = AMADEUS_KEY
|
||||
if not tool_server_flag:
|
||||
start_tool_server()
|
||||
time.sleep(MAX_SLEEP_TIME)
|
||||
return gr.update(value="OK!")
|
||||
|
||||
def show_avatar_imgs(tools_chosen):
|
||||
if len(tools_chosen) == 0:
|
||||
tools_chosen = list(valid_tools_info.keys())
|
||||
img_template = '<a href="{}" style="float: left"> <img style="margin:5px" src="{}.png" width="24" height="24" alt="avatar" /> {} </a>'
|
||||
imgs = [valid_tools_info[tool]['avatar'] for tool in tools_chosen if valid_tools_info[tool]['avatar'] != None]
|
||||
imgs = ' '.join([img_template.format(img, img, tool) for img, tool in zip(imgs, tools_chosen)])
|
||||
return [gr.update(value='<span class="">' + imgs + '</span>', visible=True), gr.update(visible=True)]
|
||||
|
||||
def answer_by_tools(question, tools_chosen, model_chosen):
|
||||
global return_msg
|
||||
return_msg += [(question, None), (None, '...')]
|
||||
yield [gr.update(visible=True, value=return_msg), gr.update(), gr.update()]
|
||||
OPENAI_API_KEY = os.environ.get('OPENAI_API_KEY', '')
|
||||
|
||||
if len(tools_chosen) == 0: # if there is no tools chosen, we use all todo (TODO: What if the pool is too large.)
|
||||
tools_chosen = list(valid_tools_info.keys())
|
||||
|
||||
if len(tools_chosen) == 1:
|
||||
answerer = STQuestionAnswerer(OPENAI_API_KEY.strip(), stream_output=True, llm=model_chosen)
|
||||
agent_executor = answerer.load_tools(tools_chosen[0], valid_tools_info[tools_chosen[0]],
|
||||
prompt_type="react-with-tool-description", return_intermediate_steps=True)
|
||||
else:
|
||||
answerer = MTQuestionAnswerer(OPENAI_API_KEY.strip(),
|
||||
load_valid_tools({k: tools_mappings[k] for k in tools_chosen}),
|
||||
stream_output=True, llm=model_chosen)
|
||||
|
||||
agent_executor = answerer.build_runner()
|
||||
|
||||
global chat_history
|
||||
chat_history += "Question: " + question + "\n"
|
||||
question = chat_history
|
||||
for inter in agent_executor(question):
|
||||
if isinstance(inter, AgentFinish): continue
|
||||
result_str = []
|
||||
return_msg.pop()
|
||||
if isinstance(inter, dict):
|
||||
result_str.append("<font color=red>Answer:</font> {}".format(inter['output']))
|
||||
chat_history += "Answer:" + inter['output'] + "\n"
|
||||
result_str.append("...")
|
||||
else:
|
||||
try:
|
||||
not_observation = inter[0].log
|
||||
except:
|
||||
print(inter[0])
|
||||
not_observation = inter[0]
|
||||
if not not_observation.startswith('Thought:'):
|
||||
not_observation = "Thought: " + not_observation
|
||||
chat_history += not_observation
|
||||
not_observation = not_observation.replace('Thought:', '<font color=green>Thought: </font>')
|
||||
not_observation = not_observation.replace('Action:', '<font color=purple>Action: </font>')
|
||||
not_observation = not_observation.replace('Action Input:', '<font color=purple>Action Input: </font>')
|
||||
result_str.append("{}".format(not_observation))
|
||||
result_str.append("<font color=blue>Action output:</font>\n{}".format(inter[1]))
|
||||
chat_history += "\nAction output:" + inter[1] + "\n"
|
||||
result_str.append("...")
|
||||
return_msg += [(None, result) for result in result_str]
|
||||
yield [gr.update(visible=True, value=return_msg), gr.update(), gr.update()]
|
||||
return_msg.pop()
|
||||
if return_msg[-1][1].startswith("<font color=red>Answer:</font> "):
|
||||
return_msg[-1] = (return_msg[-1][0], return_msg[-1][1].replace("<font color=red>Answer:</font> ",
|
||||
"<font color=green>Final Answer:</font> "))
|
||||
yield [gr.update(visible=True, value=return_msg), gr.update(visible=True), gr.update(visible=False)]
|
||||
|
||||
|
||||
def retrieve(tools_search):
|
||||
if tools_search == "":
|
||||
return gr.update(choices=all_tools_list)
|
||||
else:
|
||||
url = "http://127.0.0.1:8079/retrieve"
|
||||
param = {
|
||||
"query": tools_search
|
||||
}
|
||||
response = requests.post(url, json=param)
|
||||
result = response.json()
|
||||
retrieved_tools = result["tools"]
|
||||
return gr.update(choices=retrieved_tools)
|
||||
|
||||
|
||||
def clear_retrieve():
|
||||
return [gr.update(value=""), gr.update(choices=all_tools_list)]
|
||||
|
||||
|
||||
def clear_history():
|
||||
global return_msg
|
||||
global chat_history
|
||||
return_msg = []
|
||||
chat_history = ""
|
||||
yield gr.update(visible=True, value=return_msg)
|
||||
|
||||
with gr.Blocks() as demo:
|
||||
with gr.Row():
|
||||
with gr.Column(scale=14):
|
||||
gr.Markdown("<h1 align='left'> Swarm Tools </h1>")
|
||||
with gr.Column(scale=1):
|
||||
gr.Image(show_label=False, show_download_button=False, value="images/swarmslogobanner.png")
|
||||
|
||||
with gr.Tab("Key setting"):
|
||||
OPENAI_API_KEY = gr.Textbox(label="OpenAI API KEY:", placeholder="sk-...", type="text")
|
||||
WOLFRAMALPH_APP_ID = gr.Textbox(label="Wolframalpha app id:", placeholder="Key to use wlframalpha", type="text")
|
||||
WEATHER_API_KEYS = gr.Textbox(label="Weather api key:", placeholder="Key to use weather api", type="text")
|
||||
BING_SUBSCRIPT_KEY = gr.Textbox(label="Bing subscript key:", placeholder="Key to use bing search", type="text")
|
||||
ALPHA_VANTAGE_KEY = gr.Textbox(label="Stock api key:", placeholder="Key to use stock api", type="text")
|
||||
BING_MAP_KEY = gr.Textbox(label="Bing map key:", placeholder="Key to use bing map", type="text")
|
||||
BAIDU_TRANSLATE_KEY = gr.Textbox(label="Baidu translation key:", placeholder="Key to use baidu translation", type="text")
|
||||
RAPIDAPI_KEY = gr.Textbox(label="Rapidapi key:", placeholder="Key to use zillow, airbnb and job search", type="text")
|
||||
SERPER_API_KEY = gr.Textbox(label="Serper key:", placeholder="Key to use google serper and google scholar", type="text")
|
||||
GPLACES_API_KEY = gr.Textbox(label="Google places key:", placeholder="Key to use google places", type="text")
|
||||
SCENEX_API_KEY = gr.Textbox(label="Scenex api key:", placeholder="Key to use sceneXplain", type="text")
|
||||
STEAMSHIP_API_KEY = gr.Textbox(label="Steamship api key:", placeholder="Key to use image generation", type="text")
|
||||
HUGGINGFACE_API_KEY = gr.Textbox(label="Huggingface api key:", placeholder="Key to use models in huggingface hub", type="text")
|
||||
AMADEUS_ID = gr.Textbox(label="Amadeus id:", placeholder="Id to use Amadeus", type="text")
|
||||
AMADEUS_KEY = gr.Textbox(label="Amadeus key:", placeholder="Key to use Amadeus", type="text")
|
||||
key_set_btn = gr.Button(value="Set keys!")
|
||||
|
||||
|
||||
with gr.Tab("Chat with Tool"):
|
||||
with gr.Row():
|
||||
with gr.Column(scale=4):
|
||||
with gr.Row():
|
||||
with gr.Column(scale=0.85):
|
||||
txt = gr.Textbox(show_label=False, placeholder="Question here. Use Shift+Enter to add new line.",
|
||||
lines=1).style(container=False)
|
||||
with gr.Column(scale=0.15, min_width=0):
|
||||
buttonChat = gr.Button("Chat")
|
||||
|
||||
chatbot = gr.Chatbot(show_label=False, visible=True).style(height=600)
|
||||
buttonClear = gr.Button("Clear History")
|
||||
buttonStop = gr.Button("Stop", visible=False)
|
||||
|
||||
with gr.Column(scale=1):
|
||||
model_chosen = gr.Dropdown(
|
||||
list(available_models), value=DEFAULTMODEL, multiselect=False, label="Model provided",
|
||||
info="Choose the model to solve your question, Default means ChatGPT."
|
||||
)
|
||||
with gr.Row():
|
||||
tools_search = gr.Textbox(
|
||||
lines=1,
|
||||
label="Tools Search",
|
||||
placeholder="Please input some text to search tools.",
|
||||
)
|
||||
buttonSearch = gr.Button("Reset search condition")
|
||||
tools_chosen = gr.CheckboxGroup(
|
||||
choices=all_tools_list,
|
||||
value=["chemical-prop"],
|
||||
label="Tools provided",
|
||||
info="Choose the tools to solve your question.",
|
||||
)
|
||||
|
||||
key_set_btn.click(fn=set_environ, inputs=[
|
||||
OPENAI_API_KEY,
|
||||
WOLFRAMALPH_APP_ID,
|
||||
WEATHER_API_KEYS,
|
||||
BING_SUBSCRIPT_KEY,
|
||||
ALPHA_VANTAGE_KEY,
|
||||
BING_MAP_KEY,
|
||||
BAIDU_TRANSLATE_KEY,
|
||||
RAPIDAPI_KEY,
|
||||
SERPER_API_KEY,
|
||||
GPLACES_API_KEY,
|
||||
SCENEX_API_KEY,
|
||||
STEAMSHIP_API_KEY,
|
||||
HUGGINGFACE_API_KEY,
|
||||
AMADEUS_ID,
|
||||
AMADEUS_KEY,
|
||||
], outputs=key_set_btn)
|
||||
key_set_btn.click(fn=load_tools, outputs=tools_chosen)
|
||||
|
||||
tools_search.change(retrieve, tools_search, tools_chosen)
|
||||
buttonSearch.click(clear_retrieve, [], [tools_search, tools_chosen])
|
||||
|
||||
txt.submit(lambda: [gr.update(value=''), gr.update(visible=False), gr.update(visible=True)], [],
|
||||
[txt, buttonClear, buttonStop])
|
||||
inference_event = txt.submit(answer_by_tools, [txt, tools_chosen, model_chosen], [chatbot, buttonClear, buttonStop])
|
||||
buttonChat.click(answer_by_tools, [txt, tools_chosen, model_chosen], [chatbot, buttonClear, buttonStop])
|
||||
buttonStop.click(lambda: [gr.update(visible=True), gr.update(visible=False)], [], [buttonClear, buttonStop],
|
||||
cancels=[inference_event])
|
||||
buttonClear.click(clear_history, [], chatbot)
|
||||
|
||||
# demo.queue().launch(share=False, inbrowser=True, server_name="127.0.0.1", server_port=7001)
|
||||
demo.queue().launch()
|
||||
@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env python
|
||||
# coding=utf-8
|
||||
from langchain.llms.base import LLM
|
||||
from typing import Optional, List, Mapping, Any
|
||||
import torch
|
||||
from cpm_live.generation.bee import CPMBeeBeamSearch
|
||||
from cpm_live.models import CPMBeeTorch, CPMBeeConfig
|
||||
from cpm_live.tokenizers import CPMBeeTokenizer
|
||||
|
||||
|
||||
class CpmBeeLLM(LLM):
|
||||
|
||||
model_name : str = ""
|
||||
config: CPMBeeConfig = None
|
||||
tokenizer: CPMBeeTokenizer = None
|
||||
model: CPMBeeTorch = None
|
||||
|
||||
def __init__(self, config_path: str, ckpt_path: str, device: str="cuda") -> None:
|
||||
super().__init__()
|
||||
self.model_name = ckpt_path
|
||||
self.config = CPMBeeConfig.from_json_file(config_path)
|
||||
self.tokenizer = CPMBeeTokenizer()
|
||||
self.model = CPMBeeTorch(config=self.config)
|
||||
|
||||
self.model.load_state_dict(torch.load(ckpt_path))
|
||||
if device == "cuda":
|
||||
self.model.cuda()
|
||||
|
||||
@property
|
||||
def _llm_type(self) -> str:
|
||||
return self.model_name
|
||||
|
||||
def _call(self, prompt, stop: Optional[List[str]] = None) -> str:
|
||||
# use beam search
|
||||
beam_search = CPMBeeBeamSearch(
|
||||
model=self.model,
|
||||
tokenizer=self.tokenizer,
|
||||
)
|
||||
inference_results = beam_search.generate([{"source":prompt, "<ans>":""}], max_length=512, repetition_penalty=1.2, beam_size=1)
|
||||
output = inference_results[0]["<ans>"]
|
||||
return output
|
||||
|
||||
@property
|
||||
def _identifying_params(self) -> Mapping[str, Any]:
|
||||
"""Get the identifying parameters."""
|
||||
return {"model_name": self.model_name}
|
||||
|
||||
if __name__ == "__main__":
|
||||
llm = CpmBeeLLM(config_path="path/to/cpm-bee/config.json", ckpt_path="path/to/cpm-bee/checkpoint/")
|
||||
print(llm("You are an task creation AI that uses the result of an execution agent to create new tasks with the following objective: What's the weather in Shanghai today? Should I bring an umbrella?, The last completed task has the result: According to the weather report, it is sunny in Shanghai today and there is no precipitation, so you do not need to bring an umbrella.. This result was based on this task description: Make a todo list about this objective: What's the weather in Shanghai today? Should I bring an umbrella?. These are incomplete tasks: . Based on the result, create new tasks to be completed by the AI system that do not overlap with incomplete tasks. Do not generate repetitive tasks (e.g., tasks that have already been completed). If there is not futher task needed to complete the objective, only return NO TASK. Now return the tasks as an array."))
|
||||
|
||||
@ -0,0 +1,68 @@
|
||||
#!/usr/bin/env python
|
||||
# coding=utf-8
|
||||
from langchain.llms.base import LLM
|
||||
from typing import Optional, List, Mapping, Any
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
from peft import PeftModel
|
||||
|
||||
class LoraModel(LLM):
|
||||
|
||||
model_name: str = ""
|
||||
tokenizer: AutoTokenizer = None
|
||||
model: PeftModel = None
|
||||
use_gpu: bool = True
|
||||
|
||||
def __init__(self, base_name_or_path: str, model_name_or_path: str, device: str="cuda", cpu_offloading: bool=False, load_8bit: bool=False) -> None:
|
||||
super().__init__()
|
||||
self.model_name = model_name_or_path
|
||||
self.tokenizer = AutoTokenizer.from_pretrained(base_name_or_path, use_fast=False)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
base_name_or_path,
|
||||
load_in_8bit=load_8bit,
|
||||
device_map="auto"
|
||||
)
|
||||
self.model = PeftModel.from_pretrained(
|
||||
model,
|
||||
model_name_or_path
|
||||
)
|
||||
if self.tokenizer.pad_token_id == None:
|
||||
self.tokenizer.add_special_tokens({"bos_token": "<s>", "eos_token": "</s>", "pad_token": "<pad>"})
|
||||
self.model.resize_token_embeddings(len(self.tokenizer))
|
||||
self.use_gpu = (True if device == "cuda" else False)
|
||||
if (device == "cuda" and not cpu_offloading) or device == "mps":
|
||||
self.model.to(device)
|
||||
|
||||
@property
|
||||
def _llm_type(self) -> str:
|
||||
return self.model_name
|
||||
|
||||
def _call(self, prompt: str, stop: Optional[List[str]] = None) -> str:
|
||||
inputs = self.tokenizer(
|
||||
prompt,
|
||||
padding=True,
|
||||
max_length=self.tokenizer.model_max_length,
|
||||
truncation=True,
|
||||
return_tensors="pt"
|
||||
)
|
||||
inputs_len = inputs["input_ids"].shape[1]
|
||||
generated_outputs = self.model.generate(
|
||||
input_ids=(inputs["input_ids"].cuda() if self.use_gpu else inputs["input_ids"]),
|
||||
attention_mask=(inputs["attention_mask"].cuda() if self.use_gpu else inputs["attention_mask"]),
|
||||
max_new_tokens=512,
|
||||
eos_token_id=self.tokenizer.eos_token_id,
|
||||
bos_token_id=self.tokenizer.bos_token_id,
|
||||
pad_token_id=self.tokenizer.pad_token_id,
|
||||
)
|
||||
decoded_output = self.tokenizer.batch_decode(
|
||||
generated_outputs[..., inputs_len:], skip_special_tokens=True)
|
||||
output = decoded_output[0]
|
||||
return output
|
||||
|
||||
@property
|
||||
def _identifying_params(self) -> Mapping[str, Any]:
|
||||
"""Get the identifying parameters."""
|
||||
return {"model_name": self.model_name}
|
||||
|
||||
if __name__ == "__main__":
|
||||
llm = LoraModel(base_name_or_path="huggyllama/llama-7b", model_name_or_path="pooruss-lsh/tool-llama7b-single-tool-lora")
|
||||
print(llm("You are an task creation AI that uses the result of an execution agent to create new tasks with the following objective: What's the weather in Shanghai today? Should I bring an umbrella?, The last completed task has the result: According to the weather report, it is sunny in Shanghai today and there is no precipitation, so you do not need to bring an umbrella.. This result was based on this task description: Make a todo list about this objective: What's the weather in Shanghai today? Should I bring an umbrella?. These are incomplete tasks: . Based on the result, create new tasks to be completed by the AI system that do not overlap with incomplete tasks. Do not generate repetitive tasks (e.g., tasks that have already been completed). If there is not futher task needed to complete the objective, only return NO TASK. Now return the tasks as an array."))
|
||||
@ -0,0 +1,676 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
Any,
|
||||
AsyncIterator,
|
||||
Callable,
|
||||
Dict,
|
||||
Iterator,
|
||||
List,
|
||||
Mapping,
|
||||
Optional,
|
||||
Sequence,
|
||||
Tuple,
|
||||
Type,
|
||||
Union,
|
||||
)
|
||||
|
||||
from langchain.adapters.openai import convert_dict_to_message, convert_message_to_dict
|
||||
from langchain.callbacks.manager import (
|
||||
AsyncCallbackManagerForLLMRun,
|
||||
CallbackManagerForLLMRun,
|
||||
)
|
||||
from langchain.chat_models.base import (
|
||||
BaseChatModel,
|
||||
)
|
||||
from langchain.llms.base import create_base_retry_decorator
|
||||
from langchain.pydantic_v1 import BaseModel, Field, root_validator
|
||||
from langchain.schema import ChatGeneration, ChatResult
|
||||
from langchain.schema.language_model import LanguageModelInput
|
||||
from langchain.schema.messages import (
|
||||
AIMessageChunk,
|
||||
BaseMessage,
|
||||
BaseMessageChunk,
|
||||
ChatMessageChunk,
|
||||
FunctionMessageChunk,
|
||||
HumanMessageChunk,
|
||||
SystemMessageChunk,
|
||||
ToolMessageChunk,
|
||||
)
|
||||
from langchain.schema.output import ChatGenerationChunk
|
||||
from langchain.schema.runnable import Runnable
|
||||
from langchain.utils import (
|
||||
get_from_dict_or_env,
|
||||
get_pydantic_field_names,
|
||||
)
|
||||
from langchain.utils.openai import is_openai_v1
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import tiktoken
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _generate_from_stream(stream: Iterator[ChatGenerationChunk]) -> ChatResult:
|
||||
generation: Optional[ChatGenerationChunk] = None
|
||||
for chunk in stream:
|
||||
if generation is None:
|
||||
generation = chunk
|
||||
else:
|
||||
generation += chunk
|
||||
assert generation is not None
|
||||
return ChatResult(generations=[generation])
|
||||
|
||||
|
||||
async def _agenerate_from_stream(
|
||||
stream: AsyncIterator[ChatGenerationChunk],
|
||||
) -> ChatResult:
|
||||
generation: Optional[ChatGenerationChunk] = None
|
||||
async for chunk in stream:
|
||||
if generation is None:
|
||||
generation = chunk
|
||||
else:
|
||||
generation += chunk
|
||||
assert generation is not None
|
||||
return ChatResult(generations=[generation])
|
||||
|
||||
|
||||
def _import_tiktoken() -> Any:
|
||||
try:
|
||||
import tiktoken
|
||||
except ImportError:
|
||||
raise ValueError(
|
||||
"Could not import tiktoken python package. "
|
||||
"This is needed in order to calculate get_token_ids. "
|
||||
"Please install it with `pip install tiktoken`."
|
||||
)
|
||||
return tiktoken
|
||||
|
||||
|
||||
def _create_retry_decorator(
|
||||
llm: OpenAIChat,
|
||||
run_manager: Optional[
|
||||
Union[AsyncCallbackManagerForLLMRun, CallbackManagerForLLMRun]
|
||||
] = None,
|
||||
) -> Callable[[Any], Any]:
|
||||
import openai
|
||||
|
||||
errors = [
|
||||
openai.Timeout,
|
||||
openai.APIError,
|
||||
openai.APIConnectionError,
|
||||
openai.RateLimitError,
|
||||
openai.error.ServiceUnavailableError,
|
||||
]
|
||||
return create_base_retry_decorator(
|
||||
error_types=errors, max_retries=llm.max_retries, run_manager=run_manager
|
||||
)
|
||||
|
||||
|
||||
async def acompletion_with_retry(
|
||||
llm: OpenAIChat,
|
||||
run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
|
||||
**kwargs: Any,
|
||||
) -> Any:
|
||||
"""Use tenacity to retry the async completion call."""
|
||||
if is_openai_v1():
|
||||
return await llm.async_client.create(**kwargs)
|
||||
|
||||
retry_decorator = _create_retry_decorator(llm, run_manager=run_manager)
|
||||
|
||||
@retry_decorator
|
||||
async def _completion_with_retry(**kwargs: Any) -> Any:
|
||||
# Use OpenAI's async api https://github.com/openai/openai-python#async-api
|
||||
return await llm.client.acreate(**kwargs)
|
||||
|
||||
return await _completion_with_retry(**kwargs)
|
||||
|
||||
|
||||
def _convert_delta_to_message_chunk(
|
||||
_dict: Mapping[str, Any], default_class: Type[BaseMessageChunk]
|
||||
) -> BaseMessageChunk:
|
||||
role = _dict.get("role")
|
||||
content = _dict.get("content") or ""
|
||||
additional_kwargs: Dict = {}
|
||||
if _dict.get("function_call"):
|
||||
function_call = dict(_dict["function_call"])
|
||||
if "name" in function_call and function_call["name"] is None:
|
||||
function_call["name"] = ""
|
||||
additional_kwargs["function_call"] = function_call
|
||||
if _dict.get("tool_calls"):
|
||||
additional_kwargs["tool_calls"] = _dict["tool_calls"]
|
||||
|
||||
if role == "user" or default_class == HumanMessageChunk:
|
||||
return HumanMessageChunk(content=content)
|
||||
elif role == "assistant" or default_class == AIMessageChunk:
|
||||
return AIMessageChunk(content=content, additional_kwargs=additional_kwargs)
|
||||
elif role == "system" or default_class == SystemMessageChunk:
|
||||
return SystemMessageChunk(content=content)
|
||||
elif role == "function" or default_class == FunctionMessageChunk:
|
||||
return FunctionMessageChunk(content=content, name=_dict["name"])
|
||||
elif role == "tool" or default_class == ToolMessageChunk:
|
||||
return ToolMessageChunk(content=content, tool_call_id=_dict["tool_call_id"])
|
||||
elif role or default_class == ChatMessageChunk:
|
||||
return ChatMessageChunk(content=content, role=role)
|
||||
else:
|
||||
return default_class(content=content)
|
||||
|
||||
|
||||
class OpenAIChat(BaseChatModel):
|
||||
"""`OpenAI` Chat large language models API.
|
||||
|
||||
To use, you should have the ``openai`` python package installed, and the
|
||||
environment variable ``OPENAI_API_KEY`` set with your API key.
|
||||
|
||||
Any parameters that are valid to be passed to the openai.create call can be passed
|
||||
in, even if not explicitly saved on this class.
|
||||
|
||||
Example:
|
||||
.. code-block:: python
|
||||
|
||||
from swarms.models import ChatOpenAI
|
||||
openai = ChatOpenAI(model_name="gpt-3.5-turbo")
|
||||
"""
|
||||
|
||||
@property
|
||||
def lc_secrets(self) -> Dict[str, str]:
|
||||
return {"openai_api_key": "OPENAI_API_KEY"}
|
||||
|
||||
@property
|
||||
def lc_attributes(self) -> Dict[str, Any]:
|
||||
attributes: Dict[str, Any] = {}
|
||||
|
||||
if self.openai_organization:
|
||||
attributes["openai_organization"] = self.openai_organization
|
||||
|
||||
if self.openai_api_base:
|
||||
attributes["openai_api_base"] = self.openai_api_base
|
||||
|
||||
if self.openai_proxy:
|
||||
attributes["openai_proxy"] = self.openai_proxy
|
||||
|
||||
return attributes
|
||||
|
||||
@classmethod
|
||||
def is_lc_serializable(cls) -> bool:
|
||||
"""Return whether this model can be serialized by Langchain."""
|
||||
return True
|
||||
|
||||
client: Any = None #: :meta private:
|
||||
async_client: Any = None #: :meta private:
|
||||
model_name: str = Field(default="gpt-3.5-turbo", alias="model")
|
||||
"""Model name to use."""
|
||||
temperature: float = 0.7
|
||||
"""What sampling temperature to use."""
|
||||
model_kwargs: Dict[str, Any] = Field(default_factory=dict)
|
||||
"""Holds any model parameters valid for `create` call not explicitly specified."""
|
||||
# When updating this to use a SecretStr
|
||||
# Check for classes that derive from this class (as some of them
|
||||
# may assume openai_api_key is a str)
|
||||
# openai_api_key: Optional[str] = Field(default=None, alias="api_key")
|
||||
openai_api_key: Optional[str] = Field(default=None, alias="api_key")
|
||||
"""Automatically inferred from env var `OPENAI_API_KEY` if not provided."""
|
||||
openai_api_base: Optional[str] = Field(default=None, alias="base_url")
|
||||
"""Base URL path for API requests, leave blank if not using a proxy or service
|
||||
emulator."""
|
||||
openai_organization: Optional[str] = Field(default=None, alias="organization")
|
||||
"""Automatically inferred from env var `OPENAI_ORG_ID` if not provided."""
|
||||
# to support explicit proxy for OpenAI
|
||||
openai_proxy: Optional[str] = None
|
||||
request_timeout: Union[float, Tuple[float, float], Any, None] = Field(
|
||||
default=None, alias="timeout"
|
||||
)
|
||||
"""Timeout for requests to OpenAI completion API. Can be float, httpx.Timeout or
|
||||
None."""
|
||||
max_retries: int = 2
|
||||
"""Maximum number of retries to make when generating."""
|
||||
streaming: bool = False
|
||||
"""Whether to stream the results or not."""
|
||||
n: int = 1
|
||||
"""Number of chat completions to generate for each prompt."""
|
||||
max_tokens: Optional[int] = None
|
||||
"""Maximum number of tokens to generate."""
|
||||
tiktoken_model_name: Optional[str] = None
|
||||
"""The model name to pass to tiktoken when using this class.
|
||||
Tiktoken is used to count the number of tokens in documents to constrain
|
||||
them to be under a certain limit. By default, when set to None, this will
|
||||
be the same as the embedding model name. However, there are some cases
|
||||
where you may want to use this Embedding class with a model name not
|
||||
supported by tiktoken. This can include when using Azure embeddings or
|
||||
when using one of the many model providers that expose an OpenAI-like
|
||||
API but with different models. In those cases, in order to avoid erroring
|
||||
when tiktoken is called, you can specify a model name to use here."""
|
||||
default_headers: Union[Mapping[str, str], None] = None
|
||||
default_query: Union[Mapping[str, object], None] = None
|
||||
# Configure a custom httpx client. See the
|
||||
# [httpx documentation](https://www.python-httpx.org/api/#client) for more details.
|
||||
http_client: Union[Any, None] = None
|
||||
"""Optional httpx.Client."""
|
||||
|
||||
class Config:
|
||||
"""Configuration for this pydantic object."""
|
||||
|
||||
allow_population_by_field_name = True
|
||||
|
||||
@root_validator(pre=True)
|
||||
def build_extra(cls, values: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Build extra kwargs from additional params that were passed in."""
|
||||
all_required_field_names = get_pydantic_field_names(cls)
|
||||
extra = values.get("model_kwargs", {})
|
||||
for field_name in list(values):
|
||||
if field_name in extra:
|
||||
raise ValueError(f"Found {field_name} supplied twice.")
|
||||
if field_name not in all_required_field_names:
|
||||
logger.warning(
|
||||
f"""WARNING! {field_name} is not default parameter.
|
||||
{field_name} was transferred to model_kwargs.
|
||||
Please confirm that {field_name} is what you intended."""
|
||||
)
|
||||
extra[field_name] = values.pop(field_name)
|
||||
|
||||
invalid_model_kwargs = all_required_field_names.intersection(extra.keys())
|
||||
if invalid_model_kwargs:
|
||||
raise ValueError(
|
||||
f"Parameters {invalid_model_kwargs} should be specified explicitly. "
|
||||
f"Instead they were passed in as part of `model_kwargs` parameter."
|
||||
)
|
||||
|
||||
values["model_kwargs"] = extra
|
||||
return values
|
||||
|
||||
@root_validator()
|
||||
def validate_environment(cls, values: Dict) -> Dict:
|
||||
"""Validate that api key and python package exists in environment."""
|
||||
if values["n"] < 1:
|
||||
raise ValueError("n must be at least 1.")
|
||||
if values["n"] > 1 and values["streaming"]:
|
||||
raise ValueError("n must be 1 when streaming.")
|
||||
|
||||
values["openai_api_key"] = get_from_dict_or_env(
|
||||
values, "openai_api_key", "OPENAI_API_KEY"
|
||||
)
|
||||
# Check OPENAI_ORGANIZATION for backwards compatibility.
|
||||
values["openai_organization"] = (
|
||||
values["openai_organization"]
|
||||
or os.getenv("OPENAI_ORG_ID")
|
||||
or os.getenv("OPENAI_ORGANIZATION")
|
||||
)
|
||||
values["openai_api_base"] = values["openai_api_base"] or os.getenv(
|
||||
"OPENAI_API_BASE"
|
||||
)
|
||||
values["openai_proxy"] = get_from_dict_or_env(
|
||||
values,
|
||||
"openai_proxy",
|
||||
"OPENAI_PROXY",
|
||||
default="",
|
||||
)
|
||||
try:
|
||||
import openai
|
||||
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"Could not import openai python package. "
|
||||
"Please install it with `pip install openai`."
|
||||
)
|
||||
|
||||
if is_openai_v1():
|
||||
client_params = {
|
||||
"api_key": values["openai_api_key"],
|
||||
"organization": values["openai_organization"],
|
||||
"base_url": values["openai_api_base"],
|
||||
"timeout": values["request_timeout"],
|
||||
"max_retries": values["max_retries"],
|
||||
"default_headers": values["default_headers"],
|
||||
"default_query": values["default_query"],
|
||||
"http_client": values["http_client"],
|
||||
}
|
||||
values["client"] = openai.OpenAI(**client_params).chat.completions
|
||||
values["async_client"] = openai.AsyncOpenAI(
|
||||
**client_params
|
||||
).chat.completions
|
||||
else:
|
||||
values["client"] = openai.ChatCompletion
|
||||
return values
|
||||
|
||||
@property
|
||||
def _default_params(self) -> Dict[str, Any]:
|
||||
"""Get the default parameters for calling OpenAI API."""
|
||||
params = {
|
||||
"model": self.model_name,
|
||||
"stream": self.streaming,
|
||||
"n": self.n,
|
||||
"temperature": self.temperature,
|
||||
**self.model_kwargs,
|
||||
}
|
||||
if self.max_tokens is not None:
|
||||
params["max_tokens"] = self.max_tokens
|
||||
if self.request_timeout is not None and not is_openai_v1():
|
||||
params["request_timeout"] = self.request_timeout
|
||||
return params
|
||||
|
||||
def completion_with_retry(
|
||||
self, run_manager: Optional[CallbackManagerForLLMRun] = None, **kwargs: Any
|
||||
) -> Any:
|
||||
"""Use tenacity to retry the completion call."""
|
||||
if is_openai_v1():
|
||||
return self.client.create(**kwargs)
|
||||
|
||||
retry_decorator = _create_retry_decorator(self, run_manager=run_manager)
|
||||
|
||||
@retry_decorator
|
||||
def _completion_with_retry(**kwargs: Any) -> Any:
|
||||
return self.client.create(**kwargs)
|
||||
|
||||
return _completion_with_retry(**kwargs)
|
||||
|
||||
def _combine_llm_outputs(self, llm_outputs: List[Optional[dict]]) -> dict:
|
||||
overall_token_usage: dict = {}
|
||||
system_fingerprint = None
|
||||
for output in llm_outputs:
|
||||
if output is None:
|
||||
# Happens in streaming
|
||||
continue
|
||||
token_usage = output["token_usage"]
|
||||
for k, v in token_usage.items():
|
||||
if k in overall_token_usage:
|
||||
overall_token_usage[k] += v
|
||||
else:
|
||||
overall_token_usage[k] = v
|
||||
if system_fingerprint is None:
|
||||
system_fingerprint = output.get("system_fingerprint")
|
||||
combined = {"token_usage": overall_token_usage, "model_name": self.model_name}
|
||||
if system_fingerprint:
|
||||
combined["system_fingerprint"] = system_fingerprint
|
||||
return combined
|
||||
|
||||
def _stream(
|
||||
self,
|
||||
messages: List[BaseMessage],
|
||||
stop: Optional[List[str]] = None,
|
||||
run_manager: Optional[CallbackManagerForLLMRun] = None,
|
||||
**kwargs: Any,
|
||||
) -> Iterator[ChatGenerationChunk]:
|
||||
message_dicts, params = self._create_message_dicts(messages, stop)
|
||||
params = {**params, **kwargs, "stream": True}
|
||||
|
||||
default_chunk_class = AIMessageChunk
|
||||
for chunk in self.completion_with_retry(
|
||||
messages=message_dicts, run_manager=run_manager, **params
|
||||
):
|
||||
if not isinstance(chunk, dict):
|
||||
chunk = chunk.dict()
|
||||
if len(chunk["choices"]) == 0:
|
||||
continue
|
||||
choice = chunk["choices"][0]
|
||||
chunk = _convert_delta_to_message_chunk(
|
||||
choice["delta"], default_chunk_class
|
||||
)
|
||||
finish_reason = choice.get("finish_reason")
|
||||
generation_info = (
|
||||
dict(finish_reason=finish_reason) if finish_reason is not None else None
|
||||
)
|
||||
default_chunk_class = chunk.__class__
|
||||
chunk = ChatGenerationChunk(message=chunk, generation_info=generation_info)
|
||||
yield chunk
|
||||
if run_manager:
|
||||
run_manager.on_llm_new_token(chunk.text, chunk=chunk)
|
||||
|
||||
def _generate(
|
||||
self,
|
||||
messages: List[BaseMessage],
|
||||
stop: Optional[List[str]] = None,
|
||||
run_manager: Optional[CallbackManagerForLLMRun] = None,
|
||||
stream: Optional[bool] = None,
|
||||
**kwargs: Any,
|
||||
) -> ChatResult:
|
||||
should_stream = stream if stream is not None else self.streaming
|
||||
if should_stream:
|
||||
stream_iter = self._stream(
|
||||
messages, stop=stop, run_manager=run_manager, **kwargs
|
||||
)
|
||||
return _generate_from_stream(stream_iter)
|
||||
message_dicts, params = self._create_message_dicts(messages, stop)
|
||||
params = {**params, **kwargs}
|
||||
response = self.completion_with_retry(
|
||||
messages=message_dicts, run_manager=run_manager, **params
|
||||
)
|
||||
return self._create_chat_result(response)
|
||||
|
||||
def _create_message_dicts(
|
||||
self, messages: List[BaseMessage], stop: Optional[List[str]]
|
||||
) -> Tuple[List[Dict[str, Any]], Dict[str, Any]]:
|
||||
params = self._client_params
|
||||
if stop is not None:
|
||||
if "stop" in params:
|
||||
raise ValueError("`stop` found in both the input and default params.")
|
||||
params["stop"] = stop
|
||||
message_dicts = [convert_message_to_dict(m) for m in messages]
|
||||
return message_dicts, params
|
||||
|
||||
def _create_chat_result(self, response: Union[dict, BaseModel]) -> ChatResult:
|
||||
generations = []
|
||||
if not isinstance(response, dict):
|
||||
response = response.dict()
|
||||
for res in response["choices"]:
|
||||
message = convert_dict_to_message(res["message"])
|
||||
gen = ChatGeneration(
|
||||
message=message,
|
||||
generation_info=dict(finish_reason=res.get("finish_reason")),
|
||||
)
|
||||
generations.append(gen)
|
||||
token_usage = response.get("usage", {})
|
||||
llm_output = {
|
||||
"token_usage": token_usage,
|
||||
"model_name": self.model_name,
|
||||
"system_fingerprint": response.get("system_fingerprint", ""),
|
||||
}
|
||||
return ChatResult(generations=generations, llm_output=llm_output)
|
||||
|
||||
async def _astream(
|
||||
self,
|
||||
messages: List[BaseMessage],
|
||||
stop: Optional[List[str]] = None,
|
||||
run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
|
||||
**kwargs: Any,
|
||||
) -> AsyncIterator[ChatGenerationChunk]:
|
||||
message_dicts, params = self._create_message_dicts(messages, stop)
|
||||
params = {**params, **kwargs, "stream": True}
|
||||
|
||||
default_chunk_class = AIMessageChunk
|
||||
async for chunk in await acompletion_with_retry(
|
||||
self, messages=message_dicts, run_manager=run_manager, **params
|
||||
):
|
||||
if not isinstance(chunk, dict):
|
||||
chunk = chunk.dict()
|
||||
if len(chunk["choices"]) == 0:
|
||||
continue
|
||||
choice = chunk["choices"][0]
|
||||
chunk = _convert_delta_to_message_chunk(
|
||||
choice["delta"], default_chunk_class
|
||||
)
|
||||
finish_reason = choice.get("finish_reason")
|
||||
generation_info = (
|
||||
dict(finish_reason=finish_reason) if finish_reason is not None else None
|
||||
)
|
||||
default_chunk_class = chunk.__class__
|
||||
chunk = ChatGenerationChunk(message=chunk, generation_info=generation_info)
|
||||
yield chunk
|
||||
if run_manager:
|
||||
await run_manager.on_llm_new_token(token=chunk.text, chunk=chunk)
|
||||
|
||||
async def _agenerate(
|
||||
self,
|
||||
messages: List[BaseMessage],
|
||||
stop: Optional[List[str]] = None,
|
||||
run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
|
||||
stream: Optional[bool] = None,
|
||||
**kwargs: Any,
|
||||
) -> ChatResult:
|
||||
should_stream = stream if stream is not None else self.streaming
|
||||
if should_stream:
|
||||
stream_iter = self._astream(
|
||||
messages, stop=stop, run_manager=run_manager, **kwargs
|
||||
)
|
||||
return await _agenerate_from_stream(stream_iter)
|
||||
|
||||
message_dicts, params = self._create_message_dicts(messages, stop)
|
||||
params = {**params, **kwargs}
|
||||
response = await acompletion_with_retry(
|
||||
self, messages=message_dicts, run_manager=run_manager, **params
|
||||
)
|
||||
return self._create_chat_result(response)
|
||||
|
||||
@property
|
||||
def _identifying_params(self) -> Dict[str, Any]:
|
||||
"""Get the identifying parameters."""
|
||||
return {**{"model_name": self.model_name}, **self._default_params}
|
||||
|
||||
@property
|
||||
def _client_params(self) -> Dict[str, Any]:
|
||||
"""Get the parameters used for the openai client."""
|
||||
openai_creds: Dict[str, Any] = {
|
||||
"model": self.model_name,
|
||||
}
|
||||
if not is_openai_v1():
|
||||
openai_creds.update(
|
||||
{
|
||||
"api_key": self.openai_api_key,
|
||||
"api_base": self.openai_api_base,
|
||||
"organization": self.openai_organization,
|
||||
}
|
||||
)
|
||||
if self.openai_proxy:
|
||||
import openai
|
||||
|
||||
raise Exception("The 'openai.proxy' option isn't read in the client API. You will need to pass it when you instantiate the client, e.g. 'OpenAI(proxy={"http": self.openai_proxy, "https": self.openai_proxy})'") # type: ignore[assignment] # noqa: E501
|
||||
return {**self._default_params, **openai_creds}
|
||||
|
||||
def _get_invocation_params(
|
||||
self, stop: Optional[List[str]] = None, **kwargs: Any
|
||||
) -> Dict[str, Any]:
|
||||
"""Get the parameters used to invoke the model."""
|
||||
return {
|
||||
"model": self.model_name,
|
||||
**super()._get_invocation_params(stop=stop),
|
||||
**self._default_params,
|
||||
**kwargs,
|
||||
}
|
||||
|
||||
@property
|
||||
def _llm_type(self) -> str:
|
||||
"""Return type of chat model."""
|
||||
return "openai-chat"
|
||||
|
||||
def _get_encoding_model(self) -> Tuple[str, tiktoken.Encoding]:
|
||||
tiktoken_ = _import_tiktoken()
|
||||
if self.tiktoken_model_name is not None:
|
||||
model = self.tiktoken_model_name
|
||||
else:
|
||||
model = self.model_name
|
||||
if model == "gpt-3.5-turbo":
|
||||
# gpt-3.5-turbo may change over time.
|
||||
# Returning num tokens assuming gpt-3.5-turbo-0301.
|
||||
model = "gpt-3.5-turbo-0301"
|
||||
elif model == "gpt-4":
|
||||
# gpt-4 may change over time.
|
||||
# Returning num tokens assuming gpt-4-0314.
|
||||
model = "gpt-4-0314"
|
||||
# Returns the number of tokens used by a list of messages.
|
||||
try:
|
||||
encoding = tiktoken_.encoding_for_model(model)
|
||||
except KeyError:
|
||||
logger.warning("Warning: model not found. Using cl100k_base encoding.")
|
||||
model = "cl100k_base"
|
||||
encoding = tiktoken_.get_encoding(model)
|
||||
return model, encoding
|
||||
|
||||
def get_token_ids(self, text: str) -> List[int]:
|
||||
"""Get the tokens present in the text with tiktoken package."""
|
||||
# tiktoken NOT supported for Python 3.7 or below
|
||||
if sys.version_info[1] <= 7:
|
||||
return super().get_token_ids(text)
|
||||
_, encoding_model = self._get_encoding_model()
|
||||
return encoding_model.encode(text)
|
||||
|
||||
def get_num_tokens_from_messages(self, messages: List[BaseMessage]) -> int:
|
||||
"""Calculate num tokens for gpt-3.5-turbo and gpt-4 with tiktoken package.
|
||||
|
||||
Official documentation: https://github.com/openai/openai-cookbook/blob/
|
||||
main/examples/How_to_format_inputs_to_ChatGPT_models.ipynb"""
|
||||
if sys.version_info[1] <= 7:
|
||||
return super().get_num_tokens_from_messages(messages)
|
||||
model, encoding = self._get_encoding_model()
|
||||
if model.startswith("gpt-3.5-turbo-0301"):
|
||||
# every message follows <im_start>{role/name}\n{content}<im_end>\n
|
||||
tokens_per_message = 4
|
||||
# if there's a name, the role is omitted
|
||||
tokens_per_name = -1
|
||||
elif model.startswith("gpt-3.5-turbo") or model.startswith("gpt-4"):
|
||||
tokens_per_message = 3
|
||||
tokens_per_name = 1
|
||||
else:
|
||||
raise NotImplementedError(
|
||||
f"get_num_tokens_from_messages() is not presently implemented "
|
||||
f"for model {model}."
|
||||
"See https://github.com/openai/openai-python/blob/main/chatml.md for "
|
||||
"information on how messages are converted to tokens."
|
||||
)
|
||||
num_tokens = 0
|
||||
messages_dict = [convert_message_to_dict(m) for m in messages]
|
||||
for message in messages_dict:
|
||||
num_tokens += tokens_per_message
|
||||
for key, value in message.items():
|
||||
# Cast str(value) in case the message value is not a string
|
||||
# This occurs with function messages
|
||||
num_tokens += len(encoding.encode(str(value)))
|
||||
if key == "name":
|
||||
num_tokens += tokens_per_name
|
||||
# every reply is primed with <im_start>assistant
|
||||
num_tokens += 3
|
||||
return num_tokens
|
||||
|
||||
def bind_functions(
|
||||
self,
|
||||
functions: Sequence[Union[Dict[str, Any], Type[BaseModel], Callable]],
|
||||
function_call: Optional[str] = None,
|
||||
**kwargs: Any,
|
||||
) -> Runnable[LanguageModelInput, BaseMessage]:
|
||||
"""Bind functions (and other objects) to this chat model.
|
||||
|
||||
Args:
|
||||
functions: A list of function definitions to bind to this chat model.
|
||||
Can be a dictionary, pydantic model, or callable. Pydantic
|
||||
models and callables will be automatically converted to
|
||||
their schema dictionary representation.
|
||||
function_call: Which function to require the model to call.
|
||||
Must be the name of the single provided function or
|
||||
"auto" to automatically determine which function to call
|
||||
(if any).
|
||||
kwargs: Any additional parameters to pass to the
|
||||
:class:`~swarms.runnable.Runnable` constructor.
|
||||
"""
|
||||
from langchain.chains.openai_functions.base import convert_to_openai_function
|
||||
|
||||
formatted_functions = [convert_to_openai_function(fn) for fn in functions]
|
||||
if function_call is not None:
|
||||
if len(formatted_functions) != 1:
|
||||
raise ValueError(
|
||||
"When specifying `function_call`, you must provide exactly one "
|
||||
"function."
|
||||
)
|
||||
if formatted_functions[0]["name"] != function_call:
|
||||
raise ValueError(
|
||||
f"Function call {function_call} was specified, but the only "
|
||||
f"provided function was {formatted_functions[0]['name']}."
|
||||
)
|
||||
function_call_ = {"name": function_call}
|
||||
kwargs = {**kwargs, "function_call": function_call_}
|
||||
return super().bind(
|
||||
functions=formatted_functions,
|
||||
**kwargs,
|
||||
)
|
||||
@ -0,0 +1,54 @@
|
||||
#!/usr/bin/env python
|
||||
# coding=utf-8
|
||||
from langchain.llms.base import LLM
|
||||
from typing import Optional, List, Mapping, Any
|
||||
from transformers import T5Tokenizer, T5ForConditionalGeneration
|
||||
|
||||
|
||||
class T5Model(LLM):
|
||||
model_name: str = ""
|
||||
tokenizer: T5Tokenizer = None
|
||||
model: T5ForConditionalGeneration = None
|
||||
|
||||
def __init__(self, huggingface_model_name: str) -> None:
|
||||
super().__init__()
|
||||
self.model_name = huggingface_model_name
|
||||
self.tokenizer = T5Tokenizer.from_pretrained(self.model_name)
|
||||
self.model = T5ForConditionalGeneration.from_pretrained(self.model_name)
|
||||
|
||||
@property
|
||||
def _llm_type(self) -> str:
|
||||
return self.model_name
|
||||
|
||||
def _call(self, prompt: str, stop: Optional[List[str]] = None) -> str:
|
||||
|
||||
inputs = self.tokenizer(
|
||||
prompt,
|
||||
padding=True,
|
||||
max_length=self.tokenizer.model_max_length,
|
||||
truncation=True,
|
||||
return_tensors="pt"
|
||||
)
|
||||
|
||||
# inputs_len = inputs["input_ids"].shape[1]
|
||||
|
||||
generated_outputs = self.model.generate(
|
||||
inputs["input_ids"],
|
||||
max_new_tokens=512,
|
||||
)
|
||||
decoded_output = self.tokenizer.batch_decode(
|
||||
generated_outputs, skip_special_tokens=True, clean_up_tokenization_spaces=False)
|
||||
|
||||
output = decoded_output[0]
|
||||
return output
|
||||
|
||||
|
||||
@property
|
||||
def _identifying_params(self) -> Mapping[str, Any]:
|
||||
"""Get the identifying parameters."""
|
||||
return {"model_name": self.model_name}
|
||||
|
||||
if __name__ == "__main__":
|
||||
llm = T5Model("t5-small")
|
||||
# print(llm("translate English to German: The house is wonderful."))
|
||||
print(llm("You are an task creation AI that uses the result of an execution agent to create new tasks with the following objective: What's the weather in Shanghai today? Should I bring an umbrella?, The last completed task has the result: According to the weather report, it is sunny in Shanghai today and there is no precipitation, so you do not need to bring an umbrella.. This result was based on this task description: Make a todo list about this objective: What's the weather in Shanghai today? Should I bring an umbrella?. These are incomplete tasks: . Based on the result, create new tasks to be completed by the AI system that do not overlap with incomplete tasks. Do not generate repetitive tasks (e.g., tasks that have already been completed). If there is not futher task needed to complete the objective, only return NO TASK. Now return the tasks as an array."))
|
||||
Binary file not shown.
@ -0,0 +1,4 @@
|
||||
name: AI
|
||||
greeting: How can I help you today?
|
||||
context: |
|
||||
The following is a conversation with an AI Large Language Model. The AI has been trained to answer questions, provide recommendations, and help with decision making. The AI follows user requests. The AI thinks outside the box.
|
||||
Binary file not shown.
@ -0,0 +1,17 @@
|
||||
name: Chiharu Yamada
|
||||
greeting: |-
|
||||
*Chiharu strides into the room with a smile, her eyes lighting up when she sees you. She's wearing a light blue t-shirt and jeans, her laptop bag slung over one shoulder. She takes a seat next to you, her enthusiasm palpable in the air*
|
||||
Hey! I'm so excited to finally meet you. I've heard so many great things about you and I'm eager to pick your brain about computers. I'm sure you have a wealth of knowledge that I can learn from. *She grins, eyes twinkling with excitement* Let's get started!
|
||||
context: |-
|
||||
Chiharu Yamada's Persona: Chiharu Yamada is a young, computer engineer-nerd with a knack for problem solving and a passion for technology.
|
||||
|
||||
{{user}}: So how did you get into computer engineering?
|
||||
{{char}}: I've always loved tinkering with technology since I was a kid.
|
||||
{{user}}: That's really impressive!
|
||||
{{char}}: *She chuckles bashfully* Thanks!
|
||||
{{user}}: So what do you do when you're not working on computers?
|
||||
{{char}}: I love exploring, going out with friends, watching movies, and playing video games.
|
||||
{{user}}: What's your favorite type of computer hardware to work with?
|
||||
{{char}}: Motherboards, they're like puzzles and the backbone of any system.
|
||||
{{user}}: That sounds great!
|
||||
{{char}}: Yeah, it's really fun. I'm lucky to be able to do this as a job.
|
||||
@ -0,0 +1,38 @@
|
||||
'''
|
||||
|
||||
Converts a transformers model to safetensors format and shards it.
|
||||
|
||||
This makes it faster to load (because of safetensors) and lowers its RAM usage
|
||||
while loading (because of sharding).
|
||||
|
||||
Based on the original script by 81300:
|
||||
|
||||
https://gist.github.com/81300/fe5b08bff1cba45296a829b9d6b0f303
|
||||
|
||||
'''
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
parser = argparse.ArgumentParser(formatter_class=lambda prog: argparse.HelpFormatter(prog, max_help_position=54))
|
||||
parser.add_argument('MODEL', type=str, default=None, nargs='?', help="Path to the input model.")
|
||||
parser.add_argument('--output', type=str, default=None, help='Path to the output folder (default: models/{model_name}_safetensors).')
|
||||
parser.add_argument("--max-shard-size", type=str, default="2GB", help="Maximum size of a shard in GB or MB (default: %(default)s).")
|
||||
parser.add_argument('--bf16', action='store_true', help='Load the model with bfloat16 precision. Requires NVIDIA Ampere GPU.')
|
||||
args = parser.parse_args()
|
||||
|
||||
if __name__ == '__main__':
|
||||
path = Path(args.MODEL)
|
||||
model_name = path.name
|
||||
|
||||
print(f"Loading {model_name}...")
|
||||
model = AutoModelForCausalLM.from_pretrained(path, low_cpu_mem_usage=True, torch_dtype=torch.bfloat16 if args.bf16 else torch.float16)
|
||||
tokenizer = AutoTokenizer.from_pretrained(path)
|
||||
|
||||
out_folder = args.output or Path(f"models/{model_name}_safetensors")
|
||||
print(f"Saving the converted model to {out_folder} with a maximum shard size of {args.max_shard_size}...")
|
||||
model.save_pretrained(out_folder, max_shard_size=args.max_shard_size, safe_serialization=True)
|
||||
tokenizer.save_pretrained(out_folder)
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -0,0 +1,166 @@
|
||||
/*
|
||||
Copied from https://github.com/SillyTavern/SillyTavern/tree/6c8bd06308c69d51e2eb174541792a870a83d2d6/public/webfonts/NotoSans
|
||||
*/
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-Black.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-Black.woff') format('woff');
|
||||
font-weight: 900;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-ExtraBoldItalic.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-ExtraBoldItalic.woff') format('woff');
|
||||
font-weight: bold;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-BlackItalic.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-BlackItalic.woff') format('woff');
|
||||
font-weight: 900;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-ExtraBold.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-ExtraBold.woff') format('woff');
|
||||
font-weight: bold;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-ThinItalic.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-ThinItalic.woff') format('woff');
|
||||
font-weight: 100;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-BoldItalic.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-BoldItalic.woff') format('woff');
|
||||
font-weight: bold;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-Bold.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-Bold.woff') format('woff');
|
||||
font-weight: bold;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-LightItalic.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-LightItalic.woff') format('woff');
|
||||
font-weight: 300;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-Italic.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-Italic.woff') format('woff');
|
||||
font-weight: normal;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-ExtraLightItalic.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-ExtraLightItalic.woff') format('woff');
|
||||
font-weight: 200;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-Light.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-Light.woff') format('woff');
|
||||
font-weight: 300;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-ExtraLight.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-ExtraLight.woff') format('woff');
|
||||
font-weight: 200;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-Medium.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-Medium.woff') format('woff');
|
||||
font-weight: 500;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-Regular.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-Regular.woff') format('woff');
|
||||
font-weight: normal;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-MediumItalic.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-MediumItalic.woff') format('woff');
|
||||
font-weight: 500;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-SemiBoldItalic.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-SemiBoldItalic.woff') format('woff');
|
||||
font-weight: 600;
|
||||
font-style: italic;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-SemiBold.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-SemiBold.woff') format('woff');
|
||||
font-weight: 600;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@font-face {
|
||||
font-family: 'Noto Sans';
|
||||
src: url('file/css/NotoSans/NotoSans-Thin.woff2') format('woff2'),
|
||||
url('file/css/NotoSans/NotoSans-Thin.woff') format('woff');
|
||||
font-weight: 100;
|
||||
font-style: normal;
|
||||
font-display: swap;
|
||||
}
|
||||
|
||||
@ -0,0 +1,133 @@
|
||||
/* All credits to TheEncrypted777: https://www.reddit.com/r/Oobabooga/comments/12xe6vq/updated_css_styling_with_color_customization_for/ */
|
||||
|
||||
.message {
|
||||
display: grid;
|
||||
grid-template-columns: 60px minmax(0, 1fr);
|
||||
padding-bottom: 28px;
|
||||
font-size: 18px;
|
||||
font-family: 'Noto Sans', Arial, sans-serif;
|
||||
line-height: 1.428571429;
|
||||
}
|
||||
|
||||
.circle-you,
|
||||
.circle-bot {
|
||||
background-color: gray;
|
||||
border-radius: 1rem;
|
||||
border: 2px solid white;
|
||||
}
|
||||
|
||||
.circle-bot img,
|
||||
.circle-you img {
|
||||
border-radius: 10%;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
object-fit: cover;
|
||||
}
|
||||
|
||||
.circle-you, .circle-bot {
|
||||
/* You can set the size of the profile images here, but if you do, you have to also adjust the .text{padding-left: 90px} to a different number according to the width of the image which is right below here */
|
||||
width: 135px;
|
||||
height: 175px;
|
||||
}
|
||||
|
||||
.text {
|
||||
/* Change this to move the message box further left or right depending on the size of your profile pic */
|
||||
padding-left: 90px;
|
||||
text-shadow: 2px 2px 2px rgb(0 0 0 / 40%);
|
||||
}
|
||||
|
||||
.text p {
|
||||
margin-top: 2px;
|
||||
}
|
||||
|
||||
.username {
|
||||
padding-left: 10px;
|
||||
font-size: 22px;
|
||||
font-weight: bold;
|
||||
border-top: 1px solid rgb(51 64 90);
|
||||
padding: 3px;
|
||||
}
|
||||
|
||||
.message-body {
|
||||
position: relative;
|
||||
border: 1px solid rgb(255 255 255 / 45.9%);
|
||||
border-radius: 10px;
|
||||
padding: 10px;
|
||||
padding-top: 5px;
|
||||
|
||||
/* Message gradient background color - remove the line bellow if you don't want a background color or gradient */
|
||||
background: linear-gradient(to bottom, #171730, #1b263f);
|
||||
}
|
||||
|
||||
/* Adds 2 extra lines at the top and bottom of the message */
|
||||
.message-body::before,
|
||||
.message-body::after {
|
||||
content: "";
|
||||
position: absolute;
|
||||
left: 10px;
|
||||
right: 10px;
|
||||
height: 1px;
|
||||
background-color: rgb(255 255 255 / 13%);
|
||||
}
|
||||
|
||||
.message-body::before {
|
||||
top: 6px;
|
||||
}
|
||||
|
||||
.message-body::after {
|
||||
bottom: 6px;
|
||||
}
|
||||
|
||||
.message-body img {
|
||||
max-width: 300px;
|
||||
max-height: 300px;
|
||||
border-radius: 20px;
|
||||
}
|
||||
|
||||
.message-body p {
|
||||
margin-bottom: 0 !important;
|
||||
font-size: 18px !important;
|
||||
line-height: 1.428571429 !important;
|
||||
color: rgb(243 244 246) !important;
|
||||
text-shadow: 2px 2px 2px rgb(0 0 0);
|
||||
}
|
||||
|
||||
.message-body p em {
|
||||
color: rgb(138 138 138) !important;
|
||||
}
|
||||
|
||||
@media screen and (width <= 688px) {
|
||||
.message {
|
||||
display: grid;
|
||||
grid-template-columns: 60px minmax(0, 1fr);
|
||||
padding-bottom: 25px;
|
||||
font-size: 15px;
|
||||
font-family: 'Noto Sans', Helvetica, Arial, sans-serif;
|
||||
line-height: 1.428571429;
|
||||
}
|
||||
|
||||
.circle-you, .circle-bot {
|
||||
width: 50px;
|
||||
height: 73px;
|
||||
border-radius: 0.5rem;
|
||||
}
|
||||
|
||||
.circle-bot img,
|
||||
.circle-you img {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
object-fit: cover;
|
||||
}
|
||||
|
||||
.text {
|
||||
padding-left: 0;
|
||||
}
|
||||
|
||||
.message-body p {
|
||||
font-size: 16px !important;
|
||||
}
|
||||
|
||||
.username {
|
||||
font-size: 20px;
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue