ai-mirow-projects
/
swarms
mirror of https://github.com/kyegomez/swarms
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[REFACTOR][Agent]

Browse Source
pull/426/head
Kye 10 months ago
parent ab90a405c8
commit b3dc64cae6
14 changed files with 555 additions and 334 deletions
Show Stats Download Patch File Download Diff File

17
Transcript Generator_state.json
Unescape View File

@ -0,0 +1,17 @@
{
"agent_id": "<function agent_id at 0x16368c400>",
"agent_name": "Transcript Generator",
"agent_description": "Generate a transcript for a youtube video on what swarms are!",
"system_prompt": "\n You are a fully autonomous agent serving the user in automating tasks, workflows, and activities. \n Agent's use custom instructions, capabilities, and data to optimize LLMs for a more narrow set of tasks.\n \n You will have internal dialogues with yourself and or interact with the user to aid in these tasks. \n Your responses should be coherent, contextually relevant, and tailored to the task at hand.\n",
"sop": null,
"short_memory": "system: \n You are a fully autonomous agent serving the user in automating tasks, workflows, and activities. \n Agent's use custom instructions, capabilities, and data to optimize LLMs for a more narrow set of tasks.\n \n You will have internal dialogues with yourself and or interact with the user to aid in these tasks. \n Your responses should be coherent, contextually relevant, and tailored to the task at hand.\n\n\n\nHuman:: Generate a transcript for a youtube video on what swarms are!\n\n\nTranscript Generator: \nSwarms are composed of large numbers of independent individuals that collectively carry out complex behaviors. For example, an ant colony functions as a swarm - each ant follows simple rules but together the colony can build intricate nests and find food.\n\nIn artificial swarms, we try to emulate these naturally-occurring phenomena. By programming basic behaviors into agents and allowing them to interact, we can observe emergent group behaviors without centralized control. For example, groups of robots may be designed with attraction and repulsion forces to self-assemble or explore environments.\n\nSimilarly, swarms may allow optimization algorithms to explore solutions in parallel. Each program follows their own trajectory while sharing information to converge on the best result. High-level commands give a rough direction, but the specific behaviors emerge from the interactions at the local level. \n\nPotential applications of artificial swarms include self-configuring robot teams for search & rescue, intelligent routing of network packets, and distributed processing for enhanced efficiency. The decentralized nature of swarms provides robustness, scalability and adaptability surpassing individual agents. \n\nBy harnessing simple local rules and interactions, swarm systems transcend the capabilities of any single member. They provide distributed solutions to coordinate large numbers independent agents to achieve a collective purpose.\n\n\nTranscript Generator: \nSwarms are composed of large numbers of independent individuals that collectively carry out complex behaviors. For example, an ant colony functions as a swarm - each ant follows simple rules but together the colony can build intricate nests and find food.\n\nIn artificial swarms, we try to emulate these naturally-occurring phenomena. By programming basic behaviors into agents and allowing them to interact, we can observe emergent group behaviors without centralized control. For example, groups of robots may be designed with attraction and repulsion forces to self-assemble or explore environments.\n\nSimilarly, swarms may allow optimization algorithms to explore solutions in parallel. Each program follows their own trajectory while sharing information to converge on the best result. High-level commands give a rough direction, but the specific behaviors emerge from the interactions at the local level. \n\nPotential applications of artificial swarms include self-configuring robot teams for search & rescue, intelligent routing of network packets, and distributed processing for enhanced efficiency. The decentralized nature of swarms provides robustness, scalability and adaptability surpassing individual agents. \n\nBy harnessing simple local rules and interactions, swarm systems transcend the capabilities of any single member. They provide distributed solutions to coordinate large numbers independent agents to achieve a collective purpose.\n\n\nHuman:: what is your purpose\n\n",
"loop_interval": 1,
"retry_attempts": 3,
"retry_interval": 1,
"interactive": true,
"dashboard": false,
"dynamic_temperature": false,
"autosave": true,
"saved_state_path": "Transcript Generator_state.json",
"max_loops": 1
}

7
example.py
Unescape View File

@ -9,16 +9,17 @@ agent = Agent(
" are!"
),
llm=Anthropic(),
max_loops="auto",
max_loops=3,
autosave=True,
dashboard=False,
streaming_on=True,
verbose=True,
stopping_token="<DONE>",
interactive=True,
)
# Run the workflow on a task
agent(
out = agent(
"Generate a transcript for a youtube video on what swarms are!"
" Output a <DONE> token when done."
)
print(out)

8
pyproject.toml
Unescape View File

@ -5,7 +5,7 @@ build-backend = "poetry.core.masonry.api"
[tool.poetry]
name = "swarms"
version = "4.3.7"
version = "4.5.8"
description = "Swarms - Pytorch"
license = "MIT"
authors = ["Kye Gomez <kye@apac.ai>"]
@ -33,8 +33,7 @@ google-generativeai = "0.3.1"
langchain = "0.1.13"
langchain-core = "0.1.33"
langchain-community = "0.0.29"
langsmith = "0.1.17"
langchain-openai = "0.0.5"
langchain-experimental = "0.0.55"
faiss-cpu = "1.7.4"
backoff = "2.2.1"
datasets = "*"
@ -42,7 +41,7 @@ optimum = "1.15.0"
supervision = "0.19.0"
opencv-python = "4.9.0.80"
diffusers = "*"
anthropic = "0.2.5"
anthropic = "0.21.3"
toml = "*"
pypdf = "4.1.0"
accelerate = "*"
@ -64,7 +63,6 @@ sentence-transformers = "*"
peft = "*"
psutil = "*"
timm = "*"
supervision = "*"
sentry-sdk = "*"
[tool.poetry.dev-dependencies]

72
swarms/agents/tool_agent.py
Unescape View File

@ -2,6 +2,7 @@ from typing import Any, Optional, Callable
from swarms.structs.agent import Agent
from swarms.tools.format_tools import Jsonformer
from swarms.utils.loguru_logger import logger
class ToolAgent(Agent):
@ -68,13 +69,14 @@ class ToolAgent(Agent):
json_schema: Any = None,
max_number_tokens: int = 500,
parsing_function: Optional[Callable] = None,
llm: Any = None,
*args,
**kwargs,
):
super().__init__(
agent_name=name,
agent_description=description,
sop=f"{name} {description} {str(json_schema)}" * args,
llm=llm,
**kwargs,
)
self.name = name
@ -101,33 +103,51 @@ class ToolAgent(Agent):
Exception: If an error occurs during the execution of the tool agent.
"""
try:
self.toolagent = Jsonformer(
model=self.model,
tokenizer=self.tokenizer,
json_schema=self.json_schema,
prompt=task,
max_number_tokens=self.max_number_tokens,
*args,
**kwargs,
)
if self.model:
logger.info(f"Running {self.name} for task: {task}")
self.toolagent = Jsonformer(
model=self.model,
tokenizer=self.tokenizer,
json_schema=self.json_schema,
llm=self.llm,
prompt=task,
max_number_tokens=self.max_number_tokens,
*args,
**kwargs,
)
if self.parsing_function:
out = self.parsing_function(self.toolagent())
else:
out = self.toolagent()
return out
elif self.llm:
logger.info(f"Running {self.name} for task: {task}")
self.toolagent = Jsonformer(
json_schema=self.json_schema,
llm=self.llm,
prompt=task,
max_number_tokens=self.max_number_tokens,
*args,
**kwargs,
)
if self.parsing_function:
out = self.parsing_function(self.toolagent())
else:
out = self.toolagent()
return out
if self.parsing_function:
out = self.parsing_function(self.toolagent())
else:
out = self.toolagent()
raise Exception(
"Either model or llm should be provided to the"
" ToolAgent"
)
return out
except Exception as error:
print(f"[Error] [ToolAgent] {error}")
logger.error(
f"Error running {self.name} for task: {task}"
)
raise error
def __call__(self, task: str, *args, **kwargs):
"""Call self as a function.
Args:
task (str): _description_
Returns:
_type_: _description_
"""
return self.run(task, *args, **kwargs)

11
swarms/memory/base_vectordb.py
Unescape View File

@ -1,4 +1,4 @@
from abc import ABC, abstractmethod
from abc import ABC
class AbstractVectorDatabase(ABC):
@ -12,7 +12,6 @@ class AbstractVectorDatabase(ABC):
"""
@abstractmethod
def connect(self):
"""
Connect to the database.
@ -21,7 +20,6 @@ class AbstractVectorDatabase(ABC):
"""
@abstractmethod
def close(self):
"""
Close the database connection.
@ -30,7 +28,6 @@ class AbstractVectorDatabase(ABC):
"""
@abstractmethod
def query(self, query: str):
"""
Execute a database query.
@ -42,7 +39,6 @@ class AbstractVectorDatabase(ABC):
"""
@abstractmethod
def fetch_all(self):
"""
Fetch all rows from the result set.
@ -54,7 +50,6 @@ class AbstractVectorDatabase(ABC):
"""
@abstractmethod
def fetch_one(self):
"""
Fetch one row from the result set.
@ -66,7 +61,6 @@ class AbstractVectorDatabase(ABC):
"""
@abstractmethod
def add(self, doc: str):
"""
Add a new record to the database.
@ -79,7 +73,6 @@ class AbstractVectorDatabase(ABC):
"""
@abstractmethod
def get(self, query: str):
"""
Get a record from the database.
@ -95,7 +88,6 @@ class AbstractVectorDatabase(ABC):
"""
@abstractmethod
def update(self, doc):
"""
Update a record in the database.
@ -109,7 +101,6 @@ class AbstractVectorDatabase(ABC):
"""
@abstractmethod
def delete(self, message):
"""
Delete a record from the database.

10
swarms/models/mistral_model_api.py
Unescape View File

@ -0,0 +1,10 @@
from swarms.models.popular_llms import OpenAIChat
class MistralAPILLM(OpenAIChat):
def __init__(self, url):
super().__init__()
self.openai_proxy_url = url
def __call__(self, task: str):
super().__call__(task)

21
swarms/models/together.py
Unescape View File

@ -119,18 +119,15 @@ class TogetherLLM(AbstractLLM):
)
out = response.json()
if "choices" in out and out["choices"]:
content = (
out["choices"][0]
.get("message", {})
.get("content", None)
)
if self.streaming_enabled:
content = self.stream_response(content)
return content
else:
print("No valid response in 'choices'")
return None
content = (
out["choices"][0]
.get("message", {})
.get("content", None)
)
if self.streaming_enabled:
content = self.stream_response(content)
return content
except Exception as error:
print(

260
swarms/structs/agent.py
Unescape View File

@ -19,16 +19,13 @@ from swarms.prompts.multi_modal_autonomous_instruction_prompt import (
)
from swarms.prompts.worker_prompt import worker_tools_sop_promp
from swarms.structs.conversation import Conversation
from swarms.structs.schemas import Step
from swarms.tokenizers.base_tokenizer import BaseTokenizer
from swarms.tools.exec_tool import execute_tool_by_name
from swarms.tools.tool import BaseTool
from swarms.utils.code_interpreter import SubprocessCodeInterpreter
from swarms.utils.data_to_text import data_to_text
from swarms.utils.parse_code import extract_code_from_markdown
from swarms.utils.pdf_to_text import pdf_to_text
from swarms.utils.token_count_tiktoken import limit_tokens_from_string
from swarms.utils.execution_sandbox import execute_code_in_sandbox
# Utils
@ -207,6 +204,10 @@ class Agent:
evaluator: Optional[Callable] = None,
output_json: bool = False,
stopping_func: Optional[Callable] = None,
custom_loop_condition: Optional[Callable] = None,
sentiment_threshold: Optional[float] = None,
custom_exit_command: Optional[str] = "exit",
sentiment_analyzer: Optional[Callable] = None,
*args,
**kwargs,
):
@ -262,6 +263,10 @@ class Agent:
self.evaluator = evaluator
self.output_json = output_json
self.stopping_func = stopping_func
self.custom_loop_condition = custom_loop_condition
self.sentiment_threshold = sentiment_threshold
self.custom_exit_command = custom_exit_command
self.sentiment_analyzer = sentiment_analyzer
# The max_loops will be set dynamically if the dynamic_loop
if self.dynamic_loops:
@ -559,190 +564,161 @@ class Agent:
):
"""
Run the autonomous agent loop
Args:
task (str): The initial task to run
Agent:
1. Generate a response
2. Check stopping condition
3. If stopping condition is met, stop
4. If stopping condition is not met, generate a response
5. Repeat until stopping condition is met or max_loops is reached
"""
try:
# Activate Autonomous agent message
self.activate_autonomous_agent()
# response = task # or combined_prompt
history = self._history(self.user_name, task)
# If dashboard = True then print the dashboard
if self.dashboard:
self.print_dashboard(task)
if task:
self.short_memory.add(
role=self.user_name, content=task
)
loop_count = 0
response = None
# While the max_loops is auto or the loop count is less than the max_loops
while (
self.max_loops == "auto"
or loop_count < self.max_loops
# or self.custom_loop_condition()
):
# Loop count
loop_count += 1
self.loop_count_print(loop_count, self.max_loops)
print("\n")
# Adjust temperature, comment if no work
if self.dynamic_temperature_enabled:
print(colored("Adjusting temperature...", "blue"))
self.dynamic_temperature()
# Preparing the prompt
task = self.agent_history_prompt(history=task)
task_prompt = (
self.short_memory.return_history_as_string()
)
attempt = 0
while attempt < self.retry_attempts:
success = False
while attempt < self.retry_attempts and not success:
try:
if img:
response = self.llm(
task,
img,
**kwargs,
)
print(response)
else:
response = self.llm(
task,
**kwargs,
)
print(response)
if self.output_json:
response = extract_code_from_markdown(
response
)
response_args = (
(task_prompt, *args)
if img is None
else (task_prompt, img, *args)
)
response = self.llm(*response_args, **kwargs)
print(response)
self.short_memory.add(
role=self.agent_name, content=response
)
# Code interpreter
if self.code_interpreter:
response = extract_code_from_markdown(
response
extracted_code = (
extract_code_from_markdown(response)
)
# Execute the code in the sandbox
response = execute_code_in_sandbox(
response
)
response = task + response
task_prompt += extracted_code
response = self.llm(
response, *args, **kwargs
task_prompt, *args, **kwargs
)
self.short_memory.add(
role=self.agent_name, content=response
)
# Add the response to the history
history.append(response)
# Log each step
step = Step(
input=str(task),
task_id=str(task_id),
step_id=str(step_id),
output=str(response),
status="running",
)
if self.evaluator:
evaluated_response = self.evaluator(
response
)
out = (
f"Response: {response}\nEvaluated"
f" Response: {evaluated_response}"
print(
"Evaluated Response:"
f" {evaluated_response}"
)
out = self.short_memory.add(
"Evaluator", out
self.short_memory.add(
role=self.agent_name,
content=evaluated_response,
)
# Stopping logic for agents
if self.stopping_token:
# Check if the stopping token is in the response
if self.stopping_token in response:
break
if self.stopping_condition:
if self._check_stopping_condition(
# Sentiment analysis
if self.sentiment_analyzer:
sentiment = self.sentiment_analyzer(
response
):
break
if self.stopping_func is not None:
if self.stopping_func(response) is True:
break
# If the stopping condition is met then break
self.step_cache.append(step)
logging.info(f"Step: {step}")
# If parser exists then parse the response
if self.parser:
response = self.parser(response)
# If tools are enabled then execute the tools
if self.tools:
execute_tool_by_name(
response,
self.tools,
self.stopping_condition,
)
print(f"Sentiment: {sentiment}")
if sentiment > self.sentiment_threshold:
print(
f"Sentiment: {sentiment} is above"
" threshold:"
f" {self.sentiment_threshold}"
)
elif sentiment < self.sentiment_threshold:
print(
f"Sentiment: {sentiment} is below"
" threshold:"
f" {self.sentiment_threshold}"
)
# print(f"Sentiment: {sentiment}")
self.short_memory.add(
role=self.agent_name,
content=sentiment,
)
# If interactive mode is enabled then print the response and get user input
if self.interactive:
print(f"AI: {response}")
history.append(f"AI: {response}")
response = input("You: ")
history.append(f"Human: {response}")
# If interactive mode is not enabled then print the response
else:
# print(f"AI: {response}")
history.append(f"AI: {response}")
# print(response)
break
success = True # Mark as successful to exit the retry loop
except Exception as e:
logging.error(
f"Error generating response: {e}"
logger.error(
f"Attempt {attempt+1}: Error generating"
f" response: {e}"
)
attempt += 1
time.sleep(self.retry_interval)
time.sleep(self.loop_interval)
# Add the history to the memory
self.short_memory.add(
role=self.agent_name, content=history
)
if not success:
logger.error(
"Failed to generate a valid response after"
" retry attempts."
)
break # Exit the loop if all retry attempts fail
# Check stopping conditions
if (
self.stopping_token
and self.stopping_token in response
):
break
elif (
self.stopping_condition
and self._check_stopping_condition(response)
):
break
elif self.stopping_func and self.stopping_func(
response
):
break
if self.interactive:
user_input = input("You: ")
# User-defined exit command
if (
user_input.lower()
== self.custom_exit_command.lower()
):
print("Exiting as per user request.")
break
# If autosave is enabled then save the state
if self.autosave:
print(
colored(
(
"Autosaving agent state to"
f" {self.saved_state_path}"
),
"green",
self.short_memory.add(
role=self.user_name, content=user_input
)
)
self.save_state(self.saved_state_path)
# If return history is enabled then return the response and history
if self.return_history:
return response, history
if self.loop_interval:
logger.info(
f"Sleeping for {self.loop_interval} seconds"
)
time.sleep(self.loop_interval)
if self.autosave:
logger.info("Autosaving agent state.")
self.save_state(self.saved_state_path)
return response
except Exception as error:
logger.error(f"Error running agent: {error}")
raise
print(f"Error running agent: {error}")
raise error
def __call__(self, task: str, img: str = None, *args, **kwargs):
"""Call the agent
@ -751,7 +727,11 @@ class Agent:
task (str): _description_
img (str, optional): _description_. Defaults to None.
"""
self.run(task, img, *args, **kwargs)
try:
self.run(task, img, *args, **kwargs)
except Exception as error:
logger.error(f"Error calling agent: {error}")
raise
def agent_history_prompt(
self,

45
swarms/structs/base_swarm.py
Unescape View File

@ -1,12 +1,14 @@
import yaml
import json
import asyncio
import json
from abc import ABC
from concurrent.futures import ThreadPoolExecutor, as_completed
from typing import Any, Callable, Dict, List, Optional, Sequence
from swarms.utils.loguru_logger import logger
import yaml
from swarms.structs.agent import Agent
from swarms.structs.conversation import Conversation
from swarms.utils.loguru_logger import logger
class AbstractSwarm(ABC):
@ -34,7 +36,7 @@ class AbstractSwarm(ABC):
assign_task: Assign a task to a agent
get_all_tasks: Get all tasks
get_finished_tasks: Get all finished tasks
get_pending_tasks: Get all pending tasks
get_pending_tasks: Get all penPding tasks
pause_agent: Pause a agent
resume_agent: Resume a agent
stop_agent: Stop a agent
@ -58,7 +60,8 @@ class AbstractSwarm(ABC):
def __init__(
self,
agents: List[Agent],
agents: List[Agent] = None,
models: List[Any] = None,
max_loops: int = 200,
callbacks: Optional[Sequence[callable]] = None,
autosave: bool = False,
@ -73,6 +76,7 @@ class AbstractSwarm(ABC):
):
"""Initialize the swarm with agents"""
self.agents = agents
self.models = models
self.max_loops = max_loops
self.callbacks = callbacks
self.autosave = autosave
@ -82,6 +86,7 @@ class AbstractSwarm(ABC):
self.stopping_function = stopping_function
self.stopping_condition = stopping_condition
self.stopping_condition_args = stopping_condition_args
self.conversation = Conversation(
time_enabled=True, *args, **kwargs
)
@ -117,6 +122,29 @@ class AbstractSwarm(ABC):
if autosave:
self.save_to_json(metadata_filename)
# Handle logging
if self.agents:
logger.info(
f"Swarm initialized with {len(self.agents)} agents"
)
# Handle stopping function
if stopping_function is not None:
if not callable(stopping_function):
raise TypeError("Stopping function must be callable.")
if stopping_condition_args is None:
stopping_condition_args = {}
self.stopping_condition_args = stopping_condition_args
self.stopping_condition = stopping_condition
self.stopping_function = stopping_function
# Handle stopping condition
if stopping_condition is not None:
if stopping_condition_args is None:
stopping_condition_args = {}
self.stopping_condition_args = stopping_condition_args
self.stopping_condition = stopping_condition
# @abstractmethod
def communicate(self):
"""Communicate with the swarm through the orchestrator, protocols, and the universal communication layer"""
@ -124,6 +152,7 @@ class AbstractSwarm(ABC):
# @abstractmethod
def run(self):
"""Run the swarm"""
...
def __call__(
self,
@ -139,7 +168,11 @@ class AbstractSwarm(ABC):
Returns:
_type_: _description_
"""
return self.run(task, *args, **kwargs)
try:
return self.run(task, *args, **kwargs)
except Exception as error:
logger.error(f"Error running {self.__class__.__name__}")
raise error
def step(self):
"""Step the swarm"""

12
swarms/structs/base_workflow.py
Unescape View File

@ -19,7 +19,14 @@ class BaseWorkflow(BaseStructure):
"""
def __init__(self, *args, **kwargs):
def __init__(
self,
agents: List[Agent] = None,
task_pool: List[Task] = None,
models: List[Any] = None,
*args,
**kwargs,
):
super().__init__(*args, **kwargs)
self.task_pool = []
self.agent_pool = []
@ -69,13 +76,14 @@ class BaseWorkflow(BaseStructure):
"""
Abstract method to run the workflow.
"""
raise NotImplementedError("You must implement this method")
...
def __sequential_loop(self):
"""
Abstract method for the sequential loop.
"""
# raise NotImplementedError("You must implement this method")
...
def __log(self, message: str):
"""

13
swarms/tokenizers/__init__.py
Unescape View File

@ -1,7 +1,7 @@
from swarms.tokenizers.anthropic_tokenizer import (
AnthropicTokenizer,
import_optional_dependency,
)
# from swarms.tokenizers.anthropic_tokenizer import (
# AnthropicTokenizer,
# import_optional_dependency,
# )
from swarms.tokenizers.base_tokenizer import BaseTokenizer
from swarms.tokenizers.openai_tokenizers import OpenAITokenizer
from swarms.tokenizers.r_tokenizers import (
@ -10,12 +10,13 @@ from swarms.tokenizers.r_tokenizers import (
Tokenizer,
)
__all__ = [
"SentencePieceTokenizer",
"HuggingFaceTokenizer",
"Tokenizer",
"BaseTokenizer",
"OpenAITokenizer",
"import_optional_dependency",
"AnthropicTokenizer",
# "import_optional_dependency",
# "AnthropicTokenizer",
]

355
swarms/tools/format_tools.py
Unescape View File

@ -3,12 +3,13 @@ from typing import Any, Dict, List, Union
from termcolor import cprint
from transformers import PreTrainedModel, PreTrainedTokenizer
from pydantic import BaseModel
from swarms.tools.logits_processor import (
NumberStoppingCriteria,
OutputNumbersTokens,
StringStoppingCriteria,
)
from swarms.models.base_llm import AbstractLLM
GENERATION_MARKER = "|GENERATION|"
@ -35,21 +36,25 @@ class Jsonformer:
def __init__(
self,
model: PreTrainedModel,
tokenizer: PreTrainedTokenizer,
json_schema: Dict[str, Any],
prompt: str,
model: PreTrainedModel = None,
tokenizer: PreTrainedTokenizer = None,
json_schema: Union[Dict[str, Any], BaseModel] = None,
schemas: List[Union[Dict[str, Any], BaseModel]] = [],
prompt: str = None,
*,
debug: bool = False,
max_array_length: int = 10,
max_number_tokens: int = 6,
temperature: float = 1.0,
max_string_token_length: int = 10,
llm: AbstractLLM = None,
):
self.model = model
self.tokenizer = tokenizer
self.json_schema = json_schema
self.prompt = prompt
self.llm = llm
self.schemas = schemas
self.number_logit_processor = OutputNumbersTokens(
self.tokenizer, self.prompt
@ -88,41 +93,67 @@ class Jsonformer:
Raises:
ValueError: If a valid number cannot be generated after 3 iterations.
"""
prompt = self.get_prompt()
self.debug("[generate_number]", prompt, is_prompt=True)
input_tokens = self.tokenizer.encode(
prompt, return_tensors="pt"
).to(self.model.device)
response = self.model.generate(
input_tokens,
max_new_tokens=self.max_number_tokens,
num_return_sequences=1,
logits_processor=[self.number_logit_processor],
stopping_criteria=[
NumberStoppingCriteria(
self.tokenizer, len(input_tokens[0])
if self.model:
prompt = self.get_prompt()
self.debug("[generate_number]", prompt, is_prompt=True)
input_tokens = self.tokenizer.encode(
prompt, return_tensors="pt"
).to(self.model.device)
response = self.model.generate(
input_tokens,
max_new_tokens=self.max_number_tokens,
num_return_sequences=1,
logits_processor=[self.number_logit_processor],
stopping_criteria=[
NumberStoppingCriteria(
self.tokenizer, len(input_tokens[0])
)
],
temperature=temperature or self.temperature,
pad_token_id=self.tokenizer.eos_token_id,
)
response = self.tokenizer.decode(
response[0], skip_special_tokens=True
)
response = response[len(prompt) :]
response = response.strip().rstrip(".")
self.debug("[generate_number]", response)
try:
return float(response)
except ValueError:
if iterations > 3:
raise ValueError(
"Failed to generate a valid number"
)
return self.generate_number(
temperature=self.temperature * 1.3,
iterations=iterations + 1,
)
elif self.llm:
prompt = self.get_prompt()
self.debug("[generate_number]", prompt, is_prompt=True)
response = self.llm(prompt)
response = response[len(prompt) :]
response = response.strip().rstrip(".")
self.debug("[generate_number]", response)
try:
return float(response)
except ValueError:
if iterations > 3:
raise ValueError(
"Failed to generate a valid number"
)
return self.generate_number(
temperature=self.temperature * 1.3,
iterations=iterations + 1,
)
],
temperature=temperature or self.temperature,
pad_token_id=self.tokenizer.eos_token_id,
)
response = self.tokenizer.decode(
response[0], skip_special_tokens=True
)
response = response[len(prompt) :]
response = response.strip().rstrip(".")
self.debug("[generate_number]", response)
try:
return float(response)
except ValueError:
if iterations > 3:
raise ValueError("Failed to generate a valid number")
return self.generate_number(
temperature=self.temperature * 1.3,
iterations=iterations + 1,
)
elif self.llm and self.model:
raise ValueError("Both LLM and model cannot be None")
def generate_boolean(self) -> bool:
"""
@ -131,71 +162,118 @@ class Jsonformer:
Returns:
bool: The generated boolean value.
"""
prompt = self.get_prompt()
self.debug("[generate_boolean]", prompt, is_prompt=True)
if self.model:
prompt = self.get_prompt()
self.debug("[generate_boolean]", prompt, is_prompt=True)
input_tensor = self.tokenizer.encode(
prompt, return_tensors="pt"
)
output = self.model.forward(
input_tensor.to(self.model.device)
)
logits = output.logits[0, -1]
input_tensor = self.tokenizer.encode(
prompt, return_tensors="pt"
)
output = self.model.forward(
input_tensor.to(self.model.device)
)
logits = output.logits[0, -1]
# todo: this assumes that "true" and "false" are both tokenized to a single token
# this is probably not true for all tokenizers
# this can be fixed by looking at only the first token of both "true" and "false"
true_token_id = self.tokenizer.convert_tokens_to_ids(
"true"
)
false_token_id = self.tokenizer.convert_tokens_to_ids(
"false"
)
result = logits[true_token_id] > logits[false_token_id]
# todo: this assumes that "true" and "false" are both tokenized to a single token
# this is probably not true for all tokenizers
# this can be fixed by looking at only the first token of both "true" and "false"
true_token_id = self.tokenizer.convert_tokens_to_ids("true")
false_token_id = self.tokenizer.convert_tokens_to_ids("false")
self.debug("[generate_boolean]", result)
result = logits[true_token_id] > logits[false_token_id]
return result.item()
self.debug("[generate_boolean]", result)
elif self.llm:
prompt = self.get_prompt()
self.debug("[generate_boolean]", prompt, is_prompt=True)
return result.item()
output = self.llm(prompt)
return output if output == "true" or "false" else None
else:
raise ValueError("Both LLM and model cannot be None")
def generate_string(self) -> str:
prompt = self.get_prompt() + '"'
self.debug("[generate_string]", prompt, is_prompt=True)
input_tokens = self.tokenizer.encode(
prompt, return_tensors="pt"
).to(self.model.device)
response = self.model.generate(
input_tokens,
max_new_tokens=self.max_string_token_length,
num_return_sequences=1,
temperature=self.temperature,
stopping_criteria=[
StringStoppingCriteria(
self.tokenizer, len(input_tokens[0])
)
],
pad_token_id=self.tokenizer.eos_token_id,
)
if self.model:
prompt = self.get_prompt() + '"'
self.debug("[generate_string]", prompt, is_prompt=True)
input_tokens = self.tokenizer.encode(
prompt, return_tensors="pt"
).to(self.model.device)
response = self.model.generate(
input_tokens,
max_new_tokens=self.max_string_token_length,
num_return_sequences=1,
temperature=self.temperature,
stopping_criteria=[
StringStoppingCriteria(
self.tokenizer, len(input_tokens[0])
)
],
pad_token_id=self.tokenizer.eos_token_id,
)
# Some models output the prompt as part of the response
# This removes the prompt from the response if it is present
if (
len(response[0]) >= len(input_tokens[0])
and (
response[0][: len(input_tokens[0])] == input_tokens
).all()
):
response = response[0][len(input_tokens[0]) :]
if response.shape[0] == 1:
response = response[0]
response = self.tokenizer.decode(
response, skip_special_tokens=True
)
# Some models output the prompt as part of the response
# This removes the prompt from the response if it is present
if (
len(response[0]) >= len(input_tokens[0])
and (
response[0][: len(input_tokens[0])]
== input_tokens
).all()
):
response = response[0][len(input_tokens[0]) :]
if response.shape[0] == 1:
response = response[0]
response = self.tokenizer.decode(
response, skip_special_tokens=True
)
self.debug("[generate_string]", "|" + response + "|")
if response.count('"') < 1:
return response
return response.split('"')[0].strip()
elif self.llm:
prompt = self.get_prompt() + '"'
self.debug("[generate_string]", prompt, is_prompt=True)
response = self.llm(prompt)
self.debug("[generate_string]", "|" + response + "|")
# Some models output the prompt as part of the response
# This removes the prompt from the response if it is present
if (
len(response[0]) >= len(input_tokens[0])
and (
response[0][: len(input_tokens[0])]
== input_tokens
).all()
):
response = response[0][len(input_tokens[0]) :]
if response.shape[0] == 1:
response = response[0]
if response.count('"') < 1:
return response
self.debug("[generate_string]", "|" + response + "|")
return response.split('"')[0].strip()
if response.count('"') < 1:
return response
return response.split('"')[0].strip()
else:
raise ValueError("Both LLM and model cannot be None")
def generate_object(
self, properties: Dict[str, Any], obj: Dict[str, Any]
@ -249,43 +327,72 @@ class Jsonformer:
def generate_array(
self, item_schema: Dict[str, Any], obj: Dict[str, Any]
) -> list:
for _ in range(self.max_array_length):
# forces array to have at least one element
element = self.generate_value(item_schema, obj)
obj[-1] = element
obj.append(self.generation_marker)
input_prompt = self.get_prompt()
obj.pop()
input_tensor = self.tokenizer.encode(
input_prompt, return_tensors="pt"
)
output = self.model.forward(
input_tensor.to(self.model.device)
)
logits = output.logits[0, -1]
if self.model:
for _ in range(self.max_array_length):
# forces array to have at least one element
element = self.generate_value(item_schema, obj)
obj[-1] = element
obj.append(self.generation_marker)
input_prompt = self.get_prompt()
obj.pop()
input_tensor = self.tokenizer.encode(
input_prompt, return_tensors="pt"
)
output = self.model.forward(
input_tensor.to(self.model.device)
)
logits = output.logits[0, -1]
top_indices = logits.topk(30).indices
sorted_token_ids = top_indices[
logits[top_indices].argsort(descending=True)
]
found_comma = False
found_close_bracket = False
for token_id in sorted_token_ids:
decoded_token = self.tokenizer.decode(token_id)
if "," in decoded_token:
found_comma = True
break
if "]" in decoded_token:
found_close_bracket = True
break
if found_close_bracket or not found_comma:
break
top_indices = logits.topk(30).indices
sorted_token_ids = top_indices[
logits[top_indices].argsort(descending=True)
]
return obj
found_comma = False
found_close_bracket = False
elif self.llm:
for _ in range(self.max_array_length):
# forces array to have at least one element
element = self.generate_value(item_schema, obj)
obj[-1] = element
for token_id in sorted_token_ids:
decoded_token = self.tokenizer.decode(token_id)
if "," in decoded_token:
found_comma = True
break
if "]" in decoded_token:
found_close_bracket = True
obj.append(self.generation_marker)
input_prompt = self.get_prompt()
obj.pop()
output = self.llm(input_prompt)
found_comma = False
found_close_bracket = False
for token_id in output:
decoded_token = str(token_id)
if "," in decoded_token:
found_comma = True
break
if "]" in decoded_token:
found_close_bracket = True
break
if found_close_bracket or not found_comma:
break
if found_close_bracket or not found_comma:
break
return obj
return obj
def get_prompt(self):
template = """{prompt}\nOutput result in the following JSON schema format:\n{schema}\nResult: {progress}"""

8
swarms/tools/tool.py
Unescape View File

@ -0,0 +1,8 @@
from langchain.tools import (
BaseTool,
Tool,
StructuredTool,
tool,
) # noqa F401
__all__ = ["BaseTool", "Tool", "StructuredTool", "tool"]

50
tool_agent_with_llm.py
Unescape View File

@ -0,0 +1,50 @@
import os
from dotenv import load_dotenv
from pydantic import BaseModel, Field
from swarms import OpenAIChat, ToolAgent
from swarms.utils.json_utils import base_model_to_json
# Load the environment variables
load_dotenv()
# Initialize the OpenAIChat class
chat = OpenAIChat(
api_key=os.getenv("OPENAI_API"),
)
# Initialize the schema for the person's information
class Schema(BaseModel):
name: str = Field(..., title="Name of the person")
agent: int = Field(..., title="Age of the person")
is_student: bool = Field(
..., title="Whether the person is a student"
)
courses: list[str] = Field(
..., title="List of courses the person is taking"
)
# Convert the schema to a JSON string
tool_schema = base_model_to_json(Schema)
# Define the task to generate a person's information
task = (
"Generate a person's information based on the following schema:"
)
# Create an instance of the ToolAgent class
agent = ToolAgent(
name="dolly-function-agent",
description="Ana gent to create a child data",
llm=chat,
json_schema=tool_schema,
)
# Run the agent to generate the person's information
generated_data = agent(task)
# Print the generated data
print(f"Generated data: {generated_data}")
Write Preview
Loading…
Cancel
Save