ai-mirow-projects
/
swarms
mirror of https://github.com/kyegomez/swarms
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'c4f41f90f5'
${ noResults }
swarms/docs/site/guides/agent_evals/index.html

7656 lines
151 KiB
Raw Blame History

<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="description" content="The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework">
<meta name="author" content="Swarms">
<link rel="canonical" href="https://docs.swarms.world/guides/agent_evals/">
<link rel="icon" href="../../assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.6.1, mkdocs-material-9.6.15">
<title>Agent evals - Swarms</title>
<link rel="stylesheet" href="../../assets/stylesheets/main.342714a4.min.css">
<link rel="stylesheet" href="../../assets/stylesheets/palette.06af60db.min.css">
<style>
/* Custom Footer Styling - Base */
.md-footer-custom {
padding: 2.4rem 0 1.2rem;
border-top: 0.05rem solid var(--md-default-fg-color--lightest);
}
.md-footer-custom__inner {
margin: 0 auto;
padding: 0 1.2rem;
}
.md-footer-links {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(240px, 1fr));
gap: 2rem;
max-width: 1220px;
margin: 0 auto;
}
.md-footer-links__section {
min-width: 0;
}
.md-footer-links__title {
font-size: 0.64rem;
font-weight: 700;
margin: 0 0 1rem;
text-transform: uppercase;
letter-spacing: 0.1em;
padding-bottom: 0.4rem;
}
.md-footer-links__list {
list-style: none;
margin: 0;
padding: 0;
}
.md-footer-links__item {
margin: 0;
line-height: 1.8;
}
.md-footer-links__link {
text-decoration: none;
font-size: 0.7rem;
display: block;
padding: 0.1rem 0;
transition: color 125ms;
border-radius: 0.1rem;
}
.md-footer-links__link:hover,
.md-footer-links__link:focus {
color: var(--md-accent-fg-color);
}
/* Light Mode (Default) */
[data-md-color-scheme="default"] .md-footer-custom {
background: #ffffff;
border-top-color: #e1e5e9;
}
[data-md-color-scheme="default"] .md-footer-links__title {
color: #2e3440;
border-bottom: 0.05rem solid #e1e5e9;
}
[data-md-color-scheme="default"] .md-footer-links__link {
color: #636c76;
}
[data-md-color-scheme="default"] .md-footer-links__link:hover,
[data-md-color-scheme="default"] .md-footer-links__link:focus {
color: #1976d2;
}
/* Dark Mode (Slate) */
[data-md-color-scheme="slate"] .md-footer-custom {
background: #1F2129;
border-top-color: #404040;
}
[data-md-color-scheme="slate"] .md-footer-links__title {
color: #ffffff;
border-bottom: 0.05rem solid #404040;
}
[data-md-color-scheme="slate"] .md-footer-links__link {
color: #9ca3af;
}
[data-md-color-scheme="slate"] .md-footer-links__link:hover,
[data-md-color-scheme="slate"] .md-footer-links__link:focus {
color: #42a5f5;
}
/* Company Information Section - Base */
.md-footer-company {
padding: 1.5rem 0;
border-top: 0.05rem solid var(--md-default-fg-color--lightest);
}
.md-footer-company__inner {
margin: 0 auto;
padding: 0 1.2rem;
max-width: 1220px;
}
.md-footer-company__content {
display: flex;
justify-content: space-between;
align-items: center;
flex-wrap: wrap;
gap: 1rem;
}
.md-footer-company__brand {
flex: 1;
min-width: 200px;
}
.md-footer-company__name {
margin: 0 0 0.5rem 0;
font-size: 1.2rem;
font-weight: 700;
letter-spacing: 0.05em;
}
.md-footer-company__description {
margin: 0;
font-size: 0.8rem;
line-height: 1.4;
font-style: italic;
}
.md-footer-company__copyright {
text-align: right;
}
.md-footer-company__copyright p {
margin: 0;
font-size: 0.7rem;
opacity: 0.8;
}
/* Company Section - Light Mode */
[data-md-color-scheme="default"] .md-footer-company {
background: #f8f9fa;
border-top-color: #e1e5e9;
}
[data-md-color-scheme="default"] .md-footer-company__name {
color: #DC143C;
}
[data-md-color-scheme="default"] .md-footer-company__description {
color: #495057;
}
[data-md-color-scheme="default"] .md-footer-company__copyright p {
color: #6c757d;
}
/* Company Section - Dark Mode */
[data-md-color-scheme="slate"] .md-footer-company {
background: #1F2129;
border-top-color: #404040;
}
[data-md-color-scheme="slate"] .md-footer-company__name {
color: #DC143C;
}
[data-md-color-scheme="slate"] .md-footer-company__description {
color: #d1d5db;
}
[data-md-color-scheme="slate"] .md-footer-company__copyright p {
color: #9ca3af;
}
/* Responsive Design */
@media screen and (max-width: 76.1875em) {
.md-footer-links {
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
gap: 1.5rem;
}
.md-footer-custom {
padding: 2rem 0 1rem;
}
}
@media screen and (max-width: 59.9375em) {
.md-footer-links {
grid-template-columns: repeat(2, 1fr);
gap: 1.5rem;
}
}
@media screen and (max-width: 44.9375em) {
.md-footer-links {
grid-template-columns: 1fr;
gap: 1.5rem;
}
.md-footer-custom {
padding: 1.5rem 0 0.8rem;
}
.md-footer-custom__inner {
padding: 0 1rem;
}
/* Company section mobile styles */
.md-footer-company__content {
flex-direction: column;
text-align: center;
gap: 1rem;
}
.md-footer-company__brand {
min-width: auto;
}
.md-footer-company__copyright {
text-align: center;
}
.md-footer-company__inner {
padding: 0 1rem;
}
}
</style>
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Fira+Sans:300,300i,400,400i,700,700i%7CFira+Code:400,400i,700,700i&display=fallback">
<style>:root{--md-text-font:"Fira Sans";--md-code-font:"Fira Code"}</style>
<link rel="stylesheet" href="../../assets/_mkdocstrings.css">
<link rel="stylesheet" href="../../assets/css/extra.css">
<script>__md_scope=new URL("../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
<script id="__analytics">function __md_analytics(){function e(){dataLayer.push(arguments)}window.dataLayer=window.dataLayer||[],e("js",new Date),e("config","G-MPE9C65596"),document.addEventListener("DOMContentLoaded",(function(){document.forms.search&&document.forms.search.query.addEventListener("blur",(function(){this.value&&e("event","search",{search_term:this.value})}));document$.subscribe((function(){var t=document.forms.feedback;if(void 0!==t)for(var a of t.querySelectorAll("[type=submit]"))a.addEventListener("click",(function(a){a.preventDefault();var n=document.location.pathname,d=this.getAttribute("data-md-value");e("event","feedback",{page:n,data:d}),t.firstElementChild.disabled=!0;var r=t.querySelector(".md-feedback__note [data-md-value='"+d+"']");r&&(r.hidden=!1)})),t.hidden=!1})),location$.subscribe((function(t){e("config","G-MPE9C65596",{page_path:t.pathname})}))}));var t=document.createElement("script");t.async=!0,t.src="https://www.googletagmanager.com/gtag/js?id=G-MPE9C65596",document.getElementById("__analytics").insertAdjacentElement("afterEnd",t)}</script>
<script>"undefined"!=typeof __md_analytics&&__md_analytics()</script>
</head>
<body dir="ltr" data-md-color-scheme="default" data-md-color-primary="white" data-md-color-accent="white">
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
<a href="#understanding-agent-evaluation-mechanisms" class="md-skip">
Skip to content
</a>
</div>
<div data-md-component="announce">
<aside class="md-banner">
<div class="md-banner__inner md-grid md-typeset">
<button class="md-banner__button md-icon" aria-label="Don't show this again">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12z"/></svg>
</button>
<div style="text-align:center">
<a href="https://github.com/kyegomez/swarms">Star and contribute</a> to Swarms on GitHub!
</div>
</div>
<script>var el=document.querySelector("[data-md-component=announce]");if(el){var content=el.querySelector(".md-typeset");__md_hash(content.innerHTML)===__md_get("__announce")&&(el.hidden=!0)}</script>
</aside>
</div>
<header class="md-header" data-md-component="header">
<nav class="md-header__inner md-grid" aria-label="Header">
<a href="../.." title="Swarms" class="md-header__button md-logo" aria-label="Swarms" data-md-component="logo">
<img src="../../assets/img/swarms-logo.png" alt="logo">
</a>
<label class="md-header__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg>
</label>
<div class="md-header__title" data-md-component="header-title">
<div class="md-header__ellipsis">
<div class="md-header__topic">
<span class="md-ellipsis">
Swarms
</span>
</div>
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
Agent evals
</span>
</div>
</div>
</div>
<form class="md-header__option" data-md-component="palette">
<input class="md-option" data-md-color-media="" data-md-color-scheme="default" data-md-color-primary="white" data-md-color-accent="white" aria-label="Switch to dark mode" type="radio" name="__palette" id="__palette_0">
<label class="md-header__button md-icon" title="Switch to dark mode" for="__palette_1" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a4 4 0 0 0-4 4 4 4 0 0 0 4 4 4 4 0 0 0 4-4 4 4 0 0 0-4-4m0 10a6 6 0 0 1-6-6 6 6 0 0 1 6-6 6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg>
</label>
<input class="md-option" data-md-color-media="" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="black" aria-label="Switch to light mode" type="radio" name="__palette" id="__palette_1">
<label class="md-header__button md-icon" title="Switch to light mode" for="__palette_0" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 18c-.89 0-1.74-.2-2.5-.55C11.56 16.5 13 14.42 13 12s-1.44-4.5-3.5-5.45C10.26 6.2 11.11 6 12 6a6 6 0 0 1 6 6 6 6 0 0 1-6 6m8-9.31V4h-4.69L12 .69 8.69 4H4v4.69L.69 12 4 15.31V20h4.69L12 23.31 15.31 20H20v-4.69L23.31 12z"/></svg>
</label>
</form>
<script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script>
<label class="md-header__button md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
</label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
<label class="md-search__icon md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11z"/></svg>
</label>
<nav class="md-search__options" aria-label="Search">
<button type="reset" class="md-search__icon md-icon" title="Clear" aria-label="Clear" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12z"/></svg>
</button>
</nav>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" tabindex="0" data-md-scrollfix>
<div class="md-search-result" data-md-component="search-result">
<div class="md-search-result__meta">
Initializing search
</div>
<ol class="md-search-result__list" role="presentation"></ol>
</div>
</div>
</div>
</div>
</div>
<div class="md-header__source">
<a href="https://github.com/kyegomez/swarms" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2024 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81"/></svg>
</div>
<div class="md-source__repository">
kyegomez/swarms
</div>
</a>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<nav class="md-tabs" aria-label="Tabs" data-md-component="tabs">
<div class="md-grid">
<ul class="md-tabs__list">
<li class="md-tabs__item">
<a href="../.." class="md-tabs__link">
Home
</a>
</li>
<li class="md-tabs__item">
<a href="../../examples/" class="md-tabs__link">
Examples
</a>
</li>
<li class="md-tabs__item">
<a href="../../swarms_cloud/swarms_api/" class="md-tabs__link">
Swarms Cloud API
</a>
</li>
<li class="md-tabs__item">
<a href="../../swarms_platform/" class="md-tabs__link">
Swarms Marketplace
</a>
</li>
<li class="md-tabs__item">
<a href="../../swarms_rs/overview/" class="md-tabs__link">
Swarms Rust
</a>
</li>
<li class="md-tabs__item">
<a href="../../contributors/main/" class="md-tabs__link">
Contributors
</a>
</li>
<li class="md-tabs__item">
<a href="../../swarms/concept/vision/" class="md-tabs__link">
About Swarms
</a>
</li>
</ul>
</div>
</nav>
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary md-nav--lifted" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href="../.." title="Swarms" class="md-nav__button md-logo" aria-label="Swarms" data-md-component="logo">
<img src="../../assets/img/swarms-logo.png" alt="logo">
</a>
Swarms
</label>
<div class="md-nav__source">
<a href="https://github.com/kyegomez/swarms" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2024 Fonticons, Inc.--><path d="M439.55 236.05 244 40.45a28.87 28.87 0 0 0-40.81 0l-40.66 40.63 51.52 51.52c27.06-9.14 52.68 16.77 43.39 43.68l49.66 49.66c34.23-11.8 61.18 31 35.47 56.69-26.49 26.49-70.21-2.87-56-37.34L240.22 199v121.85c25.3 12.54 22.26 41.85 9.08 55a34.34 34.34 0 0 1-48.55 0c-17.57-17.6-11.07-46.91 11.25-56v-123c-20.8-8.51-24.6-30.74-18.64-45L142.57 101 8.45 235.14a28.86 28.86 0 0 0 0 40.81l195.61 195.6a28.86 28.86 0 0 0 40.8 0l194.69-194.69a28.86 28.86 0 0 0 0-40.81"/></svg>
</div>
<div class="md-source__repository">
kyegomez/swarms
</div>
</a>
</div>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1" >
<label class="md-nav__link" for="__nav_1" id="__nav_1_label" tabindex="0">
<span class="md-ellipsis">
Home
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_1_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1">
<span class="md-nav__icon md-icon"></span>
Home
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../.." class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_2" >
<label class="md-nav__link" for="__nav_1_2" id="__nav_1_2_label" tabindex="0">
<span class="md-ellipsis">
Onboarding
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_1_2_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_2">
<span class="md-nav__icon md-icon"></span>
Onboarding
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/install/install/" class="md-nav__link">
<span class="md-ellipsis">
Installation
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/install/env/" class="md-nav__link">
<span class="md-ellipsis">
Environment Configuration
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../quickstart/" class="md-nav__link">
<span class="md-ellipsis">
Quickstart
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/" class="md-nav__link">
<span class="md-ellipsis">
Agents
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/" class="md-nav__link">
<span class="md-ellipsis">
Multi-Agent Architectures
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/features/" class="md-nav__link">
<span class="md-ellipsis">
Feature Set
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_3" >
<label class="md-nav__link" for="__nav_1_3" id="__nav_1_3_label" tabindex="0">
<span class="md-ellipsis">
Agents
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_1_3_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_3">
<span class="md-nav__icon md-icon"></span>
Agents
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/framework/agents_explained/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/" class="md-nav__link">
<span class="md-ellipsis">
Agents
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/agent/" class="md-nav__link">
<span class="md-ellipsis">
Reference
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/models/agent_and_models/" class="md-nav__link">
<span class="md-ellipsis">
LLM Providers
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/tools/tools_examples/" class="md-nav__link">
<span class="md-ellipsis">
Tools and MCP
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/agent_multi_agent_communication/" class="md-nav__link">
<span class="md-ellipsis">
Multi-Agent Helpers
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/create_agents_yaml/" class="md-nav__link">
<span class="md-ellipsis">
Running Agents with YAML
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_3_8" >
<label class="md-nav__link" for="__nav_1_3_8" id="__nav_1_3_8_label" tabindex="0">
<span class="md-ellipsis">
Additional Capabilities
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="3" aria-labelledby="__nav_1_3_8_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_3_8">
<span class="md-nav__icon md-icon"></span>
Additional Capabilities
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_3_8_1" >
<label class="md-nav__link" for="__nav_1_3_8_1" id="__nav_1_3_8_1_label" tabindex="0">
<span class="md-ellipsis">
Tools Reference
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_1_3_8_1_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_3_8_1">
<span class="md-nav__icon md-icon"></span>
Tools Reference
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/tools/main/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/tools/build_tool/" class="md-nav__link">
<span class="md-ellipsis">
What are tools?
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/structured_outputs/" class="md-nav__link">
<span class="md-ellipsis">
Structured Outputs
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/agent_mcp/" class="md-nav__link">
<span class="md-ellipsis">
Agent MCP Integration
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_3_8_2" >
<label class="md-nav__link" for="__nav_1_3_8_2" id="__nav_1_3_8_2_label" tabindex="0">
<span class="md-ellipsis">
Long Term Memory Reference
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_1_3_8_2_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_3_8_2">
<span class="md-nav__icon md-icon"></span>
Long Term Memory Reference
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/memory/diy_memory/" class="md-nav__link">
<span class="md-ellipsis">
Integrating RAG with Agents
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_4" >
<label class="md-nav__link" for="__nav_1_4" id="__nav_1_4_label" tabindex="0">
<span class="md-ellipsis">
Third-Party Agents
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_1_4_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_4">
<span class="md-nav__icon md-icon"></span>
Third-Party Agents
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/agents/new_agent/" class="md-nav__link">
<span class="md-ellipsis">
Creating Custom Agents
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/openai_assistant/" class="md-nav__link">
<span class="md-ellipsis">
OpenAI Assistant
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/external_party_agents/" class="md-nav__link">
<span class="md-ellipsis">
Integrating External Agents from Griptape, Langchain, etc
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_5" >
<label class="md-nav__link" for="__nav_1_5" id="__nav_1_5_label" tabindex="0">
<span class="md-ellipsis">
Reasoning Agents
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_1_5_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_5">
<span class="md-nav__icon md-icon"></span>
Reasoning Agents
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/agents/reasoning_agents_overview/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/consistency_agent/" class="md-nav__link">
<span class="md-ellipsis">
Self Consistency Agent
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/iterative_agent/" class="md-nav__link">
<span class="md-ellipsis">
IRE Agent
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/reasoning_duo/" class="md-nav__link">
<span class="md-ellipsis">
Reasoning Duo
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/reflexion_agent/" class="md-nav__link">
<span class="md-ellipsis">
Reflexion Agent
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/gkp_agent/" class="md-nav__link">
<span class="md-ellipsis">
GKP Agent
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/agent_judge/" class="md-nav__link">
<span class="md-ellipsis">
Agent Judge
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/agents/reasoning_agent_router/" class="md-nav__link">
<span class="md-ellipsis">
Reasoning Agent Router
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_6" >
<label class="md-nav__link" for="__nav_1_6" id="__nav_1_6_label" tabindex="0">
<span class="md-ellipsis">
Multi-Agent Architectures
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_1_6_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_6">
<span class="md-nav__icon md-icon"></span>
Multi-Agent Architectures
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/concept/swarm_architectures/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/concept/why/" class="md-nav__link">
<span class="md-ellipsis">
Benefits
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/concept/how_to_choose_swarms/" class="md-nav__link">
<span class="md-ellipsis">
Choosing Multi Agent Architecture
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_6_4" >
<label class="md-nav__link" for="__nav_1_6_4" id="__nav_1_6_4_label" tabindex="0">
<span class="md-ellipsis">
Board of Directors
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="3" aria-labelledby="__nav_1_6_4_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_6_4">
<span class="md-nav__icon md-icon"></span>
Board of Directors
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/structs/board_of_directors/board_of_directors_swarm/" class="md-nav__link">
<span class="md-ellipsis">
BoardOfDirectorsSwarm
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/board_of_directors/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/board_of_directors/board_of_directors_workflow/" class="md-nav__link">
<span class="md-ellipsis">
Workflow
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/board_of_directors/board_of_directors_roles/" class="md-nav__link">
<span class="md-ellipsis">
Roles & Responsibilities
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/board_of_directors/board_of_directors_decision_making/" class="md-nav__link">
<span class="md-ellipsis">
Decision Making
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_6_5" >
<label class="md-nav__link" for="__nav_1_6_5" id="__nav_1_6_5_label" tabindex="0">
<span class="md-ellipsis">
Documentation
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="3" aria-labelledby="__nav_1_6_5_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_6_5">
<span class="md-nav__icon md-icon"></span>
Documentation
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/structs/overview/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/custom_swarm/" class="md-nav__link">
<span class="md-ellipsis">
Custom Multi Agent Architectures
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/majorityvoting/" class="md-nav__link">
<span class="md-ellipsis">
MajorityVoting
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/round_robin_swarm/" class="md-nav__link">
<span class="md-ellipsis">
RoundRobin
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/moa/" class="md-nav__link">
<span class="md-ellipsis">
Mixture of Agents
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/spreadsheet_swarm/" class="md-nav__link">
<span class="md-ellipsis">
SpreadSheetSwarm
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/forest_swarm/" class="md-nav__link">
<span class="md-ellipsis">
ForestSwarm
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/malt/" class="md-nav__link">
<span class="md-ellipsis">
MALT
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/various_execution_methods/" class="md-nav__link">
<span class="md-ellipsis">
Various Execution Methods
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/deep_research_swarm/" class="md-nav__link">
<span class="md-ellipsis">
Deep Research Swarm
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/council_of_judges/" class="md-nav__link">
<span class="md-ellipsis">
Council of Judges
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/heavy_swarm/" class="md-nav__link">
<span class="md-ellipsis">
Heavy Swarm
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_6_5_13" >
<label class="md-nav__link" for="__nav_1_6_5_13" id="__nav_1_6_5_13_label" tabindex="0">
<span class="md-ellipsis">
Hiearchical Architectures
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_1_6_5_13_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_6_5_13">
<span class="md-nav__icon md-icon"></span>
Hiearchical Architectures
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/structs/multi_swarm_orchestration/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/hierarchical_swarm/" class="md-nav__link">
<span class="md-ellipsis">
HierarchicalSwarm
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/auto_agent_builder/" class="md-nav__link">
<span class="md-ellipsis">
Auto Agent Builder
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/hhcs/" class="md-nav__link">
<span class="md-ellipsis">
Hybrid Hierarchical-Cluster Swarm
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/auto_swarm_builder/" class="md-nav__link">
<span class="md-ellipsis">
Auto Swarm Builder
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/swarm_matcher/" class="md-nav__link">
<span class="md-ellipsis">
Swarm Matcher
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_6_5_14" >
<label class="md-nav__link" for="__nav_1_6_5_14" id="__nav_1_6_5_14_label" tabindex="0">
<span class="md-ellipsis">
Routers
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_1_6_5_14_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_6_5_14">
<span class="md-nav__icon md-icon"></span>
Routers
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/structs/swarm_router/" class="md-nav__link">
<span class="md-ellipsis">
SwarmRouter
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/multi_agent_router/" class="md-nav__link">
<span class="md-ellipsis">
MultiAgentRouter
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/model_router/" class="md-nav__link">
<span class="md-ellipsis">
ModelRouter
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_6_5_15" >
<label class="md-nav__link" for="__nav_1_6_5_15" id="__nav_1_6_5_15_label" tabindex="0">
<span class="md-ellipsis">
Configuration
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_1_6_5_15_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_6_5_15">
<span class="md-nav__icon md-icon"></span>
Configuration
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/config/board_config/" class="md-nav__link">
<span class="md-ellipsis">
Board of Directors Configuration
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_6_5_16" >
<label class="md-nav__link" for="__nav_1_6_5_16" id="__nav_1_6_5_16_label" tabindex="0">
<span class="md-ellipsis">
Rearrangers
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_1_6_5_16_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_6_5_16">
<span class="md-nav__icon md-icon"></span>
Rearrangers
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/structs/swarm_rearrange/" class="md-nav__link">
<span class="md-ellipsis">
SwarmRearrange
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/agent_rearrange/" class="md-nav__link">
<span class="md-ellipsis">
AgentRearrange
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_6_5_17" >
<label class="md-nav__link" for="__nav_1_6_5_17" id="__nav_1_6_5_17_label" tabindex="0">
<span class="md-ellipsis">
GroupChats
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_1_6_5_17_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_6_5_17">
<span class="md-nav__icon md-icon"></span>
GroupChats
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/structs/group_chat/" class="md-nav__link">
<span class="md-ellipsis">
GroupChat
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/interactive_groupchat/" class="md-nav__link">
<span class="md-ellipsis">
Interactive Group Chat
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_6_5_18" >
<label class="md-nav__link" for="__nav_1_6_5_18" id="__nav_1_6_5_18_label" tabindex="0">
<span class="md-ellipsis">
Workflows
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_1_6_5_18_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_6_5_18">
<span class="md-nav__icon md-icon"></span>
Workflows
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/structs/concurrentworkflow/" class="md-nav__link">
<span class="md-ellipsis">
ConcurrentWorkflow
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/sequential_workflow/" class="md-nav__link">
<span class="md-ellipsis">
SequentialWorkflow
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/graph_workflow/" class="md-nav__link">
<span class="md-ellipsis">
GraphWorkflow
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_6_5_19" >
<label class="md-nav__link" for="__nav_1_6_5_19" id="__nav_1_6_5_19_label" tabindex="0">
<span class="md-ellipsis">
Storage
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_1_6_5_19_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_6_5_19">
<span class="md-nav__icon md-icon"></span>
Storage
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/structs/agent_registry/" class="md-nav__link">
<span class="md-ellipsis">
AgentRegistry
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="../../swarms/structs/conversation/" class="md-nav__link">
<span class="md-ellipsis">
Communication Structure
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_7" >
<label class="md-nav__link" for="__nav_1_7" id="__nav_1_7_label" tabindex="0">
<span class="md-ellipsis">
Tools
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_1_7_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_7">
<span class="md-nav__icon md-icon"></span>
Tools
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_tools/overview/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/tools/base_tool/" class="md-nav__link">
<span class="md-ellipsis">
BaseTool Reference
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/tools/mcp_client_call/" class="md-nav__link">
<span class="md-ellipsis">
MCP Client Utils
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_7_4" >
<label class="md-nav__link" for="__nav_1_7_4" id="__nav_1_7_4_label" tabindex="0">
<span class="md-ellipsis">
Vertical Tools
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="3" aria-labelledby="__nav_1_7_4_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_7_4">
<span class="md-nav__icon md-icon"></span>
Vertical Tools
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_tools/finance/" class="md-nav__link">
<span class="md-ellipsis">
Finance
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_tools/search/" class="md-nav__link">
<span class="md-ellipsis">
Search
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_7_4_3" >
<label class="md-nav__link" for="__nav_1_7_4_3" id="__nav_1_7_4_3_label" tabindex="0">
<span class="md-ellipsis">
Social Media
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_1_7_4_3_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_7_4_3">
<span class="md-nav__icon md-icon"></span>
Social Media
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_tools/twitter/" class="md-nav__link">
<span class="md-ellipsis">
Twitter
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_8" >
<label class="md-nav__link" for="__nav_1_8" id="__nav_1_8_label" tabindex="0">
<span class="md-ellipsis">
Deployment Solutions
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_1_8_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_8">
<span class="md-nav__icon md-icon"></span>
Deployment Solutions
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_cloud/cloud_run/" class="md-nav__link">
<span class="md-ellipsis">
Deploy on Google Cloud Run
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_cloud/phala_deploy/" class="md-nav__link">
<span class="md-ellipsis">
Deploy on Phala
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_1_9" >
<label class="md-nav__link" for="__nav_1_9" id="__nav_1_9_label" tabindex="0">
<span class="md-ellipsis">
More About Us
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_1_9_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_1_9">
<span class="md-nav__icon md-icon"></span>
More About Us
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/ecosystem/" class="md-nav__link">
<span class="md-ellipsis">
Swarms Ecosystem
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/support/" class="md-nav__link">
<span class="md-ellipsis">
Technical Support
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_2" >
<label class="md-nav__link" for="__nav_2" id="__nav_2_label" tabindex="0">
<span class="md-ellipsis">
Examples
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_2_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_2">
<span class="md-nav__icon md-icon"></span>
Examples
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../examples/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/cookbook_index/" class="md-nav__link">
<span class="md-ellipsis">
CookBook Index
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/paper_implementations/" class="md-nav__link">
<span class="md-ellipsis">
Paper Implementations
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../examples/templates/" class="md-nav__link">
<span class="md-ellipsis">
Templates & Applications
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_2_5" >
<label class="md-nav__link" for="__nav_2_5" id="__nav_2_5_label" tabindex="0">
<span class="md-ellipsis">
Basic Examples
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_2_5_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_2_5">
<span class="md-nav__icon md-icon"></span>
Basic Examples
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_2_5_1" >
<label class="md-nav__link" for="__nav_2_5_1" id="__nav_2_5_1_label" tabindex="0">
<span class="md-ellipsis">
Individual Agents
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="3" aria-labelledby="__nav_2_5_1_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_2_5_1">
<span class="md-nav__icon md-icon"></span>
Individual Agents
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/examples/basic_agent/" class="md-nav__link">
<span class="md-ellipsis">
Basic Agent
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_2_5_1_2" >
<label class="md-nav__link" for="__nav_2_5_1_2" id="__nav_2_5_1_2_label" tabindex="0">
<span class="md-ellipsis">
Tool Usage
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_2_5_1_2_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_2_5_1_2">
<span class="md-nav__icon md-icon"></span>
Tool Usage
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/examples/vision_tools/" class="md-nav__link">
<span class="md-ellipsis">
Agents with Vision and Tool Usage
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/agent_with_tools/" class="md-nav__link">
<span class="md-ellipsis">
Agents with Callable Tools
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/agent_structured_outputs/" class="md-nav__link">
<span class="md-ellipsis">
Agent with Structured Outputs
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/agent_with_mcp.md" class="md-nav__link">
<span class="md-ellipsis">
Agent With MCP Integration
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_2_5_1_3" >
<label class="md-nav__link" for="__nav_2_5_1_3" id="__nav_2_5_1_3_label" tabindex="0">
<span class="md-ellipsis">
Vision
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_2_5_1_3_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_2_5_1_3">
<span class="md-nav__icon md-icon"></span>
Vision
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/examples/vision_processing/" class="md-nav__link">
<span class="md-ellipsis">
Agents with Vision
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/multiple_images/" class="md-nav__link">
<span class="md-ellipsis">
Agent with Multiple Images
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_2_5_1_4" >
<label class="md-nav__link" for="__nav_2_5_1_4" id="__nav_2_5_1_4_label" tabindex="0">
<span class="md-ellipsis">
Utils
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_2_5_1_4_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_2_5_1_4">
<span class="md-nav__icon md-icon"></span>
Utils
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../examples/agent_stream/" class="md-nav__link">
<span class="md-ellipsis">
Agent with Streaming
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/agent_output_types/" class="md-nav__link">
<span class="md-ellipsis">
Agent Output Types
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/ui/main/" class="md-nav__link">
<span class="md-ellipsis">
Gradio Chat Interface
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_2_5_1_5" >
<label class="md-nav__link" for="__nav_2_5_1_5" id="__nav_2_5_1_5_label" tabindex="0">
<span class="md-ellipsis">
LLM Providers
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_2_5_1_5_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_2_5_1_5">
<span class="md-nav__icon md-icon"></span>
LLM Providers
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/examples/model_providers/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/openai_example/" class="md-nav__link">
<span class="md-ellipsis">
OpenAI
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/claude/" class="md-nav__link">
<span class="md-ellipsis">
Anthropic
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/groq/" class="md-nav__link">
<span class="md-ellipsis">
Groq
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/cohere/" class="md-nav__link">
<span class="md-ellipsis">
Cohere
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/deepseek/" class="md-nav__link">
<span class="md-ellipsis">
DeepSeek
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/ollama/" class="md-nav__link">
<span class="md-ellipsis">
Ollama
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/openrouter/" class="md-nav__link">
<span class="md-ellipsis">
OpenRouter
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/xai/" class="md-nav__link">
<span class="md-ellipsis">
XAI
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/vllm_integration/" class="md-nav__link">
<span class="md-ellipsis">
VLLM
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/llama4/" class="md-nav__link">
<span class="md-ellipsis">
Llama4
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_2_6" >
<label class="md-nav__link" for="__nav_2_6" id="__nav_2_6_label" tabindex="0">
<span class="md-ellipsis">
Advanced Examples
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_2_6_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_2_6">
<span class="md-nav__icon md-icon"></span>
Advanced Examples
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_2_6_1" >
<label class="md-nav__link" for="__nav_2_6_1" id="__nav_2_6_1_label" tabindex="0">
<span class="md-ellipsis">
Multi-Agent Architectures
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="3" aria-labelledby="__nav_2_6_1_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_2_6_1">
<span class="md-nav__icon md-icon"></span>
Multi-Agent Architectures
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/examples/hierarchical_swarm_example/" class="md-nav__link">
<span class="md-ellipsis">
HierarchicalSwarm Examples
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/board_of_directors_example.md" class="md-nav__link">
<span class="md-ellipsis">
Board of Directors Example
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/hhcs_examples/" class="md-nav__link">
<span class="md-ellipsis">
Hybrid Hierarchical-Cluster Swarm Example
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/groupchat_example/" class="md-nav__link">
<span class="md-ellipsis">
Group Chat Example
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/sequential_example/" class="md-nav__link">
<span class="md-ellipsis">
Sequential Workflow Example
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/swarm_router/" class="md-nav__link">
<span class="md-ellipsis">
SwarmRouter Example
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/multi_agent_router_minimal/" class="md-nav__link">
<span class="md-ellipsis">
MultiAgentRouter Minimal Example
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/concurrent_workflow/" class="md-nav__link">
<span class="md-ellipsis">
ConcurrentWorkflow Example
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/moa_example/" class="md-nav__link">
<span class="md-ellipsis">
Mixture of Agents Example
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/unique_swarms/" class="md-nav__link">
<span class="md-ellipsis">
Unique Swarms
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/agents_as_tools/" class="md-nav__link">
<span class="md-ellipsis">
Agents as Tools
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/aggregate/" class="md-nav__link">
<span class="md-ellipsis">
Aggregate Multi-Agent Responses
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/igc_example/" class="md-nav__link">
<span class="md-ellipsis">
Interactive GroupChat Example
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_2_6_1_14" >
<label class="md-nav__link" for="__nav_2_6_1_14" id="__nav_2_6_1_14_label" tabindex="0">
<span class="md-ellipsis">
Applications
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_2_6_1_14_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_2_6_1_14">
<span class="md-nav__icon md-icon"></span>
Applications
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/examples/swarms_dao/" class="md-nav__link">
<span class="md-ellipsis">
Swarms DAO
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/swarms_of_browser_agents/" class="md-nav__link">
<span class="md-ellipsis">
Swarms of Browser Agents
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/vllm/" class="md-nav__link">
<span class="md-ellipsis">
ConcurrentWorkflow with VLLM Agents
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_3" >
<label class="md-nav__link" for="__nav_3" id="__nav_3_label" tabindex="0">
<span class="md-ellipsis">
Swarms Cloud API
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_3_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_3">
<span class="md-nav__icon md-icon"></span>
Swarms Cloud API
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_cloud/swarms_api/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_cloud/quickstart/" class="md-nav__link">
<span class="md-ellipsis">
Quickstart
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_cloud/mcp/" class="md-nav__link">
<span class="md-ellipsis">
MCP Server
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_cloud/rate_limits/" class="md-nav__link">
<span class="md-ellipsis">
Rate Limits
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_cloud/best_practices/" class="md-nav__link">
<span class="md-ellipsis">
Best Practices
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_3_6" >
<label class="md-nav__link" for="__nav_3_6" id="__nav_3_6_label" tabindex="0">
<span class="md-ellipsis">
Capabilities
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_3_6_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_3_6">
<span class="md-nav__icon md-icon"></span>
Capabilities
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_3_6_1" >
<label class="md-nav__link" for="__nav_3_6_1" id="__nav_3_6_1_label" tabindex="0">
<span class="md-ellipsis">
Agents
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="3" aria-labelledby="__nav_3_6_1_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_3_6_1">
<span class="md-nav__icon md-icon"></span>
Agents
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_cloud/agent_api/" class="md-nav__link">
<span class="md-ellipsis">
Individual Agent Completions
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_cloud/swarms_api_tools/" class="md-nav__link">
<span class="md-ellipsis">
Tools
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_3_6_2" >
<label class="md-nav__link" for="__nav_3_6_2" id="__nav_3_6_2_label" tabindex="0">
<span class="md-ellipsis">
Multi-Agent
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="3" aria-labelledby="__nav_3_6_2_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_3_6_2">
<span class="md-nav__icon md-icon"></span>
Multi-Agent
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_cloud/swarm_types/" class="md-nav__link">
<span class="md-ellipsis">
Multi Agent Architectures Available
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_3_6_2_2" >
<label class="md-nav__link" for="__nav_3_6_2_2" id="__nav_3_6_2_2_label" tabindex="0">
<span class="md-ellipsis">
Examples
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="4" aria-labelledby="__nav_3_6_2_2_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_3_6_2_2">
<span class="md-nav__icon md-icon"></span>
Examples
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/examples/swarms_api_medical/" class="md-nav__link">
<span class="md-ellipsis">
Medical Swarm
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/examples/swarms_api_finance/" class="md-nav__link">
<span class="md-ellipsis">
Finance Swarm
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_3_7" >
<label class="md-nav__link" for="__nav_3_7" id="__nav_3_7_label" tabindex="0">
<span class="md-ellipsis">
Clients
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_3_7_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_3_7">
<span class="md-nav__icon md-icon"></span>
Clients
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_cloud/api_clients/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_cloud/python_client/" class="md-nav__link">
<span class="md-ellipsis">
Python Client
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_cloud/rust_client/" class="md-nav__link">
<span class="md-ellipsis">
Rust Client
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_3_8" >
<label class="md-nav__link" for="__nav_3_8" id="__nav_3_8_label" tabindex="0">
<span class="md-ellipsis">
Pricing
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_3_8_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_3_8">
<span class="md-nav__icon md-icon"></span>
Pricing
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_cloud/api_pricing/" class="md-nav__link">
<span class="md-ellipsis">
Pricing
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_cloud/subscription_tiers/" class="md-nav__link">
<span class="md-ellipsis">
Subscription Tiers
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_4" >
<label class="md-nav__link" for="__nav_4" id="__nav_4_label" tabindex="0">
<span class="md-ellipsis">
Swarms Marketplace
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_4_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_4">
<span class="md-nav__icon md-icon"></span>
Swarms Marketplace
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_platform/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_4_2" >
<label class="md-nav__link" for="__nav_4_2" id="__nav_4_2_label" tabindex="0">
<span class="md-ellipsis">
Marketplace
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_4_2_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_4_2">
<span class="md-nav__icon md-icon"></span>
Marketplace
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_platform/share_and_discover/" class="md-nav__link">
<span class="md-ellipsis">
Share and Discover Agents, Prompts, and Tools
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_platform/monetize/" class="md-nav__link">
<span class="md-ellipsis">
Monetize Your Prompts, Agents, and Tools
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_4_3" >
<label class="md-nav__link" for="__nav_4_3" id="__nav_4_3_label" tabindex="0">
<span class="md-ellipsis">
Platform
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_4_3_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_4_3">
<span class="md-nav__icon md-icon"></span>
Platform
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_platform/apps_page/" class="md-nav__link">
<span class="md-ellipsis">
Customize Your Sidebar
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_platform/playground_page/" class="md-nav__link">
<span class="md-ellipsis">
Playground
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_platform/apikeys/" class="md-nav__link">
<span class="md-ellipsis">
API Key Management
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_platform/account_management/" class="md-nav__link">
<span class="md-ellipsis">
Account Management
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_5" >
<label class="md-nav__link" for="__nav_5" id="__nav_5_label" tabindex="0">
<span class="md-ellipsis">
Swarms Rust
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_5_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_5">
<span class="md-nav__icon md-icon"></span>
Swarms Rust
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms_rs/overview/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms_rs/agents/" class="md-nav__link">
<span class="md-ellipsis">
Agents
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_6" >
<label class="md-nav__link" for="__nav_6" id="__nav_6_label" tabindex="0">
<span class="md-ellipsis">
Contributors
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_6_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_6">
<span class="md-nav__icon md-icon"></span>
Contributors
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../contributors/main/" class="md-nav__link">
<span class="md-ellipsis">
Overview
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../contributors/environment_setup/" class="md-nav__link">
<span class="md-ellipsis">
Environment Setup
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../corporate/bounty_program/" class="md-nav__link">
<span class="md-ellipsis">
Bounty Program
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_6_4" >
<label class="md-nav__link" for="__nav_6_4" id="__nav_6_4_label" tabindex="0">
<span class="md-ellipsis">
Development Guides
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_6_4_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_6_4">
<span class="md-nav__icon md-icon"></span>
Development Guides
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/framework/code_cleanliness/" class="md-nav__link">
<span class="md-ellipsis">
Code Style Guide & Best Practices
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/framework/test/" class="md-nav__link">
<span class="md-ellipsis">
Adding Tests
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../contributors/docs/" class="md-nav__link">
<span class="md-ellipsis">
Adding Documentation
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../contributors/tools/" class="md-nav__link">
<span class="md-ellipsis">
New Tools & Plugins
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_6_5" >
<label class="md-nav__link" for="__nav_6_5" id="__nav_6_5_label" tabindex="0">
<span class="md-ellipsis">
Architecture & Design
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="2" aria-labelledby="__nav_6_5_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_6_5">
<span class="md-nav__icon md-icon"></span>
Architecture & Design
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/concept/framework_architecture/" class="md-nav__link">
<span class="md-ellipsis">
Understanding Swarms Architecture
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/concept/philosophy/" class="md-nav__link">
<span class="md-ellipsis">
Development Philosophy & Principles
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_7" >
<label class="md-nav__link" for="__nav_7" id="__nav_7_label" tabindex="0">
<span class="md-ellipsis">
About Swarms
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_7_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_7">
<span class="md-nav__icon md-icon"></span>
About Swarms
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../swarms/concept/vision/" class="md-nav__link">
<span class="md-ellipsis">
Vision & Mission
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/concept/swarm_ecosystem/" class="md-nav__link">
<span class="md-ellipsis">
Swarm Ecosystem
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../swarms/products/" class="md-nav__link">
<span class="md-ellipsis">
Products
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#understanding-agent-evaluation-mechanisms" class="md-nav__link">
<span class="md-ellipsis">
Understanding Agent Evaluation Mechanisms
</span>
</a>
<nav class="md-nav" aria-label="Understanding Agent Evaluation Mechanisms">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#1-introduction-to-agent-evaluation-mechanisms" class="md-nav__link">
<span class="md-ellipsis">
1. Introduction to Agent Evaluation Mechanisms
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#2-key-components-of-agent-evaluation" class="md-nav__link">
<span class="md-ellipsis">
2. Key Components of Agent Evaluation
</span>
</a>
<nav class="md-nav" aria-label="2. Key Components of Agent Evaluation">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#a-performance-metrics" class="md-nav__link">
<span class="md-ellipsis">
a. Performance Metrics
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#b-evaluation-criteria" class="md-nav__link">
<span class="md-ellipsis">
b. Evaluation Criteria
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#3-the-process-of-agent-evaluation" class="md-nav__link">
<span class="md-ellipsis">
3. The Process of Agent Evaluation
</span>
</a>
<nav class="md-nav" aria-label="3. The Process of Agent Evaluation">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#a-define-evaluation-metrics" class="md-nav__link">
<span class="md-ellipsis">
a. Define Evaluation Metrics
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#b-collect-data" class="md-nav__link">
<span class="md-ellipsis">
b. Collect Data
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#c-analyze-performance" class="md-nav__link">
<span class="md-ellipsis">
c. Analyze Performance
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#d-generate-reports" class="md-nav__link">
<span class="md-ellipsis">
d. Generate Reports
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#4-tracking-agent-accuracy" class="md-nav__link">
<span class="md-ellipsis">
4. Tracking Agent Accuracy
</span>
</a>
<nav class="md-nav" aria-label="4. Tracking Agent Accuracy">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#a-define-correctness-criteria" class="md-nav__link">
<span class="md-ellipsis">
a. Define Correctness Criteria
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#b-monitor-agent-actions" class="md-nav__link">
<span class="md-ellipsis">
b. Monitor Agent Actions
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#c-compare-against-correctness-criteria" class="md-nav__link">
<span class="md-ellipsis">
c. Compare Against Correctness Criteria
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#d-calculate-accuracy-metrics" class="md-nav__link">
<span class="md-ellipsis">
d. Calculate Accuracy Metrics
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#5-measuring-agent-accuracy" class="md-nav__link">
<span class="md-ellipsis">
5. Measuring Agent Accuracy
</span>
</a>
<nav class="md-nav" aria-label="5. Measuring Agent Accuracy">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#a-data-labeling" class="md-nav__link">
<span class="md-ellipsis">
a. Data Labeling
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#b-establish-baseline-performance" class="md-nav__link">
<span class="md-ellipsis">
b. Establish Baseline Performance
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#c-regular-evaluations" class="md-nav__link">
<span class="md-ellipsis">
c. Regular Evaluations
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#d-feedback-and-improvement" class="md-nav__link">
<span class="md-ellipsis">
d. Feedback and Improvement
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#6-visualizing-agent-evaluation-with-mermaid-graphs" class="md-nav__link">
<span class="md-ellipsis">
6. Visualizing Agent Evaluation with Mermaid Graphs
</span>
</a>
<nav class="md-nav" aria-label="6. Visualizing Agent Evaluation with Mermaid Graphs">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#a-overall-evaluation-process" class="md-nav__link">
<span class="md-ellipsis">
a. Overall Evaluation Process
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#b-accuracy-tracking" class="md-nav__link">
<span class="md-ellipsis">
b. Accuracy Tracking
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#c-continuous-improvement-cycle" class="md-nav__link">
<span class="md-ellipsis">
c. Continuous Improvement Cycle
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#7-case-study-evaluating-a-chatbot-agent" class="md-nav__link">
<span class="md-ellipsis">
7. Case Study: Evaluating a Chatbot Agent
</span>
</a>
<nav class="md-nav" aria-label="7. Case Study: Evaluating a Chatbot Agent">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#a-define-evaluation-metrics_1" class="md-nav__link">
<span class="md-ellipsis">
a. Define Evaluation Metrics
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#b-collect-data_1" class="md-nav__link">
<span class="md-ellipsis">
b. Collect Data
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#c-analyze-performance_1" class="md-nav__link">
<span class="md-ellipsis">
c. Analyze Performance
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#d-generate-reports_1" class="md-nav__link">
<span class="md-ellipsis">
d. Generate Reports
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#8-best-practices-for-agent-evaluation" class="md-nav__link">
<span class="md-ellipsis">
8. Best Practices for Agent Evaluation
</span>
</a>
<nav class="md-nav" aria-label="8. Best Practices for Agent Evaluation">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#a-use-realistic-scenarios" class="md-nav__link">
<span class="md-ellipsis">
a. Use Realistic Scenarios
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#b-continuous-monitoring" class="md-nav__link">
<span class="md-ellipsis">
b. Continuous Monitoring
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#c-incorporate-user-feedback" class="md-nav__link">
<span class="md-ellipsis">
c. Incorporate User Feedback
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#d-regular-updates" class="md-nav__link">
<span class="md-ellipsis">
d. Regular Updates
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#conclusion" class="md-nav__link">
<span class="md-ellipsis">
Conclusion
</span>
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-content" data-md-component="content">
<article class="md-content__inner md-typeset">
<h1>Agent evals</h1>
<h3 id="understanding-agent-evaluation-mechanisms">Understanding Agent Evaluation Mechanisms<a class="headerlink" href="#understanding-agent-evaluation-mechanisms" title="Permanent link">&para;</a></h3>
<p>Agent evaluation mechanisms play a crucial role in ensuring that autonomous agents, particularly in multi-agent systems, perform their tasks effectively and efficiently. This blog delves into the intricacies of agent evaluation, the importance of accuracy tracking, and the methodologies used to measure and visualize agent performance. We'll use Mermaid graphs to provide clear visual representations of these processes.</p>
<h4 id="1-introduction-to-agent-evaluation-mechanisms">1. Introduction to Agent Evaluation Mechanisms<a class="headerlink" href="#1-introduction-to-agent-evaluation-mechanisms" title="Permanent link">&para;</a></h4>
<p>Agent evaluation mechanisms refer to the processes and criteria used to assess the performance of agents within a system. These mechanisms are essential for:</p>
<ul>
<li><strong>Ensuring Reliability:</strong> Agents must consistently perform their designated tasks correctly.</li>
<li><strong>Improving Performance:</strong> Evaluation helps in identifying areas where agents can improve.</li>
<li><strong>Maintaining Accountability:</strong> It provides a way to hold agents accountable for their actions.</li>
</ul>
<h3 id="2-key-components-of-agent-evaluation">2. Key Components of Agent Evaluation<a class="headerlink" href="#2-key-components-of-agent-evaluation" title="Permanent link">&para;</a></h3>
<p>To effectively evaluate agents, several components and metrics are considered:</p>
<h4 id="a-performance-metrics">a. Performance Metrics<a class="headerlink" href="#a-performance-metrics" title="Permanent link">&para;</a></h4>
<p>These are quantitative measures used to assess how well an agent is performing. Common performance metrics include:</p>
<ul>
<li><strong>Accuracy:</strong> The percentage of correct actions or decisions made by the agent.</li>
<li><strong>Precision and Recall:</strong> Precision measures the number of true positive results divided by the number of all positive results, while recall measures the number of true positive results divided by the number of positives that should have been retrieved.</li>
<li><strong>F1 Score:</strong> The harmonic mean of precision and recall.</li>
<li><strong>Response Time:</strong> How quickly an agent responds to a given task or query.</li>
</ul>
<h4 id="b-evaluation-criteria">b. Evaluation Criteria<a class="headerlink" href="#b-evaluation-criteria" title="Permanent link">&para;</a></h4>
<p>Evaluation criteria define the standards or benchmarks against which agent performance is measured. These criteria are often task-specific and may include:</p>
<ul>
<li><strong>Task Completion Rate:</strong> The percentage of tasks successfully completed by the agent.</li>
<li><strong>Error Rate:</strong> The frequency of errors made by the agent during task execution.</li>
<li><strong>Resource Utilization:</strong> How efficiently an agent uses resources such as memory and CPU.</li>
</ul>
<h3 id="3-the-process-of-agent-evaluation">3. The Process of Agent Evaluation<a class="headerlink" href="#3-the-process-of-agent-evaluation" title="Permanent link">&para;</a></h3>
<p>The evaluation process involves several steps, which can be visualized using Mermaid graphs:</p>
<h4 id="a-define-evaluation-metrics">a. Define Evaluation Metrics<a class="headerlink" href="#a-define-evaluation-metrics" title="Permanent link">&para;</a></h4>
<p>The first step is to define the metrics that will be used to evaluate the agent. This involves identifying the key performance indicators (KPIs) relevant to the agent's tasks.</p>
<pre class="mermaid"><code>graph TD
A[Define Evaluation Metrics] --&gt; B[Identify KPIs]
B --&gt; C[Accuracy]
B --&gt; D[Precision and Recall]
B --&gt; E[F1 Score]
B --&gt; F[Response Time]</code></pre>
<h4 id="b-collect-data">b. Collect Data<a class="headerlink" href="#b-collect-data" title="Permanent link">&para;</a></h4>
<p>Data collection involves gathering information on the agent's performance. This data can come from logs, user feedback, or direct observations.</p>
<pre class="mermaid"><code>graph TD
A[Collect Data] --&gt; B[Logs]
A --&gt; C[User Feedback]
A --&gt; D[Direct Observations]</code></pre>
<h4 id="c-analyze-performance">c. Analyze Performance<a class="headerlink" href="#c-analyze-performance" title="Permanent link">&para;</a></h4>
<p>Once data is collected, it is analyzed to assess the agent's performance against the defined metrics. This step may involve statistical analysis, machine learning models, or other analytical techniques.</p>
<pre class="mermaid"><code>graph TD
A[Analyze Performance] --&gt; B[Statistical Analysis]
A --&gt; C[Machine Learning Models]
A --&gt; D[Other Analytical Techniques]</code></pre>
<h4 id="d-generate-reports">d. Generate Reports<a class="headerlink" href="#d-generate-reports" title="Permanent link">&para;</a></h4>
<p>After analysis, performance reports are generated. These reports provide insights into how well the agent is performing and identify areas for improvement.</p>
<pre class="mermaid"><code>graph TD
A[Generate Reports] --&gt; B[Performance Insights]
B --&gt; C[Identify Areas for Improvement]</code></pre>
<h3 id="4-tracking-agent-accuracy">4. Tracking Agent Accuracy<a class="headerlink" href="#4-tracking-agent-accuracy" title="Permanent link">&para;</a></h3>
<p>Accuracy tracking is a critical aspect of agent evaluation. It involves measuring how often an agent's actions or decisions are correct. The following steps outline the process of tracking agent accuracy:</p>
<h4 id="a-define-correctness-criteria">a. Define Correctness Criteria<a class="headerlink" href="#a-define-correctness-criteria" title="Permanent link">&para;</a></h4>
<p>The first step is to define what constitutes a correct action or decision for the agent.</p>
<pre class="mermaid"><code>graph TD
A[Define Correctness Criteria] --&gt; B[Task-Specific Standards]
B --&gt; C[Action Accuracy]
B --&gt; D[Decision Accuracy]</code></pre>
<h4 id="b-monitor-agent-actions">b. Monitor Agent Actions<a class="headerlink" href="#b-monitor-agent-actions" title="Permanent link">&para;</a></h4>
<p>Agents' actions are continuously monitored to track their performance. This monitoring can be done in real-time or through periodic evaluations.</p>
<pre class="mermaid"><code>graph TD
A[Monitor Agent Actions] --&gt; B[Real-Time Monitoring]
A --&gt; C[Periodic Evaluations]</code></pre>
<h4 id="c-compare-against-correctness-criteria">c. Compare Against Correctness Criteria<a class="headerlink" href="#c-compare-against-correctness-criteria" title="Permanent link">&para;</a></h4>
<p>Each action or decision made by the agent is compared against the defined correctness criteria to determine its accuracy.</p>
<pre class="mermaid"><code>graph TD
A[Compare Against Correctness Criteria] --&gt; B[Evaluate Each Action]
B --&gt; C[Correct or Incorrect?]</code></pre>
<h4 id="d-calculate-accuracy-metrics">d. Calculate Accuracy Metrics<a class="headerlink" href="#d-calculate-accuracy-metrics" title="Permanent link">&para;</a></h4>
<p>Accuracy metrics are calculated based on the comparison results. These metrics provide a quantitative measure of the agent's accuracy.</p>
<pre class="mermaid"><code>graph TD
A[Calculate Accuracy Metrics] --&gt; B[Accuracy Percentage]
A --&gt; C[Error Rate]</code></pre>
<h3 id="5-measuring-agent-accuracy">5. Measuring Agent Accuracy<a class="headerlink" href="#5-measuring-agent-accuracy" title="Permanent link">&para;</a></h3>
<p>Measuring agent accuracy involves several steps and considerations:</p>
<h4 id="a-data-labeling">a. Data Labeling<a class="headerlink" href="#a-data-labeling" title="Permanent link">&para;</a></h4>
<p>To measure accuracy, the data used for evaluation must be accurately labeled. This involves annotating the data with the correct actions or decisions.</p>
<pre class="mermaid"><code>graph TD
A[Data Labeling] --&gt; B[Annotate Data with Correct Actions]
B --&gt; C[Ensure Accuracy of Labels]</code></pre>
<h4 id="b-establish-baseline-performance">b. Establish Baseline Performance<a class="headerlink" href="#b-establish-baseline-performance" title="Permanent link">&para;</a></h4>
<p>A baseline performance level is established by evaluating a sample set of data. This baseline serves as a reference point for measuring improvements or declines in accuracy.</p>
<pre class="mermaid"><code>graph TD
A[Establish Baseline Performance] --&gt; B[Evaluate Sample Data]
B --&gt; C[Set Performance Benchmarks]</code></pre>
<h4 id="c-regular-evaluations">c. Regular Evaluations<a class="headerlink" href="#c-regular-evaluations" title="Permanent link">&para;</a></h4>
<p>Agents are regularly evaluated to measure their accuracy over time. This helps in tracking performance trends and identifying any deviations from the expected behavior.</p>
<pre class="mermaid"><code>graph TD
A[Regular Evaluations] --&gt; B[Track Performance Over Time]
B --&gt; C[Identify Performance Trends]
B --&gt; D[Detect Deviations]</code></pre>
<h4 id="d-feedback-and-improvement">d. Feedback and Improvement<a class="headerlink" href="#d-feedback-and-improvement" title="Permanent link">&para;</a></h4>
<p>Feedback from evaluations is used to improve the agent's performance. This may involve retraining the agent, adjusting its algorithms, or refining its decision-making processes.</p>
<pre class="mermaid"><code>graph TD
A[Feedback and Improvement] --&gt; B[Use Evaluation Feedback]
B --&gt; C[Retrain Agent]
B --&gt; D[Adjust Algorithms]
B --&gt; E[Refine Decision-Making Processes]</code></pre>
<h3 id="6-visualizing-agent-evaluation-with-mermaid-graphs">6. Visualizing Agent Evaluation with Mermaid Graphs<a class="headerlink" href="#6-visualizing-agent-evaluation-with-mermaid-graphs" title="Permanent link">&para;</a></h3>
<p>Mermaid graphs provide a clear and concise way to visualize the agent evaluation process. Here are some examples of how Mermaid graphs can be used:</p>
<h4 id="a-overall-evaluation-process">a. Overall Evaluation Process<a class="headerlink" href="#a-overall-evaluation-process" title="Permanent link">&para;</a></h4>
<pre class="mermaid"><code>graph TD
A[Define Evaluation Metrics] --&gt; B[Collect Data]
B --&gt; C[Analyze Performance]
C --&gt; D[Generate Reports]</code></pre>
<h4 id="b-accuracy-tracking">b. Accuracy Tracking<a class="headerlink" href="#b-accuracy-tracking" title="Permanent link">&para;</a></h4>
<pre class="mermaid"><code>graph TD
A[Define Correctness Criteria] --&gt; B[Monitor Agent Actions]
B --&gt; C[Compare Against Correctness Criteria]
C --&gt; D[Calculate Accuracy Metrics]</code></pre>
<h4 id="c-continuous-improvement-cycle">c. Continuous Improvement Cycle<a class="headerlink" href="#c-continuous-improvement-cycle" title="Permanent link">&para;</a></h4>
<pre class="mermaid"><code>graph TD
A[Regular Evaluations] --&gt; B[Track Performance Over Time]
B --&gt; C[Identify Performance Trends]
C --&gt; D[Detect Deviations]
D --&gt; E[Feedback and Improvement]
E --&gt; A</code></pre>
<h3 id="7-case-study-evaluating-a-chatbot-agent">7. Case Study: Evaluating a Chatbot Agent<a class="headerlink" href="#7-case-study-evaluating-a-chatbot-agent" title="Permanent link">&para;</a></h3>
<p>To illustrate the agent evaluation process, let's consider a case study involving a chatbot agent designed to assist customers in an e-commerce platform.</p>
<h4 id="a-define-evaluation-metrics_1">a. Define Evaluation Metrics<a class="headerlink" href="#a-define-evaluation-metrics_1" title="Permanent link">&para;</a></h4>
<p>For the chatbot, key performance metrics might include:</p>
<ul>
<li><strong>Response Accuracy:</strong> The percentage of correct responses provided by the chatbot.</li>
<li><strong>Response Time:</strong> The average time taken by the chatbot to respond to user queries.</li>
<li><strong>Customer Satisfaction:</strong> Measured through user feedback and ratings.</li>
</ul>
<h4 id="b-collect-data_1">b. Collect Data<a class="headerlink" href="#b-collect-data_1" title="Permanent link">&para;</a></h4>
<p>Data is collected from chatbot interactions, including user queries, responses, and feedback.</p>
<h4 id="c-analyze-performance_1">c. Analyze Performance<a class="headerlink" href="#c-analyze-performance_1" title="Permanent link">&para;</a></h4>
<p>Performance analysis involves comparing the chatbot's responses against a predefined set of correct responses and calculating accuracy metrics.</p>
<h4 id="d-generate-reports_1">d. Generate Reports<a class="headerlink" href="#d-generate-reports_1" title="Permanent link">&para;</a></h4>
<p>Reports are generated to provide insights into the chatbot's performance, highlighting areas where it excels and areas needing improvement.</p>
<h3 id="8-best-practices-for-agent-evaluation">8. Best Practices for Agent Evaluation<a class="headerlink" href="#8-best-practices-for-agent-evaluation" title="Permanent link">&para;</a></h3>
<p>Here are some best practices to ensure effective agent evaluation:</p>
<h4 id="a-use-realistic-scenarios">a. Use Realistic Scenarios<a class="headerlink" href="#a-use-realistic-scenarios" title="Permanent link">&para;</a></h4>
<p>Evaluate agents in realistic scenarios that closely mimic real-world conditions. This ensures that the evaluation results are relevant and applicable.</p>
<h4 id="b-continuous-monitoring">b. Continuous Monitoring<a class="headerlink" href="#b-continuous-monitoring" title="Permanent link">&para;</a></h4>
<p>Continuously monitor agent performance to detect and address issues promptly. This helps in maintaining high performance levels.</p>
<h4 id="c-incorporate-user-feedback">c. Incorporate User Feedback<a class="headerlink" href="#c-incorporate-user-feedback" title="Permanent link">&para;</a></h4>
<p>User feedback is invaluable for improving agent performance. Incorporate feedback into the evaluation process to identify and rectify shortcomings.</p>
<h4 id="d-regular-updates">d. Regular Updates<a class="headerlink" href="#d-regular-updates" title="Permanent link">&para;</a></h4>
<p>Regularly update the evaluation metrics and criteria to keep pace with evolving tasks and requirements.</p>
<h3 id="conclusion">Conclusion<a class="headerlink" href="#conclusion" title="Permanent link">&para;</a></h3>
<p>Agent evaluation mechanisms are vital for ensuring the reliability, efficiency, and effectiveness of autonomous agents. By defining clear evaluation metrics, continuously monitoring performance, and using feedback for improvement, we can develop agents that consistently perform at high levels. Visualizing the evaluation process with tools like Mermaid graphs further aids in understanding and communication. Through diligent evaluation and continuous improvement, we can harness the full potential of autonomous agents in various applications.</p>
<aside class="md-source-file">
<span class="md-source-file__fact">
<span class="md-icon" title="Last update">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M21 13.1c-.1 0-.3.1-.4.2l-1 1 2.1 2.1 1-1c.2-.2.2-.6 0-.8l-1.3-1.3c-.1-.1-.2-.2-.4-.2m-1.9 1.8-6.1 6V23h2.1l6.1-6.1zM12.5 7v5.2l4 2.4-1 1L11 13V7zM11 21.9c-5.1-.5-9-4.8-9-9.9C2 6.5 6.5 2 12 2c5.3 0 9.6 4.1 10 9.3-.3-.1-.6-.2-1-.2s-.7.1-1 .2C19.6 7.2 16.2 4 12 4c-4.4 0-8 3.6-8 8 0 4.1 3.1 7.5 7.1 7.9l-.1.2z"/></svg>
</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="July 26, 2025 08:54:32 UTC">July 26, 2025</span>
</span>
<span class="md-source-file__fact">
<span class="md-icon" title="Created">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M14.47 15.08 11 13V7h1.5v5.25l3.08 1.83c-.41.28-.79.62-1.11 1m-1.39 4.84c-.36.05-.71.08-1.08.08-4.42 0-8-3.58-8-8s3.58-8 8-8 8 3.58 8 8c0 .37-.03.72-.08 1.08.69.1 1.33.32 1.92.64.1-.56.16-1.13.16-1.72 0-5.5-4.5-10-10-10S2 6.5 2 12s4.47 10 10 10c.59 0 1.16-.06 1.72-.16-.32-.59-.54-1.23-.64-1.92M18 15v3h-3v2h3v3h2v-3h3v-2h-3v-3z"/></svg>
</span>
<span class="git-revision-date-localized-plugin git-revision-date-localized-plugin-date" title="July 26, 2025 08:54:32 UTC">July 26, 2025</span>
</span>
<span class="md-source-file__fact">
<span class="md-icon" title="Contributors">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 5.5A3.5 3.5 0 0 1 15.5 9a3.5 3.5 0 0 1-3.5 3.5A3.5 3.5 0 0 1 8.5 9 3.5 3.5 0 0 1 12 5.5M5 8c.56 0 1.08.15 1.53.42-.15 1.43.27 2.85 1.13 3.96C7.16 13.34 6.16 14 5 14a3 3 0 0 1-3-3 3 3 0 0 1 3-3m14 0a3 3 0 0 1 3 3 3 3 0 0 1-3 3c-1.16 0-2.16-.66-2.66-1.62a5.54 5.54 0 0 0 1.13-3.96c.45-.27.97-.42 1.53-.42M5.5 18.25c0-2.07 2.91-3.75 6.5-3.75s6.5 1.68 6.5 3.75V20h-13zM0 20v-1.5c0-1.39 1.89-2.56 4.45-2.9-.59.68-.95 1.62-.95 2.65V20zm24 0h-3.5v-1.75c0-1.03-.36-1.97-.95-2.65 2.56.34 4.45 1.51 4.45 2.9z"/></svg>
</span>
<nav>
</nav>
</span>
</aside>
</article>
</div>
<script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script>
</div>
<button type="button" class="md-top md-icon" data-md-component="top" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg>
Back to top
</button>
</main>
<footer class="md-footer">
<!-- Custom Footer Links Section -->
<div class="md-footer-custom">
<div class="md-footer-custom__inner md-grid">
<div class="md-footer-links">
<div class="md-footer-links__section">
<h4 class="md-footer-links__title">Getting Started</h4>
<ul class="md-footer-links__list">
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/swarms/install/install/" class="md-footer-links__link">
Installation
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/quickstart/" class="md-footer-links__link">
Quickstart
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/swarms/install/env/" class="md-footer-links__link">
Environment Setup
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/swarms/examples/basic_agent/" class="md-footer-links__link">
Basic Agent Example
</a>
</li>
</ul>
</div>
<div class="md-footer-links__section">
<h4 class="md-footer-links__title">Core Capabilities</h4>
<ul class="md-footer-links__list">
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/swarms/structs/agent/" class="md-footer-links__link">
Agents
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/swarms/tools/tools_examples/" class="md-footer-links__link">
Tools and MCP
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/swarms/concept/swarm_architectures/" class="md-footer-links__link">
Multi-Agent Architectures
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/swarms/structs/sequential_workflow/" class="md-footer-links__link">
Sequential Workflow
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/swarms/structs/concurrentworkflow/" class="md-footer-links__link">
Concurrent Workflow
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/swarms/structs/hierarchical_swarm/" class="md-footer-links__link">
Hierarchical Swarm
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/swarms/structs/swarm_router/" class="md-footer-links__link">
Swarm Router
</a>
</li>
</ul>
</div>
<div class="md-footer-links__section">
<h4 class="md-footer-links__title">Templates & Applications</h4>
<ul class="md-footer-links__list">
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/examples/index/" class="md-footer-links__link">
Examples Overview
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/examples/cookbook_index/" class="md-footer-links__link">
Cookbook
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/examples/templates/" class="md-footer-links__link">
Templates
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/examples/paper_implementations/" class="md-footer-links__link">
Paper Implementations
</a>
</li>
</ul>
</div>
<div class="md-footer-links__section">
<h4 class="md-footer-links__title">Contributors</h4>
<ul class="md-footer-links__list">
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/contributors/main/" class="md-footer-links__link">
Contributing
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/swarms/framework/code_cleanliness/" class="md-footer-links__link">
Code Style Guide
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/contributors/docs/" class="md-footer-links__link">
Adding Documentation
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/corporate/bounty_program/" class="md-footer-links__link">
Bounty Program
</a>
</li>
<li class="md-footer-links__item">
<a href="https://docs.swarms.world/en/latest/swarms/support/" class="md-footer-links__link">
Support
</a>
</li>
</ul>
</div>
<div class="md-footer-links__section">
<h4 class="md-footer-links__title">Community</h4>
<ul class="md-footer-links__list">
<li class="md-footer-links__item">
<a href="https://twitter.com/swarms_corp" class="md-footer-links__link">
Twitter
</a>
</li>
<li class="md-footer-links__item">
<a href="https://discord.gg/jM3Z6M9uMq" class="md-footer-links__link">
Discord
</a>
</li>
<li class="md-footer-links__item">
<a href="https://www.youtube.com/channel/UC9yXyitkbU_WSy7bd_41SqQ" class="md-footer-links__link">
YouTube
</a>
</li>
<li class="md-footer-links__item">
<a href="https://www.linkedin.com/company/the-swarm-corporation" class="md-footer-links__link">
LinkedIn
</a>
</li>
<li class="md-footer-links__item">
<a href="https://medium.com/@kyeg" class="md-footer-links__link">
Blog
</a>
</li>
<li class="md-footer-links__item">
<a href="https://lu.ma/5p2jnc2v" class="md-footer-links__link">
Events
</a>
</li>
<li class="md-footer-links__item">
<a href="https://cal.com/swarms/swarms-onboarding-session" class="md-footer-links__link">
Onboarding Session
</a>
</li>
</ul>
</div>
</div>
</div>
</div>
<!-- Company Information Section -->
<div class="md-footer-company">
<div class="md-footer-company__inner md-grid">
<div class="md-footer-company__content">
<div class="md-footer-company__brand">
<h3 class="md-footer-company__name">Swarms</h3>
<p class="md-footer-company__description">
Automating the world economy with multi-agent collaboration
</p>
</div>
<div class="md-footer-company__copyright">
<p>&copy; 2024 Swarms. All rights reserved.</p>
</div>
</div>
</div>
</div>
<!-- Original Material Footer -->
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
<div class="md-copyright__highlight">
TGSC Corp 2024. All rights reserved.
</div>
Made with
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs
</a>
</div>
<div class="md-social">
<a href="https://x.com/swarms_corp" target="_blank" rel="noopener" title="x.com" class="md-social__link">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!--! Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2024 Fonticons, Inc.--><path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253"/></svg>
</a>
<a href="https://github.com/kyegomez/swarms" target="_blank" rel="noopener" title="github.com" class="md-social__link">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 496 512"><!--! Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2024 Fonticons, Inc.--><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6m-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3m44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9M244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8M97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1m-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7m32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1m-11.4-14.7c-1.6 1-1.6 3.6 0 5.9s4.3 3.3 5.6 2.3c1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2"/></svg>
</a>
<a href="https://discord.gg/jM3Z6M9uMq" target="_blank" rel="noopener" title="discord.gg" class="md-social__link">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2024 Fonticons, Inc.--><path d="M524.531 69.836a1.5 1.5 0 0 0-.764-.7A485 485 0 0 0 404.081 32.03a1.82 1.82 0 0 0-1.923.91 338 338 0 0 0-14.9 30.6 447.9 447.9 0 0 0-134.426 0 310 310 0 0 0-15.135-30.6 1.89 1.89 0 0 0-1.924-.91 483.7 483.7 0 0 0-119.688 37.107 1.7 1.7 0 0 0-.788.676C39.068 183.651 18.186 294.69 28.43 404.354a2.02 2.02 0 0 0 .765 1.375 487.7 487.7 0 0 0 146.825 74.189 1.9 1.9 0 0 0 2.063-.676A348 348 0 0 0 208.12 430.4a1.86 1.86 0 0 0-1.019-2.588 321 321 0 0 1-45.868-21.853 1.885 1.885 0 0 1-.185-3.126 251 251 0 0 0 9.109-7.137 1.82 1.82 0 0 1 1.9-.256c96.229 43.917 200.41 43.917 295.5 0a1.81 1.81 0 0 1 1.924.233 235 235 0 0 0 9.132 7.16 1.884 1.884 0 0 1-.162 3.126 301.4 301.4 0 0 1-45.89 21.83 1.875 1.875 0 0 0-1 2.611 391 391 0 0 0 30.014 48.815 1.86 1.86 0 0 0 2.063.7A486 486 0 0 0 610.7 405.729a1.88 1.88 0 0 0 .765-1.352c12.264-126.783-20.532-236.912-86.934-334.541M222.491 337.58c-28.972 0-52.844-26.587-52.844-59.239s23.409-59.241 52.844-59.241c29.665 0 53.306 26.82 52.843 59.239 0 32.654-23.41 59.241-52.843 59.241m195.38 0c-28.971 0-52.843-26.587-52.843-59.239s23.409-59.241 52.843-59.241c29.667 0 53.307 26.82 52.844 59.239 0 32.654-23.177 59.241-52.844 59.241"/></svg>
</a>
<a href="https://www.youtube.com/@kyegomez3242" target="_blank" rel="noopener" title="www.youtube.com" class="md-social__link">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 576 512"><!--! Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2024 Fonticons, Inc.--><path d="M549.655 124.083c-6.281-23.65-24.787-42.276-48.284-48.597C458.781 64 288 64 288 64S117.22 64 74.629 75.486c-23.497 6.322-42.003 24.947-48.284 48.597-11.412 42.867-11.412 132.305-11.412 132.305s0 89.438 11.412 132.305c6.281 23.65 24.787 41.5 48.284 47.821C117.22 448 288 448 288 448s170.78 0 213.371-11.486c23.497-6.321 42.003-24.171 48.284-47.821 11.412-42.867 11.412-132.305 11.412-132.305s0-89.438-11.412-132.305m-317.51 213.508V175.185l142.739 81.205z"/></svg>
</a>
<a href="https://www.linkedin.com/company/swarms-corp/" target="_blank" rel="noopener" title="www.linkedin.com" class="md-social__link">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2024 Fonticons, Inc.--><path d="M416 32H31.9C14.3 32 0 46.5 0 64.3v383.4C0 465.5 14.3 480 31.9 480H416c17.6 0 32-14.5 32-32.3V64.3c0-17.8-14.4-32.3-32-32.3M135.4 416H69V202.2h66.5V416zm-33.2-243c-21.3 0-38.5-17.3-38.5-38.5S80.9 96 102.2 96c21.2 0 38.5 17.3 38.5 38.5 0 21.3-17.2 38.5-38.5 38.5m282.1 243h-66.4V312c0-24.8-.5-56.7-34.5-56.7-34.6 0-39.9 27-39.9 54.9V416h-66.4V202.2h63.7v29.2h.9c8.9-16.8 30.6-34.5 62.9-34.5 67.2 0 79.7 44.3 79.7 101.9z"/></svg>
</a>
</div>
</div>
</div>
</footer>
</footer>
</div>
<div class="md-dialog" data-md-component="dialog">
<div class="md-dialog__inner md-typeset"></div>
</div>
<script id="__config" type="application/json">{"base": "../..", "features": ["content.code.copy", "content.code.annotate", "navigation.tabs", "navigation.sections", "navigation.top", "navigation.footer", "announce.dismiss"], "search": "../../assets/javascripts/workers/search.d50fe291.min.js", "tags": null, "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": null}</script>
<script src="../../assets/javascripts/bundle.56ea9cef.min.js"></script>
</body>
</html>
Reference in new issue View Git Blame Copy Permalink