updated readme files and moved everything to the playground demos

pull/570/head
Richard Hein 8 months ago
parent 8e01de789d
commit a44630a591

@ -0,0 +1,39 @@
# Swarms Chatbot Agent
## Installation
The Swarms Chatbot Agent is separated into 2 major parts, the [API server](#chatbot-api-server-fastapi) and the [UI](#chatbot-ui-nextjs).
## Requirements
### Using Local LLMs using vLLM
* Linux for vLLM. Although this example can support other LLM hosts, we use vLLM for scalable distributed inference so the chatbot can scale up and out across GPUs and multiple servers using Ray, an optional dependency.
* An NVidia GPU is expected but this example can run on CPU only if configured to do so. This demo currently expects vLLM which requires Linux or WSL to run.
### Using OpenAI ChatGPT
In theory, any OpenAI compatible LLM endpoint is supported via the OpenAIChatLLM wrapper.
### Quickstart
* Start vLLM using Docker container by running the [dockerRunVllm](./server/dockerRunVllm.sh). Adjust the script to select your desired model and set the HUGGING_FACE_HUB_TOKEN.
* Start the Chatbot API Server with the following shell command:
```bash
uvicorn server:app --port 8888
```
## Chatbot API Server (FastAPI)
This API is written in Python and depends on FastAPI.
Follow the instructions in the [API Server README.md](./server/README.md) to install and start the API server.
## Chatbot UI (NextJS)
The chatbot-ui submodule is the frontend UI; it's a NextJS Single Page Application (SPA).
Follow the instructions in the [Chatbot-ui Submodule README.md](./chatbot-ui/README.md) to install and start the application.

@ -8,13 +8,11 @@
* Switching from vLLM to another host like Olama requires commenting/uncommenting some code at this time, but will be dynamic later.
* Support for all the Swarms models will be added but currently the prompts are designed for Llama 2 and 3. Some work needs to be done to support other prompt formats and models that are not compatible with Llama, such as ChatGPT.
# Running vLLM
## Running vLLM
Running vLLM in a docker container saves a lot of trouble. Use dockerRunVllm.sh to set up and start vLLM. This command will allow you to control vLLM using docker commands:
```
```bash
docker stop vllm
docker start vllm
@ -24,21 +22,21 @@ docker attach vllm
Run the dockerRunVllm.sh command again to get a fresh copy of the latest vLLM docker image (you will be prompted to rename or remove the existing one if the name is the same.)
#Starting the Chatbot API Server
## Starting the Chatbot API Server
In order to start the server you have to run uvicorn or FastAPI CLI or use the following launch.json in VSCode/Cursor or whatever to debug it.
## Start server with uvicorn
### Start server with uvicorn
Run the following shell cmd:
```
uvicorn swarms.server.server:app --port 8888
```bash
uvicorn server:app --port 8888
```
To debug using uvicorn use this launch.json configuration:
```
```json
"configurations": [
{
"name": "Python: FastAPI",
@ -58,17 +56,19 @@ To debug using uvicorn use this launch.json configuration:
}
}
]
```
## Start server using FastAPI CLI
### Start server using FastAPI CLI
You can run the Chatbot server in production mode using FastAPI CLI:
```
```bash
fastapi run swarms/server/server.py --port 8888
```
To run in dev mode use this command:
```
```bash
fastapi dev swarms/server/server.py --port 8888
```

Before

Width:  |  Height:  |  Size: 146 KiB

After

Width:  |  Height:  |  Size: 146 KiB

Loading…
Cancel
Save