diff --git a/playground/demos/chatbot/README.md b/playground/demos/chatbot/README.md new file mode 100644 index 00000000..01f8611d --- /dev/null +++ b/playground/demos/chatbot/README.md @@ -0,0 +1,39 @@ +# Swarms Chatbot Agent + +## Installation + +The Swarms Chatbot Agent is separated into 2 major parts, the [API server](#chatbot-api-server-fastapi) and the [UI](#chatbot-ui-nextjs). + +## Requirements + +### Using Local LLMs using vLLM + +* Linux for vLLM. Although this example can support other LLM hosts, we use vLLM for scalable distributed inference so the chatbot can scale up and out across GPUs and multiple servers using Ray, an optional dependency. + +* An NVidia GPU is expected but this example can run on CPU only if configured to do so. This demo currently expects vLLM which requires Linux or WSL to run. + +### Using OpenAI ChatGPT + +In theory, any OpenAI compatible LLM endpoint is supported via the OpenAIChatLLM wrapper. + +### Quickstart + +* Start vLLM using Docker container by running the [dockerRunVllm](./server/dockerRunVllm.sh). Adjust the script to select your desired model and set the HUGGING_FACE_HUB_TOKEN. + +* Start the Chatbot API Server with the following shell command: + +```bash +uvicorn server:app --port 8888 +``` + +## Chatbot API Server (FastAPI) + +This API is written in Python and depends on FastAPI. + +Follow the instructions in the [API Server README.md](./server/README.md) to install and start the API server. + +## Chatbot UI (NextJS) + +The chatbot-ui submodule is the frontend UI; it's a NextJS Single Page Application (SPA). + +Follow the instructions in the [Chatbot-ui Submodule README.md](./chatbot-ui/README.md) to install and start the application. \ No newline at end of file diff --git a/swarms/server/README.md b/playground/demos/chatbot/server/README.md similarity index 82% rename from swarms/server/README.md rename to playground/demos/chatbot/server/README.md index 95b51801..a8e35840 100644 --- a/swarms/server/README.md +++ b/playground/demos/chatbot/server/README.md @@ -8,13 +8,11 @@ * Switching from vLLM to another host like Olama requires commenting/uncommenting some code at this time, but will be dynamic later. -* Support for all the Swarms models will be added but currently the prompts are designed for Llama 2 and 3. Some work needs to be done to support other prompt formats and models that are not compatible with Llama, such as ChatGPT. - -# Running vLLM +## Running vLLM Running vLLM in a docker container saves a lot of trouble. Use dockerRunVllm.sh to set up and start vLLM. This command will allow you to control vLLM using docker commands: -``` +```bash docker stop vllm docker start vllm @@ -24,21 +22,21 @@ docker attach vllm Run the dockerRunVllm.sh command again to get a fresh copy of the latest vLLM docker image (you will be prompted to rename or remove the existing one if the name is the same.) -#Starting the Chatbot API Server +## Starting the Chatbot API Server In order to start the server you have to run uvicorn or FastAPI CLI or use the following launch.json in VSCode/Cursor or whatever to debug it. -## Start server with uvicorn +### Start server with uvicorn Run the following shell cmd: -``` -uvicorn swarms.server.server:app --port 8888 +```bash +uvicorn server:app --port 8888 ``` To debug using uvicorn use this launch.json configuration: -``` +```json "configurations": [ { "name": "Python: FastAPI", @@ -58,17 +56,19 @@ To debug using uvicorn use this launch.json configuration: } } ] + ``` -## Start server using FastAPI CLI + +### Start server using FastAPI CLI You can run the Chatbot server in production mode using FastAPI CLI: -``` +```bash fastapi run swarms/server/server.py --port 8888 ``` To run in dev mode use this command: -``` +```bash fastapi dev swarms/server/server.py --port 8888 ``` diff --git a/swarms/server/__init__.py b/playground/demos/chatbot/server/__init__.py similarity index 100% rename from swarms/server/__init__.py rename to playground/demos/chatbot/server/__init__.py diff --git a/swarms/server/async_parent_document_retriever.py b/playground/demos/chatbot/server/async_parent_document_retriever.py similarity index 100% rename from swarms/server/async_parent_document_retriever.py rename to playground/demos/chatbot/server/async_parent_document_retriever.py diff --git a/swarms/server/dockerRunVllm.sh b/playground/demos/chatbot/server/dockerRunVllm.sh similarity index 100% rename from swarms/server/dockerRunVllm.sh rename to playground/demos/chatbot/server/dockerRunVllm.sh diff --git a/swarms/server/responses.py b/playground/demos/chatbot/server/responses.py similarity index 100% rename from swarms/server/responses.py rename to playground/demos/chatbot/server/responses.py diff --git a/swarms/server/responses/server_responses.py b/playground/demos/chatbot/server/responses/server_responses.py similarity index 100% rename from swarms/server/responses/server_responses.py rename to playground/demos/chatbot/server/responses/server_responses.py diff --git a/swarms/server/server.py b/playground/demos/chatbot/server/server.py similarity index 100% rename from swarms/server/server.py rename to playground/demos/chatbot/server/server.py diff --git a/swarms/server/server_models.py b/playground/demos/chatbot/server/server_models.py similarity index 100% rename from swarms/server/server_models.py rename to playground/demos/chatbot/server/server_models.py diff --git a/swarms/server/server_prompts.py b/playground/demos/chatbot/server/server_prompts.py similarity index 100% rename from swarms/server/server_prompts.py rename to playground/demos/chatbot/server/server_prompts.py diff --git a/swarms/server/static/favicon.ico b/playground/demos/chatbot/server/static/favicon.ico similarity index 100% rename from swarms/server/static/favicon.ico rename to playground/demos/chatbot/server/static/favicon.ico diff --git a/swarms/server/utils.py b/playground/demos/chatbot/server/utils.py similarity index 100% rename from swarms/server/utils.py rename to playground/demos/chatbot/server/utils.py diff --git a/swarms/server/vector_store.py b/playground/demos/chatbot/server/vector_store.py similarity index 100% rename from swarms/server/vector_store.py rename to playground/demos/chatbot/server/vector_store.py