|
|
@ -28,7 +28,7 @@ docker run --rm --env "HF_TOKEN=<your hugging face token>" \
|
|
|
|
--ipc=host \
|
|
|
|
--ipc=host \
|
|
|
|
-p 8000:8000 \
|
|
|
|
-p 8000:8000 \
|
|
|
|
llm-serving:vllm-cpu \
|
|
|
|
llm-serving:vllm-cpu \
|
|
|
|
--model NousResearch/Hermes-3-Llama-3.1-8B
|
|
|
|
--model <Huggingface Model Path>
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
* Start the Chatbot API Server with the following shell command:
|
|
|
|
* Start the Chatbot API Server with the following shell command:
|
|
|
@ -37,6 +37,15 @@ docker run --rm --env "HF_TOKEN=<your hugging face token>" \
|
|
|
|
uvicorn server:app --port 8888
|
|
|
|
uvicorn server:app --port 8888
|
|
|
|
```
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Start the Chatbot UI
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
From the chatbot-ui directory:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
|
|
|
yarn install
|
|
|
|
|
|
|
|
yarn run dev.
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
## Chatbot API Server (FastAPI)
|
|
|
|
## Chatbot API Server (FastAPI)
|
|
|
|
|
|
|
|
|
|
|
|
This API is written in Python and depends on FastAPI.
|
|
|
|
This API is written in Python and depends on FastAPI.
|
|
|
|