From 045186ccf6ca4e015e1e3788c1d96e35ffdef8b3 Mon Sep 17 00:00:00 2001 From: Mike Bird Date: Wed, 20 Mar 2024 16:29:05 -0400 Subject: [PATCH] language model service --- docs/getting-started/setup.mdx | 2 -- docs/services/language-model.mdx | 17 ++++++++++++++--- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/docs/getting-started/setup.mdx b/docs/getting-started/setup.mdx index e9ad18c..0f18339 100644 --- a/docs/getting-started/setup.mdx +++ b/docs/getting-started/setup.mdx @@ -3,8 +3,6 @@ title: "Setup" description: "Get your 01 up and running" --- -# Hosted 01OS - ## Captive portal To connect your 01, you will use the captive portal. diff --git a/docs/services/language-model.mdx b/docs/services/language-model.mdx index 7d2f1d8..eb0aa8d 100644 --- a/docs/services/language-model.mdx +++ b/docs/services/language-model.mdx @@ -3,13 +3,24 @@ title: "Language Model" description: "The LLM that powers your 01" --- -## Llamafile +## llamafile -Llamafile is cool! +llamafile lets you distribute and run LLMs with a single file. Read more about llamafile [here](https://github.com/Mozilla-Ocho/llamafile) + +```bash +# Set the LLM service to llamafile +poetry run 01 --llm-service llamafile +``` ## Llamaedge -Llamaedge is also cool! +llamaedge makes it easy for you to run LLM inference apps and create OpenAI-compatible API services for the Llama2 series of LLMs locally. +Read more about Llamaedge [here](https://github.com/LlamaEdge/LlamaEdge) + +```bash +# Set the LLM service to Llamaedge +poetry run 01 --llm-service llamaedge +``` ## Hosted Models