You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
39 lines
893 B
39 lines
893 B
---
|
|
title: "Language Model"
|
|
description: "The LLM that powers your 01"
|
|
---
|
|
|
|
## llamafile
|
|
|
|
llamafile lets you distribute and run LLMs with a single file. Read more about llamafile [here](https://github.com/Mozilla-Ocho/llamafile)
|
|
|
|
```bash
|
|
# Set the LLM service to llamafile
|
|
poetry run 01 --llm-service llamafile
|
|
```
|
|
|
|
## Llamaedge
|
|
|
|
llamaedge makes it easy for you to run LLM inference apps and create OpenAI-compatible API services for the Llama2 series of LLMs locally.
|
|
Read more about Llamaedge [here](https://github.com/LlamaEdge/LlamaEdge)
|
|
|
|
```bash
|
|
# Set the LLM service to Llamaedge
|
|
poetry run 01 --llm-service llamaedge
|
|
```
|
|
|
|
## Hosted Models
|
|
|
|
01OS leverages liteLLM which supports [many hosted models](https://docs.litellm.ai/docs/providers/).
|
|
|
|
To select your providers
|
|
|
|
```bash
|
|
# Set the LLM service
|
|
poetry run 01 --llm-service openai
|
|
```
|
|
|
|
## Other Models
|
|
|
|
More instructions coming soon!
|