From 70d68167512c5fda2e34bb2b2535f59381689194 Mon Sep 17 00:00:00 2001 From: Mike Bird Date: Tue, 30 Jul 2024 15:09:36 -0400 Subject: [PATCH] link to lmc doc --- docs/hardware/custom_hardware.mdx | 69 ++++--------------------------- 1 file changed, 7 insertions(+), 62 deletions(-) diff --git a/docs/hardware/custom_hardware.mdx b/docs/hardware/custom_hardware.mdx index 6eb9e0f..50cddd8 100644 --- a/docs/hardware/custom_hardware.mdx +++ b/docs/hardware/custom_hardware.mdx @@ -19,65 +19,10 @@ To transmit audio commands to 01, send LMC audio chunks to the websocket defined To support the incoming `L`anguage `M`odel `C`omputer architecture, we extend OpenAI's messages format to include additional information, and a new role called `computer`: -```python -# The user sends a message. -{"role": "user", "type": "message", "content": "What's 2380*3875?"} - -# The assistant runs some code. -{"role": "assistant", "type": "code", "format": "python", "content": "2380*3875"} - -# The computer responds with the result of the code. -{"role": "computer", "type": "console", "format": "output", "content": "9222500"} - -# The assistant sends a message. -{"role": "assistant", "type": "message", "content": "The result of multiplying 2380 by 3875 is 9222500."} -``` - -## Anatomy - -Each message in the LMC architecture has the following parameters (`format` is only present for some types): - -``` -{ - "role": "", # Who is sending the message. - "type": "", # What kind of message is being sent. - "format": "" # Some types need to be further specified, so they optionally use this parameter. - "content": "", # What the message says. -} -``` - -| Parameter | Description | -| --------- | ------------------------------------- | -| `role` | The sender of the message. | -| `type` | The kind of message being sent. | -| `content` | The actual content of the message. | -| `format` | The format of the content (optional). | - -## Roles - -| Role | Description | -| ----------- | ------------------------------------------------------- | -| `user` | The individual interacting with the system. | -| `assistant` | The language model. | -| `computer` | The system that executes the language model's commands. | - -## Possible Message Types / Formats - -Any role can produce any of the following formats, but we've included a `Common Roles` column to give you a sense of the message type's usage. - -| Type | Format | Content Description | Common Roles | -| ------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | -| message | None | A text-only message. | `user`, `assistant` | -| console | active_line | The active line of code (from the most recent code block) that's executing. | `computer` | -| console | output | Text output resulting from `print()` statements in Python, `console.log()` statements in Javascript, etc. **This includes errors.** | `computer` | -| image | base64 | A `base64` image in PNG format (default) | `user`, `computer` | -| image | base64.png | A `base64` image in PNG format | `user`, `computer` | -| image | base64.jpeg | A `base64` image in JPEG format | `user`, `computer` | -| image | path | A path to an image. | `user`, `computer` | -| code | html | HTML code that should be executed. | `assistant`, `computer` | -| code | javascript | JavaScript code that should be executed. | `assistant`, `computer` | -| code | python | Python code that should be executed. | `assistant` | -| code | r | R code that should be executed. | `assistant` | -| code | applescript | AppleScript code that should be executed. | `assistant` | -| code | shell | Shell code that should be executed. | `assistant` | -| audio | wav | audio in wav format for websocket. | `user` | + + Read about LMC messages protocol here. +