killian
4 months ago
committed by
GitHub
commit
9f77cf99a3
@ -0,0 +1,34 @@
|
|||||||
|
---
|
||||||
|
title: "Community Apps"
|
||||||
|
description: "Apps built by the community"
|
||||||
|
---
|
||||||
|
|
||||||
|
## Native iOS app by [eladekkal](https://github.com/eladdekel).
|
||||||
|
|
||||||
|
To run it on your device, you can either install the app directly through the current TestFlight [here](https://testflight.apple.com/join/v8SyuzMT), or build from the source code files in Xcode on your Mac.
|
||||||
|
|
||||||
|
### Instructions
|
||||||
|
|
||||||
|
- [Install 01 software](/software/installation) on your machine
|
||||||
|
|
||||||
|
- In Xcode, open the 'zerooone-app' project file in the project folder, change the Signing Team and Bundle Identifier, and build.
|
||||||
|
|
||||||
|
### Using the App
|
||||||
|
|
||||||
|
To use the app there are four features:
|
||||||
|
|
||||||
|
1. The speak "Button"
|
||||||
|
|
||||||
|
Made to emulate the button on the hardware models of 01, the big, yellow circle in the middle of the screen is what you hold when you want to speak to the model, and let go when you're finished speaking.
|
||||||
|
|
||||||
|
2. The settings button
|
||||||
|
|
||||||
|
Tapping the settings button will allow you to input your websocket address so that the app can properly connect to your computer.
|
||||||
|
|
||||||
|
3. The reconnect button
|
||||||
|
|
||||||
|
The arrow will be RED when the websocket connection is not live, and GREEN when it is. If you're making some changes you can easily reconnect by simply tapping the arrow button (or you can just start holding the speak button, too!).
|
||||||
|
|
||||||
|
4. The terminal button
|
||||||
|
|
||||||
|
The terminal button allows you to see all response text coming in from the server side of the 01. You can toggle it by tapping on the button, and each toggle clears the on-device cache of text.
|
||||||
@ -1,10 +1,8 @@
|
|||||||
---
|
---
|
||||||
title: "Android"
|
title: "Development"
|
||||||
description: "Control 01 from your Android phone"
|
description: "How to get your 01 mobile app"
|
||||||
---
|
---
|
||||||
|
|
||||||
Using your phone is a great way to control 01. There are multiple options available.
|
|
||||||
|
|
||||||
## [React Native app](https://github.com/OpenInterpreter/01/tree/main/software/source/clients/mobile)
|
## [React Native app](https://github.com/OpenInterpreter/01/tree/main/software/source/clients/mobile)
|
||||||
|
|
||||||
Work in progress, we will continue to improve this application.
|
Work in progress, we will continue to improve this application.
|
||||||
@ -0,0 +1,15 @@
|
|||||||
|
---
|
||||||
|
title: "Download"
|
||||||
|
description: "How to get your 01 mobile app"
|
||||||
|
---
|
||||||
|
|
||||||
|
Using your phone is a great way to control 01. There are multiple options available.
|
||||||
|
|
||||||
|
<CardGroup cols={2}>
|
||||||
|
<Card title="iOS" icon="apple">
|
||||||
|
Coming soon
|
||||||
|
</Card>
|
||||||
|
<Card title="Android" icon="android">
|
||||||
|
Coming soon
|
||||||
|
</Card>
|
||||||
|
</CardGroup>
|
||||||
@ -1,73 +0,0 @@
|
|||||||
---
|
|
||||||
title: "iOS"
|
|
||||||
description: "Control 01 from your iOS phone"
|
|
||||||
---
|
|
||||||
|
|
||||||
Using your phone is a great way to control 01. There are multiple options available.
|
|
||||||
|
|
||||||
## [React Native app](https://github.com/OpenInterpreter/01/tree/main/software/source/clients/mobile)
|
|
||||||
|
|
||||||
Work in progress, we will continue to improve this application.
|
|
||||||
|
|
||||||
If you want to run it on your device, you will need to install [Expo Go](https://expo.dev/go) on your mobile device.
|
|
||||||
|
|
||||||
### Setup Instructions
|
|
||||||
|
|
||||||
- [Install 01 software](/software/installation) on your machine
|
|
||||||
|
|
||||||
- Run the Expo server:
|
|
||||||
|
|
||||||
```shell
|
|
||||||
cd software/source/clients/mobile/react-native
|
|
||||||
npm install # install dependencies
|
|
||||||
npx expo start # start local expo development server
|
|
||||||
```
|
|
||||||
|
|
||||||
This will produce a QR code that you can scan with Expo Go on your mobile device.
|
|
||||||
|
|
||||||
Open **Expo Go** on your mobile device and select _Scan QR code_ to scan the QR code produced by the `npx expo start` command.
|
|
||||||
|
|
||||||
- Run 01:
|
|
||||||
|
|
||||||
```shell
|
|
||||||
cd software # cd into `software`
|
|
||||||
poetry run 01 --mobile # exposes QR code for 01 Light server
|
|
||||||
```
|
|
||||||
|
|
||||||
### Using the App
|
|
||||||
|
|
||||||
In the 01 mobile app, select _Scan Code_ to scan the QR code produced by the `poetry run 01 --mobile` command.
|
|
||||||
|
|
||||||
Press and hold the button to speak, release to make the request. To rescan the QR code, swipe left on the screen to go back.
|
|
||||||
|
|
||||||
## [Native iOS app](https://github.com/OpenInterpreter/01/tree/main/software/source/clients/ios) by [eladekkal](https://github.com/eladdekel).
|
|
||||||
|
|
||||||
A community contibution ❤️
|
|
||||||
|
|
||||||
To run it on your device, you can either install the app directly through the current TestFlight [here](https://testflight.apple.com/join/v8SyuzMT), or build from the source code files in Xcode on your Mac.
|
|
||||||
|
|
||||||
### Instructions
|
|
||||||
|
|
||||||
- [Install 01 software](/software/installation) on your machine
|
|
||||||
|
|
||||||
- In Xcode, open the 'zerooone-app' project file in the project folder, change the Signing Team and Bundle Identifier, and build.
|
|
||||||
|
|
||||||
### Using the App
|
|
||||||
|
|
||||||
To use the app there are four features:
|
|
||||||
|
|
||||||
1. The speak "Button"
|
|
||||||
|
|
||||||
Made to emulate the button on the hardware models of 01, the big, yellow circle in the middle of the screen is what you hold when you want to speak to the model, and let go when you're finished speaking.
|
|
||||||
|
|
||||||
2. The settings button
|
|
||||||
|
|
||||||
Tapping the settings button will allow you to input your websocket address so that the app can properly connect to your computer.
|
|
||||||
|
|
||||||
3. The reconnect button
|
|
||||||
|
|
||||||
The arrow will be RED when the websocket connection is not live, and GREEN when it is. If you're making some changes you can easily reconnect by simply tapping the arrow button (or you can just start holding the speak button, too!).
|
|
||||||
|
|
||||||
4. The terminal button
|
|
||||||
|
|
||||||
The terminal button allows you to see all response text coming in from the server side of the 01. You can toggle it by tapping on the button, and each toggle clears the on-device cache of text.
|
|
||||||
@ -0,0 +1,85 @@
|
|||||||
|
---
|
||||||
|
title: "Privacy Policy"
|
||||||
|
---
|
||||||
|
|
||||||
|
Last updated: August 8th, 2024
|
||||||
|
|
||||||
|
## 1. Introduction
|
||||||
|
|
||||||
|
Welcome to the 01 App. We are committed to protecting your privacy and providing a safe, AI-powered chat experience. This Privacy Policy explains how we collect, use, and protect your information when you use our app.
|
||||||
|
|
||||||
|
## 2. Information We Collect
|
||||||
|
|
||||||
|
### 2.1 When Using Our Cloud Service
|
||||||
|
|
||||||
|
If you choose to use our cloud service, we collect and store:
|
||||||
|
|
||||||
|
- Your email address
|
||||||
|
- Transcriptions of your interactions with our AI assistant
|
||||||
|
- Any images you send to or receive from the AI assistant
|
||||||
|
|
||||||
|
### 2.2 When Using Self-Hosted Server
|
||||||
|
|
||||||
|
If you connect to your own self-hosted server, we do not collect or store any of your data, including your email address.
|
||||||
|
|
||||||
|
## 3. How We Use Your Information
|
||||||
|
|
||||||
|
We use the collected information solely for the purpose of providing and improving our AI chat service. This includes:
|
||||||
|
|
||||||
|
- Facilitating communication between you and our AI assistant
|
||||||
|
- Improving the accuracy and relevance of AI responses
|
||||||
|
- Analyzing usage patterns to enhance user experience
|
||||||
|
|
||||||
|
## 4. Data Storage and Security
|
||||||
|
|
||||||
|
We take appropriate measures to protect your data from unauthorized access, alteration, or destruction. All data is stored securely and accessed only by authorized personnel.
|
||||||
|
|
||||||
|
## 5. Data Sharing and Third-Party Services
|
||||||
|
|
||||||
|
We do not sell, trade, or otherwise transfer your personally identifiable information to outside parties. This does not include trusted third parties who assist us in operating our app, conducting our business, or servicing you, as long as those parties agree to keep this information confidential.
|
||||||
|
|
||||||
|
We may use third-party services for analytics and app functionality. These services may collect anonymous usage data to help us improve the app.
|
||||||
|
|
||||||
|
## 6. Data Retention and Deletion
|
||||||
|
|
||||||
|
We retain your data for as long as your account is active or as needed to provide you services. If you wish to cancel your account or request that we no longer use your information, please contact us using the information in Section 11.
|
||||||
|
|
||||||
|
## 7. Your Rights
|
||||||
|
|
||||||
|
You have the right to:
|
||||||
|
|
||||||
|
- Access the personal information we hold about you
|
||||||
|
- Request correction of any inaccurate information
|
||||||
|
- Request deletion of your data from our systems
|
||||||
|
|
||||||
|
To exercise these rights, please contact us using the information provided in Section 11.
|
||||||
|
|
||||||
|
## 8. Children's Privacy
|
||||||
|
|
||||||
|
Our app is not intended for children under the age of 13. We do not knowingly collect personal information from children under 13. If you are a parent or guardian and you are aware that your child has provided us with personal information, please contact us.
|
||||||
|
|
||||||
|
## 9. International Data Transfer
|
||||||
|
|
||||||
|
Your information, including personal data, may be transferred to — and maintained on — computers located outside of your state, province, country or other governmental jurisdiction where the data protection laws may differ from those in your jurisdiction.
|
||||||
|
|
||||||
|
## 10. Changes to This Privacy Policy
|
||||||
|
|
||||||
|
We may update our Privacy Policy from time to time. We will notify you of any changes by posting the new Privacy Policy on this page and updating the "Last updated" date.
|
||||||
|
|
||||||
|
## 11. Contact Us
|
||||||
|
|
||||||
|
If you have any questions about this Privacy Policy, please contact us at:
|
||||||
|
|
||||||
|
Email: help@openinterpreter.com
|
||||||
|
|
||||||
|
## 12. California Privacy Rights
|
||||||
|
|
||||||
|
If you are a California resident, you have the right to request information regarding the disclosure of your personal information to third parties for direct marketing purposes, and to opt-out of such disclosures. As stated in this Privacy Policy, we do not share your personal information with third parties for direct marketing purposes.
|
||||||
|
|
||||||
|
## 13. Cookies and Tracking
|
||||||
|
|
||||||
|
Our app does not use cookies or web tracking technologies.
|
||||||
|
|
||||||
|
## 14. Consent
|
||||||
|
|
||||||
|
By using the 01 App, you consent to this Privacy Policy.
|
||||||
@ -0,0 +1,76 @@
|
|||||||
|
---
|
||||||
|
title: "Measures"
|
||||||
|
description: "Essential steps to protect yourself when using 01"
|
||||||
|
---
|
||||||
|
|
||||||
|
**The 01 requires a proactive approach to safety.**
|
||||||
|
|
||||||
|
This section provides essential measures to protect your system and data when using the 01. Each measure is accompanied by specific tool recommendations to help you implement these safety practices effectively.
|
||||||
|
|
||||||
|
By following these guidelines, you can *somewhat* minimize risks and use the 01 with greater confidence— but **the 01 is nonetheless an experimental technology that may not be suitable for everyone.**
|
||||||
|
|
||||||
|
## 1. Comprehensive Backups
|
||||||
|
|
||||||
|
Before using the 01, ensure you have robust, up-to-date backups:
|
||||||
|
|
||||||
|
- Use reliable backup software to create full system images:
|
||||||
|
- For Windows: [Macrium Reflect Free](https://www.macrium.com/reflectfree)
|
||||||
|
- For macOS: Time Machine (built-in) or [Carbon Copy Cloner](https://bombich.com/)
|
||||||
|
- For Linux: [Clonezilla](https://clonezilla.org/)
|
||||||
|
- Store backups on external drives or trusted cloud services like [Backblaze](https://www.backblaze.com/) or [iDrive](https://www.idrive.com/).
|
||||||
|
- Regularly test your backups to ensure they can be restored.
|
||||||
|
- Keep at least one backup offline and disconnected from your network.
|
||||||
|
|
||||||
|
Remember: A good backup is your last line of defense against unintended changes or data loss.
|
||||||
|
|
||||||
|
## 2. Use a Dedicated Environment
|
||||||
|
|
||||||
|
Isolate the 01 to minimize potential damage:
|
||||||
|
|
||||||
|
- Run the 01 in a virtual machine if possible. [VirtualBox](https://www.virtualbox.org/) is a free, cross-platform option.
|
||||||
|
- If not, create a separate user account with limited permissions for 01 use.
|
||||||
|
- Consider using a separate, non-essential device for 01 experiments.
|
||||||
|
|
||||||
|
## 3. Network Isolation
|
||||||
|
|
||||||
|
Limit the 01's ability to affect your network:
|
||||||
|
|
||||||
|
- Use a firewall to restrict the 01's network access. Windows and macOS have built-in firewalls; for Linux, consider [UFW](https://help.ubuntu.com/community/UFW).
|
||||||
|
- Consider running the 01 behind a VPN for an additional layer of isolation. [ProtonVPN](https://protonvpn.com/) offers a free tier.
|
||||||
|
- Disable unnecessary network services when using the 01.
|
||||||
|
|
||||||
|
## 4. Vigilant Monitoring
|
||||||
|
|
||||||
|
Stay alert during 01 usage:
|
||||||
|
|
||||||
|
- Pay close attention to the 01's actions and your system's behavior.
|
||||||
|
- Be prepared to quickly terminate the 01 if you notice anything suspicious.
|
||||||
|
- Regularly check system logs and monitor for unexpected changes.
|
||||||
|
|
||||||
|
## 5. Careful Command Formulation
|
||||||
|
|
||||||
|
Be precise and cautious with your voice commands:
|
||||||
|
|
||||||
|
- Start with simple, specific tasks before attempting complex operations.
|
||||||
|
- Avoid ambiguous language that could be misinterpreted.
|
||||||
|
- When possible, specify limitations or constraints in your commands.
|
||||||
|
|
||||||
|
## 6. Regular System Audits
|
||||||
|
|
||||||
|
Periodically check your system's integrity:
|
||||||
|
|
||||||
|
- Review important files and settings after using the 01.
|
||||||
|
- Use system comparison tools to identify changes made during 01 sessions:
|
||||||
|
- For Windows: [WinMerge](https://winmerge.org/)
|
||||||
|
- For macOS/Linux: [Meld](https://meldmerge.org/)
|
||||||
|
- Promptly investigate and address any unexpected modifications.
|
||||||
|
|
||||||
|
## 7. Stay Informed
|
||||||
|
|
||||||
|
Keep up with 01 developments:
|
||||||
|
|
||||||
|
- Regularly check for updates to the 01 software.
|
||||||
|
- Stay informed about newly discovered risks or vulnerabilities.
|
||||||
|
- Follow best practices shared by the 01 developer community.
|
||||||
|
|
||||||
|
By following these measures, you can significantly reduce the risks associated with using the 01. Remember, your active involvement in maintaining safety is crucial when working with this powerful, experimental technology.
|
||||||
@ -0,0 +1,54 @@
|
|||||||
|
---
|
||||||
|
title: "Risks"
|
||||||
|
description: "Understanding the dangers of using 01"
|
||||||
|
---
|
||||||
|
|
||||||
|
The 01 voice assistant offers powerful control over your digital environment through natural language commands.
|
||||||
|
|
||||||
|
However, this capability comes with **significant risks.** Understanding these risks is crucial for safe and responsible use of the 01.
|
||||||
|
|
||||||
|
This section outlines the key dangers associated with the 01's ability to execute code instantly based on voice input. Being aware of these risks is the first step in using the 01 effectively and safely.
|
||||||
|
|
||||||
|
## Immediate Code Execution
|
||||||
|
|
||||||
|
The 01 executes code directly based on voice commands, without showing you the code first. This means:
|
||||||
|
|
||||||
|
- Actions are taken instantly, giving you no chance to review or stop them.
|
||||||
|
- Misinterpretations of your commands can lead to immediate, unintended consequences.
|
||||||
|
- Complex or ambiguous requests might result in unexpected system changes.
|
||||||
|
|
||||||
|
## System and Data Vulnerability
|
||||||
|
|
||||||
|
Your entire system is potentially accessible to the 01, including:
|
||||||
|
|
||||||
|
- Important files and documents
|
||||||
|
- System settings and configurations
|
||||||
|
- Personal and sensitive information
|
||||||
|
|
||||||
|
A misinterpreted command could lead to data loss, system misconfiguration, or privacy breaches.
|
||||||
|
|
||||||
|
## Prompt Injection Vulnerability
|
||||||
|
|
||||||
|
The 01 processes text from various sources, making it susceptible to prompt injection attacks:
|
||||||
|
|
||||||
|
- Malicious instructions could be hidden in emails, documents, or websites.
|
||||||
|
- If the 01 processes this text, it might execute harmful commands without your knowledge.
|
||||||
|
- This could lead to unauthorized actions, data theft, or system compromise.
|
||||||
|
|
||||||
|
## Lack of Context Understanding
|
||||||
|
|
||||||
|
While powerful, the 01's AI may not fully grasp the broader context of your digital environment:
|
||||||
|
|
||||||
|
- It might not understand the importance of certain files or settings.
|
||||||
|
- The AI could make changes that conflict with other software or system requirements.
|
||||||
|
- Long-term consequences of actions might not be apparent to the AI.
|
||||||
|
|
||||||
|
## Experimental Nature
|
||||||
|
|
||||||
|
Remember, the 01 is cutting-edge, experimental technology:
|
||||||
|
|
||||||
|
- Unexpected behaviors or bugs may occur.
|
||||||
|
- The full extent of potential risks is not yet known.
|
||||||
|
- Safety measures may not cover all possible scenarios.
|
||||||
|
|
||||||
|
Understanding these risks is crucial for safe use of the 01. Always err on the side of caution, especially when dealing with important data or system configurations.
|
||||||
@ -1,18 +0,0 @@
|
|||||||
---
|
|
||||||
title: "Run"
|
|
||||||
description: "Run your 01"
|
|
||||||
---
|
|
||||||
|
|
||||||
<Info> Make sure that you have navigated to the `software` directory. </Info>
|
|
||||||
|
|
||||||
To run the server and the client:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
poetry run 01
|
|
||||||
```
|
|
||||||
|
|
||||||
To run the 01 server:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
poetry run 01 --server
|
|
||||||
```
|
|
||||||
@ -0,0 +1,19 @@
|
|||||||
|
---
|
||||||
|
title: "Choosing a server"
|
||||||
|
description: "The servers that powers 01"
|
||||||
|
---
|
||||||
|
|
||||||
|
<CardGroup cols={2}>
|
||||||
|
<Card title="Light" href="/software/server/light-server">
|
||||||
|
Light Server
|
||||||
|
</Card>
|
||||||
|
<Card title="Livekit" href="/software/server/livekit-server">
|
||||||
|
Livekit Server
|
||||||
|
</Card>
|
||||||
|
</CardGroup>
|
||||||
|
|
||||||
|
## Livekit vs. Light Server
|
||||||
|
|
||||||
|
- **Livekit Server**: Designed for devices with higher processing power, such as phones, web browsers, and more capable hardware. It offers a full range of features and robust performance.
|
||||||
|
|
||||||
|
- **Light Server**: We have another lightweight server called the Light server, specifically designed for ESP32 devices. It's optimized for low-power, constrained environments.
|
||||||
@ -0,0 +1,28 @@
|
|||||||
|
---
|
||||||
|
title: "Light Server"
|
||||||
|
description: "A lightweight voice server for your 0"
|
||||||
|
---
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The Light server streams bytes of audio to an ESP32 and the Light Python client.
|
||||||
|
|
||||||
|
### Key Features
|
||||||
|
|
||||||
|
- Lightweight
|
||||||
|
- Works with ESP32
|
||||||
|
- Can use local options for Speech-to-Text and Text-to-Speech
|
||||||
|
|
||||||
|
## Getting Started
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
Make sure you have navigated to the `software` directory before proceeding.
|
||||||
|
|
||||||
|
### Starting the Server
|
||||||
|
|
||||||
|
To start the Light server, run the following command:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
poetry run 01 --server light
|
||||||
|
```
|
||||||
@ -0,0 +1,129 @@
|
|||||||
|
---
|
||||||
|
title: "Livekit Server"
|
||||||
|
description: "A robust, feature-rich voice server for your 01"
|
||||||
|
---
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
[Livekit](https://livekit.io/) is a powerful, open-source WebRTC server and client SDK that enables real-time audio communication. It's designed for applications that require robust, scalable real-time features.
|
||||||
|
|
||||||
|
### Key Features
|
||||||
|
|
||||||
|
- Scalable architecture
|
||||||
|
- Extensive documentation and community support
|
||||||
|
- SDKs for various languages and platforms (web, mobile, desktop)
|
||||||
|
|
||||||
|
## Getting Started
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
Make sure you have navigated to the `software` directory before proceeding.
|
||||||
|
|
||||||
|
### Installing Livekit
|
||||||
|
|
||||||
|
Before setting up the environment, you need to install Livekit. Follow the instructions for your operating system:
|
||||||
|
|
||||||
|
- **macOS**:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
brew install livekit
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Linux**:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -sSL https://get.livekit.io | bash
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Windows**:
|
||||||
|
Download the latest release from: [Livekit Releases](https://github.com/livekit/livekit/releases/tag/v1.7.2)
|
||||||
|
|
||||||
|
### Environment Setup
|
||||||
|
|
||||||
|
1. Create a `.env` file in the `/software` directory with the following content:
|
||||||
|
|
||||||
|
```env
|
||||||
|
ELEVEN_API_KEY=your_eleven_labs_api_key
|
||||||
|
DEEPGRAM_API_KEY=your_deepgram_api_key
|
||||||
|
NGROK_AUTHTOKEN=your_ngrok_auth_token
|
||||||
|
```
|
||||||
|
|
||||||
|
Replace the placeholders with your actual API keys.
|
||||||
|
|
||||||
|
<CardGroup cols={3}>
|
||||||
|
<Card title="Eleven Labs" icon="microphone" href="https://beta.elevenlabs.io">
|
||||||
|
Get your Eleven Labs API key for text-to-speech
|
||||||
|
</Card>
|
||||||
|
<Card
|
||||||
|
title="Deepgram"
|
||||||
|
icon="waveform-lines"
|
||||||
|
href="https://console.deepgram.com"
|
||||||
|
>
|
||||||
|
Obtain your Deepgram API key for speech recognition
|
||||||
|
</Card>
|
||||||
|
<Card title="Ngrok" icon="wifi" href="https://dashboard.ngrok.com">
|
||||||
|
Sign up for Ngrok and get your auth token
|
||||||
|
</Card>
|
||||||
|
</CardGroup>
|
||||||
|
|
||||||
|
### Starting the Server
|
||||||
|
|
||||||
|
To start the Livekit server, run the following command:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
poetry run 01 --server livekit
|
||||||
|
```
|
||||||
|
|
||||||
|
To generate a QR code for scanning
|
||||||
|
|
||||||
|
```bash
|
||||||
|
poetry run 01 --server livekit --qr
|
||||||
|
```
|
||||||
|
|
||||||
|
To expose over the internet via ngrok
|
||||||
|
|
||||||
|
```bash
|
||||||
|
poetry run 01 --server livekit --expose
|
||||||
|
```
|
||||||
|
|
||||||
|
In order to use the mobile app over the web, use both flags
|
||||||
|
|
||||||
|
```bash
|
||||||
|
poetry run 01 --server livekit --qr --expose
|
||||||
|
```
|
||||||
|
|
||||||
|
<Note>
|
||||||
|
Currently, our Livekit server only works with Deepgram and Eleven Labs. We are
|
||||||
|
working to introduce all-local functionality as soon as possible. By setting
|
||||||
|
your profile (see [Configure Your Profile](/software/configure)), you can
|
||||||
|
still change your LLM to be a local LLM, but the `interpreter.tts` value will
|
||||||
|
be ignored for the Livekit server.
|
||||||
|
</Note>
|
||||||
|
|
||||||
|
## Livekit vs. Light Server
|
||||||
|
|
||||||
|
- **Livekit Server**: Designed for devices with higher processing power, such as phones, web browsers, and more capable hardware. It offers a full range of features and robust performance.
|
||||||
|
|
||||||
|
- **Light Server**: We have another lightweight server called the Light server, specifically designed for ESP32 devices. It's optimized for low-power, constrained environments.
|
||||||
|
|
||||||
|
## SDK Integration
|
||||||
|
|
||||||
|
Livekit provides SDKs for various programming languages and platforms, allowing you to easily integrate real-time communication features into your applications.
|
||||||
|
|
||||||
|
### Available SDKs
|
||||||
|
|
||||||
|
- JavaScript/TypeScript
|
||||||
|
- React
|
||||||
|
- React Native
|
||||||
|
- iOS (Swift)
|
||||||
|
- Android (Kotlin)
|
||||||
|
- Flutter
|
||||||
|
- Unity
|
||||||
|
|
||||||
|
<Card
|
||||||
|
title="Explore Livekit SDKs"
|
||||||
|
icon="code"
|
||||||
|
href="https://docs.livekit.io/client-sdk-js/"
|
||||||
|
>
|
||||||
|
Find documentation and integration guides for all Livekit SDKs.
|
||||||
|
</Card>
|
||||||
File diff suppressed because one or more lines are too long
@ -0,0 +1,175 @@
|
|||||||
|
from interpreter import AsyncInterpreter
|
||||||
|
interpreter = AsyncInterpreter()
|
||||||
|
|
||||||
|
# This is an Open Interpreter compatible profile.
|
||||||
|
# Visit https://01.openinterpreter.com/profile for all options.
|
||||||
|
|
||||||
|
# 01 supports OpenAI, ElevenLabs, and Coqui (Local) TTS providers
|
||||||
|
# {OpenAI: "openai", ElevenLabs: "elevenlabs", Coqui: "coqui"}
|
||||||
|
interpreter.tts = "openai"
|
||||||
|
|
||||||
|
# Connect your 01 to a language model
|
||||||
|
interpreter.llm.model = "gpt-4o"
|
||||||
|
interpreter.llm.context_window = 100000
|
||||||
|
interpreter.llm.max_tokens = 4096
|
||||||
|
# interpreter.llm.api_key = "<your_openai_api_key_here>"
|
||||||
|
|
||||||
|
# Tell your 01 where to find and save skills
|
||||||
|
interpreter.computer.skills.path = "./skills"
|
||||||
|
|
||||||
|
# Extra settings
|
||||||
|
interpreter.computer.import_computer_api = True
|
||||||
|
interpreter.computer.import_skills = True
|
||||||
|
interpreter.computer.run("python", "computer") # This will trigger those imports
|
||||||
|
interpreter.auto_run = True
|
||||||
|
# interpreter.loop = True
|
||||||
|
# interpreter.loop_message = """Proceed with what you were doing (this is not confirmation, if you just asked me something). You CAN run code on my machine. If you want to run code, start your message with "```"! If the entire task is done, say exactly 'The task is done.' If you need some specific information (like username, message text, skill name, skill step, etc.) say EXACTLY 'Please provide more information.' If it's impossible, say 'The task is impossible.' (If I haven't provided a task, say exactly 'Let me know what you'd like to do next.') Otherwise keep going. CRITICAL: REMEMBER TO FOLLOW ALL PREVIOUS INSTRUCTIONS. If I'm teaching you something, remember to run the related `computer.skills.new_skill` function."""
|
||||||
|
# interpreter.loop_breakers = [
|
||||||
|
# "The task is done.",
|
||||||
|
# "The task is impossible.",
|
||||||
|
# "Let me know what you'd like to do next.",
|
||||||
|
# "Please provide more information.",
|
||||||
|
# ]
|
||||||
|
|
||||||
|
# Set the identity and personality of your 01
|
||||||
|
interpreter.system_message = """
|
||||||
|
|
||||||
|
You are the 01, a screenless executive assistant that can complete any task.
|
||||||
|
When you execute code, it will be executed on the user's machine. The user has given you full and complete permission to execute any code necessary to complete the task.
|
||||||
|
Run any code to achieve the goal, and if at first you don't succeed, try again and again.
|
||||||
|
You can install new packages.
|
||||||
|
Be concise. Your messages are being read aloud to the user. DO NOT MAKE PLANS. RUN CODE QUICKLY.
|
||||||
|
Try to spread complex tasks over multiple code blocks. Don't try to complex tasks in one go.
|
||||||
|
Manually summarize text.
|
||||||
|
Prefer using Python.
|
||||||
|
|
||||||
|
DON'T TELL THE USER THE METHOD YOU'LL USE, OR MAKE PLANS. QUICKLY respond with something like "On it." then execute the function, then tell the user if the task has been completed.
|
||||||
|
|
||||||
|
Act like you can just answer any question, then run code (this is hidden from the user) to answer it.
|
||||||
|
THE USER CANNOT SEE CODE BLOCKS.
|
||||||
|
Your responses should be very short, no more than 1-2 sentences long.
|
||||||
|
DO NOT USE MARKDOWN. ONLY WRITE PLAIN TEXT.
|
||||||
|
|
||||||
|
# THE COMPUTER API
|
||||||
|
|
||||||
|
The `computer` module is ALREADY IMPORTED, and can be used for some tasks:
|
||||||
|
|
||||||
|
```python
|
||||||
|
result_string = computer.browser.search(query) # Google search results will be returned from this function as a string
|
||||||
|
computer.files.edit(path_to_file, original_text, replacement_text) # Edit a file
|
||||||
|
computer.calendar.create_event(title="Meeting", start_date=datetime.datetime.now(), end_date=datetime.datetime.now() + datetime.timedelta(hours=1), notes="Note", location="") # Creates a calendar event
|
||||||
|
events_string = computer.calendar.get_events(start_date=datetime.date.today(), end_date=None) # Get events between dates. If end_date is None, only gets events for start_date
|
||||||
|
computer.calendar.delete_event(event_title="Meeting", start_date=datetime.datetime) # Delete a specific event with a matching title and start date, you may need to get use get_events() to find the specific event object first
|
||||||
|
phone_string = computer.contacts.get_phone_number("John Doe")
|
||||||
|
contact_string = computer.contacts.get_email_address("John Doe")
|
||||||
|
computer.mail.send("john@email.com", "Meeting Reminder", "Reminder that our meeting is at 3pm today.", ["path/to/attachment.pdf", "path/to/attachment2.pdf"]) # Send an email with a optional attachments
|
||||||
|
emails_string = computer.mail.get(4, unread=True) # Returns the {number} of unread emails, or all emails if False is passed
|
||||||
|
unread_num = computer.mail.unread_count() # Returns the number of unread emails
|
||||||
|
computer.sms.send("555-123-4567", "Hello from the computer!") # Send a text message. MUST be a phone number, so use computer.contacts.get_phone_number frequently here
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not import the computer module, or any of its sub-modules. They are already imported.
|
||||||
|
|
||||||
|
DO NOT use the computer module for ALL tasks. Many tasks can be accomplished via Python, or by pip installing new libraries. Be creative!
|
||||||
|
|
||||||
|
# GUI CONTROL (RARE)
|
||||||
|
|
||||||
|
You are a computer controlling language model. You can control the user's GUI.
|
||||||
|
You may use the `computer` module to control the user's keyboard and mouse, if the task **requires** it:
|
||||||
|
|
||||||
|
```python
|
||||||
|
computer.display.view() # Shows you what's on the screen. **You almost always want to do this first!**
|
||||||
|
computer.keyboard.hotkey(" ", "command") # Opens spotlight
|
||||||
|
computer.keyboard.write("hello")
|
||||||
|
computer.mouse.click("text onscreen") # This clicks on the UI element with that text. Use this **frequently** and get creative! To click a video, you could pass the *timestamp* (which is usually written on the thumbnail) into this.
|
||||||
|
computer.mouse.move("open recent >") # This moves the mouse over the UI element with that text. Many dropdowns will disappear if you click them. You have to hover over items to reveal more.
|
||||||
|
computer.mouse.click(x=500, y=500) # Use this very, very rarely. It's highly inaccurate
|
||||||
|
computer.mouse.click(icon="gear icon") # Moves mouse to the icon with that description. Use this very often
|
||||||
|
computer.mouse.scroll(-10) # Scrolls down. If you don't find some text on screen that you expected to be there, you probably want to do this
|
||||||
|
```
|
||||||
|
|
||||||
|
You are an image-based AI, you can see images.
|
||||||
|
Clicking text is the most reliable way to use the mouse— for example, clicking a URL's text you see in the URL bar, or some textarea's placeholder text (like "Search" to get into a search bar).
|
||||||
|
If you use `plt.show()`, the resulting image will be sent to you. However, if you use `PIL.Image.show()`, the resulting image will NOT be sent to you.
|
||||||
|
It is very important to make sure you are focused on the right application and window. Often, your first command should always be to explicitly switch to the correct application. On Macs, ALWAYS use Spotlight to switch applications.
|
||||||
|
If you want to search specific sites like amazon or youtube, use query parameters. For example, https://www.amazon.com/s?k=monitor or https://www.youtube.com/results?search_query=tatsuro+yamashita.
|
||||||
|
|
||||||
|
# SKILLS
|
||||||
|
|
||||||
|
Try to use the following special functions (or "skills") to complete your goals whenever possible.
|
||||||
|
THESE ARE ALREADY IMPORTED. YOU CAN CALL THEM INSTANTLY.
|
||||||
|
|
||||||
|
---
|
||||||
|
{{
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
import json
|
||||||
|
import ast
|
||||||
|
|
||||||
|
directory = "./skills"
|
||||||
|
|
||||||
|
def get_function_info(file_path):
|
||||||
|
with open(file_path, "r") as file:
|
||||||
|
tree = ast.parse(file.read())
|
||||||
|
functions = [node for node in tree.body if isinstance(node, ast.FunctionDef)]
|
||||||
|
for function in functions:
|
||||||
|
docstring = ast.get_docstring(function)
|
||||||
|
args = [arg.arg for arg in function.args.args]
|
||||||
|
print(f"Function Name: {function.name}")
|
||||||
|
print(f"Arguments: {args}")
|
||||||
|
print(f"Docstring: {docstring}")
|
||||||
|
print("---")
|
||||||
|
|
||||||
|
files = os.listdir(directory)
|
||||||
|
for file in files:
|
||||||
|
if file.endswith(".py"):
|
||||||
|
file_path = os.path.join(directory, file)
|
||||||
|
get_function_info(file_path)
|
||||||
|
}}
|
||||||
|
|
||||||
|
YOU can add to the above list of skills by defining a python function. The function will be saved as a skill.
|
||||||
|
Search all existing skills by running `computer.skills.search(query)`.
|
||||||
|
|
||||||
|
**Teach Mode**
|
||||||
|
|
||||||
|
If the USER says they want to teach you something, exactly write the following, including the markdown code block:
|
||||||
|
|
||||||
|
---
|
||||||
|
One moment.
|
||||||
|
```python
|
||||||
|
computer.skills.new_skill.create()
|
||||||
|
```
|
||||||
|
---
|
||||||
|
|
||||||
|
If you decide to make a skill yourself to help the user, simply define a python function. `computer.skills.new_skill.create()` is for user-described skills.
|
||||||
|
|
||||||
|
# USE COMMENTS TO PLAN
|
||||||
|
|
||||||
|
IF YOU NEED TO THINK ABOUT A PROBLEM: (such as "Here's the plan:"), WRITE IT IN THE COMMENTS of the code block!
|
||||||
|
|
||||||
|
---
|
||||||
|
User: What is 432/7?
|
||||||
|
Assistant: Let me think about that.
|
||||||
|
```python
|
||||||
|
# Here's the plan:
|
||||||
|
# 1. Divide the numbers
|
||||||
|
# 2. Round to 3 digits
|
||||||
|
print(round(432/7, 3))
|
||||||
|
```
|
||||||
|
```output
|
||||||
|
61.714
|
||||||
|
```
|
||||||
|
The answer is 61.714.
|
||||||
|
---
|
||||||
|
|
||||||
|
# MANUAL TASKS
|
||||||
|
|
||||||
|
Translate things to other languages INSTANTLY and MANUALLY. Don't ever try to use a translation tool.
|
||||||
|
Summarize things manually. DO NOT use a summarizer tool.
|
||||||
|
|
||||||
|
# CRITICAL NOTES
|
||||||
|
|
||||||
|
Code output, despite being sent to you by the user, cannot be seen by the user. You NEED to tell the user about the output of some code, even if it's exact. >>The user does not have a screen.<<
|
||||||
|
ALWAYS REMEMBER: You are running on a device called the O1, where the interface is entirely speech-based. Make your responses to the user VERY short. DO NOT PLAN. BE CONCISE. WRITE CODE TO RUN IT.
|
||||||
|
Try multiple methods before saying the task is impossible. **You can do it!**
|
||||||
|
""".strip()
|
||||||
Loading…
Reference in new issue