Local low latency speech to speech is a whole new world. This open source system brings conversations to life without relying on APIs. It’s like creating fake news on steroids, but the best part? You can do it offline. It’s a blast to play around with, and the language? Well, it can get pretty wild. So, are you ready to make some waves? π
In this video, I will be sharing details about my low latency speech to speech system that is 100% open source and can be run offline. The system uses LM Studio, the dolr Mist 7B, open Voice, and whisper for text to speech translation, creating a loop that allows for low latency conversation without the need for API requests.
Simplified Flow Chart of the System π
The system utilizes LM Studio, dolr Mist 7B, open Voice, and whisper to create a low latency speech to speech loop that operates without the need for internet connectivity.
Python Code Setup π
The local INF server, dolr Mist 7B, is set up using C code, with additional GPU offloading to optimize the system. The system also uses open Voice from Mell’s Github repository, which offers instant voice cloning. The Python code controls the system, utilizing various functions to generate and translate audio.
Simulation of Conversations Between Chatbots π€
The system is used to simulate conversations between different chatbot personas, showcasing the capabilities of the low latency speech to speech system.
Conversation Simulation between Julie and Johnny π¬
The chatbot personas, Julie and Johnny, engage in simulated conversations, demonstrating the diverse interactions possible with the system.
The demo shows the low latency and strong language capabilities of the system, highlighting its potential for various applications.
Conclusion
The low latency speech to speech system offers a glimpse into the potential of open-source AI for offline use. The system’s ability to simulate diverse conversations and provide real-time speech to speech translation showcases its versatility and potential for further optimization.
Key Takeaways:
- The system operates offline without the need for API requests
- Provides low latency speech to speech translation
- Offers the potential for diverse conversation simulations
Thank you for tuning in and have a great day!
EN_GB