I turned the most evil thing in the universe into a lightswitch.
I did not want Google or Amazon listening to me, so I created my own passive-aggressive voice assistant based on the main villain from Portal 2 to control my smart home.
You can now test out the GLaDOS Voice Generator TTS module.
This is an ongoing DIY project by a Portal fan, not a product available commercially. Code is distributed as is to work as an example for others.
About the DIY voice assistant project
I wanted to control my smart home with voice, but having Amazon or Google devices around creeped me out because their devices and AI are just black boxes. I have no idea what they are doing in the background, and it’s out of my control.
So, I created my own voice assistant that runs locally with Python and a potato running Ubuntu This project is still ongoing, and this is a short overview of what I’ve done so far. You can follow me (@nerdaxic) on Twitter; I post updates there. This project seems to have turned into functional art.
Why GLaDOS as voice assistant?
Mostly because I found GLaDOS funny. The character’s voice is recognizable and relatively easy to generate with a female TTS engine and Melodyne.
In addition, GLaDOS is a well-written character in the portal game series, with a good back-story and personality. This is important to write the responses for the assistant to stay in character.
- Local Trigger word detection using PocketSphinx
- Speech to text processing using Google’s API (for now)
- GLaDOS Text-to-Speech generation using locally hosted TTS
- Animatronic eye control using servos
- Round LCD for an eye to display textures
GLaDOS Text-to-speech engine
The voice assistant uses a locally hosted TTS API to generate the voice samples.
The open source neural TTS runs quickly enough, that the voice samples do not need to be cached.
If you wish to try it out, you can test my public GLaDOS Voice Generator
Works with Home Assistant
Device communicates locally with Home Assistant server to speak out notifications, get sensor data and control lights, scenes and devices.
- “Turn on living room ac”
- “When is the sauna ready”
- “What is the bedroom humidity”
Infrared face recognition
There is Raspberry Pi NoIR camera built into the bezel of the eye.
Idea is to mount the head on a neck mechanism, and use the face coordinates from camera to turn the head towards you when you speak to GLaDOS. Even in the dark!
Large animatronic eye can move in and out, and open and close eyelids.
Eye has a round LCD and neopixel-ring to provide the light and texture.
What can it do?
|Basic functionality||Novelty||Home Assistant|
|Tell the current time||Magic 8 ball answers||Control lights and scenes|
|Timers||Lore friendly jokes||Control devices and AC|
|Add things to shopping list||Answer to pleasantries||Tell the temperature / humidity|
|Current weather||Move eye around||Speak notifications aloud|
|Weekly weather forecast||IR-camera for face recognition||Estimate when sauna is ready|
Skills in development
- Local speech recognition for added data security
- Wolfram alpha integration for math and numeric data
“How much sodium in 25 grams of salt”
“What was the oil price in 1972”
GLaDOS Animatronic Hardware
Animatronic GLaDOS body is mostly 3D-printed, based on YouTuber Mr. Volt’s open source design.
Check out his original video on YouTube.
The main difference between our projects is that Mr. Volt used Amazon Echo as the voice assistant, where I made my own. His published design was not complete, so I had to re-design some of the internal mechanics of my own.
|Power supply for Digital + Audio||Raspberry Pi 15W USB-C Power supply|
|Camera||Raspberry Pi NoIR camera|
|Microcontroller||Teensy 4, to control the eye LCD and NeoPixels|
|Eye lights||Adafruit NeoPixel Diffused 5mm Through-Hole for the “REC” light|
|Eye lights||Adafruit 16 x 5050 NeoPixel Ring|
|Eye LCD||1.28 Inch TFT LCD Display Module Round, GC9A01 Driver SPI Interface 240 x 240|
Adafruit stereo amplifier
GLaDOS has two paper-cone speakers built into her hears giving her stereo sound.
These are powered by 3,7W Class D amplifier. Audio in, audio out. Set volume by jumper.
ReSpeaker Mic Array
Far-field microphone array device capable of detecting voices up to 5m away. Detects sound direction and improves audio quality by onboard processing.
|Audio amplifier||Adafruit Stereo 3.7W Class D Audio Amplifier|
|Speakers||Visaton FRS 7|
|Microphone & Audio interface||ReSpeaker Mic Array V2.0|
Mechanics are powered from their own power supply to allow more power for the servos and prevent brown-outs.
|Power supply||MeanWell LRS-50-5 5V|
|Servo controller||Pololu Micro Maestro|
|Servo: Eye movement||35 kg DS3235 (Control Angle 180)|
|Servo: Eyelids||25 kg DS3225 (Control Angle 180)|
|Screws||Various M3 and M4 screws|
|Jumper wires||0.32 mm²/22 AWG assortment|
Software & AI-Voice Assistant pipeline
Software works in few main parts built into glados.py and gladosTTS.py files.
1) Keyword detection with Pocketsphinx LiveSpeech
Main script hooks into the microphone plugged into the Raspberry Pi and is constantly listening for the selected trigger word. If Pocketsphinx detects the triggerword it is passed to the speech-to-text engine.
2) Python speech_recognition library
Once triggerword is detected, speech_recognition starts to listen for input from the microphone. After a small timeout the data is then passed to recognizer built in the library. This sends the recorded audio to Google for speech-to-text recognition and the service returns the spoken input as text.
This is not optimal and I’m looking for reliable solutions that can work locally, but with this solution I know what is being sent to Google and that it’s not being tied to any Google account.
3) Natural language processing & voice assistant skills
Once the command has been gotten from the speech regonition API, command is parsed. Currently this is done with a long if-else statement:
if 'cancel' in command: return elif 'time' in command: readTime() elif 'turn on' in command and 'daylight' in command: activateScene("scene.daylight") else: # tell user about an error and log the failed command
4) GLaDOS Text-to-Speech
In the Portal games GLaDOS has a distinctive and calm female voice. Originally the voice was generated by recording opera singer Ellen McLain speak the lines and pushing the audio through an autotune process to throw off the intonation to make her sound more robotic.
Local GLaDOS Text-to-speech (TTS) Voice Generator
I have moved to a neural network based GLaDOS TTS engine that can generate GLaDOS voice locally on the computer that is running the voice assistant. This makes the whole process more robust and secure. Things like weather forecasts take about 10 seconds to generate on an old laptop.
The TTS model was originally developed and published by Reddit user u/R2D2_FISH. Code was adapted to work with GLaDOS Voice Assistant, you can get it to play around with on my GitHub.
Online GLaDOS Voice Generator
I used to use glados.c-net.org to generate the voice samples over the API. Service uses the more traditional female TTS which is then pushed through Melodyne, but the process is really slow. With the que it can take half an hour to generate a simple voice sample!
This is why I have created local TTS cache of the audio samples and tried to break down the sentences into short segments. If the script finds the TTS sample from the memory card, play it. If not, generate it.
Most of the tasks and responses are repetitive, like time, weather and shopping list items. Device rarely needs to generate new samples.
Free and Open source (FOSS) Python voice assistant
You can download the code for free from GitHub.
You can use it for reference or build your own system for free!
GLaDOS Voice Assistant:
Old version GLaDOS Voice Assistant that works on Raspberry Pi:
AI-based stand-alone GLaDOS Text-to-speech engine:
Mr. Volt’s CAD-files:
Who needs Alexa when you can have GLaDOS from Portal?
If there’s one thing that the Raspberry Pi is known for, it’s the versatility it provides in home automation projects. The most exciting aspect of the project is the GLaDOS voice feature.
Onpas hieno! Suomalainen YouTube-käyttäjä teki Portal-pelin GlaDOS-tekoälyrobotista huiman hienon ääniavustajan.
Rantanen kertoi, että hän halusi muuttaa ”universumin pahimman asian valokatkaisijaksi”. Pitkäjänteisellä työllä ja 3D-tulostimella Rantanen onnistuikin luomaan hienon GLaDOS-pään, jonka sisällä laitteen sydämenä hyrrää Raspberry Pi 4 -tietokone.
Henri Rantanen is constructing a being of malice and resentment to control his thermostat and tell him the weather.
A self-programmed home assistant is a neat enough idea, but Rantanen really sells the fantasy of having GLaDOS in your home with the primarily 3D printed mechanical build surrounding the computer.
GLaDOS Voice Assistant Passive-Aggressively Automates Home.
Like many of us, nerdaxic gave in to the convenience and perceived simplicity of various home automation products made by Google and Amazon. Also like many of us, he found it a bit difficult to accept the privacy implications that surround such cloud connected devices.
Instead of giving in and buying back into the closed ecosystems he’d left behind, and decided to open his home to a murderous, passive aggressive, sarcastic, slightly insane AI: GLaDOS
Henri rakensi älykotiinsa avustajan – yli tonnin maksanut robotti osaa kertoa kun sauna on valmis.
Google ja Amazon arveluttivat liikaa, joten suomalaismies rakensi Portal-pelien GLaDOS-robotin omaksi ääniavustajakseen.
Hi, Epic work. so impressed.
Have you managed to install the GLaDOS TTS into Home assistant. As a TTS-service for notifications or do you know any one that have?
I have GLaDOS Voice Assistant as notification service in my Home Assistant. In theory it should be not-too-big-of-a-job to make the GLaDOS TTS engine run stand-alone in Home Assistant, but current model does not run on Raspberry Pi due to missing instruction set in the CPU.
Hi, very cool!
I love portal and glados, and was hoping to do the same. Only I don’t have this level of knowledge to create what you did.
Question, I am trying to run the glados-tts standalone engine, but i just can’t seem to get it working. Am I supposed to keep pip installing all necessary packages? If so, I get stuck at the espeak part, it just does not want to be found.
I also tried editing my path env variables (I’m on windows), but that didn;t seem to work. I also tried it on my raspberry pi, but encountered even more errors there.
If this is the wrong place to ask, my bad! Love your work.
Yeah, my blog is not a tech support platform – but here I can give you some pointers.
GLaDOS Text-to-speech (TTS) Voice Generator needs quite a lot of dependencies that need to be installed first. Check the read-me here.
Note that currently the TTS engine wont run on Raspberry Pi due to it’s CPU missing a required instruction set.*
I recommend setting up a virtual machine with Ubuntu 18 LTS and follow the instructions. With dependencies sorted I think it should run on Windows, but I have never tried.
Hey, is this project is too heavy for a Raspberry Pi 3 B (1gb ram) to run it? Also, can I modify it to recgonize commands in Portuguese?
Unortunately the GLaDOS-TTS will not run on Raspberry Pi computers due to their CPU missing a needed instuction set. If you can host the TTS on external server it is possible to set up on one, but that kind of defeats the purpose.
About Portugese, maybe, but it will not be an easy task. Have to translate all the commands, find a speech-to-text engine that works, and would need to have Portugese voice samples to train the HiFiGAN to sound like Portugese GLaDOS.
Count me out, but feel free to play with it! 😅