Smart Mirror with Voice and Gesture Control

Finished product

Finished product

This is how it looks like! Video here: https://www.youtube.com/watch?v=8swdfbnZn9E
I built a smart mirror that does voice recognition with a built-in microphone and gesture recognition with two distance sensors on each side of the frame. I also made a small operating system that you can install apps on and it’s really easy for developers to build apps for. The software I built runs on Electron (a web app engine) and provides APIs for developers to access the sensors and the microphones. However I have not published it yet because it needs some refinements. Features: – Voice control: When you clap your hands a microphone detects it and the voice recognition starts. You can say commands to launch apps such as “shakira video” to launch YouTube with a video of shakira, “play the radio” to launch the radio app, “show me the news” for the news app or “weather” for the weather.
  1. Gesture control: You can use gesture control in apps to navigate through the content or to play games. For example if you search for “pictures of cats” you can navigate through the pictures.
  2. Smartphone remote control: With a smartphone app I made you can see the list of all the installed apps and launch and interact with them.  The whole project cost about 400€.

Here you can see some slides with more info: https://docs.google.com/presentation/d/1g7epekWB7_d-9sFHxVkCuLePXaNvUdDchLt7i3u3QNQ/edit?usp=sharing

"Blueprints""Blueprints"

“Blueprints”

I borrowed the one-way mirror from my University so I had to design the hardware to be easy to take apart. I made two parts: the frame, which holds the glass and the sesnors¡, and the back support, which holds the TV, the microphone, the Raspberry Pi and the power adapters. The back is hung on to a wall and then the frame is hung on to the back.
Building the frame and backBuilding the frame and back

Building the frame and back

For the back support I used a wooden square and I cut a hole in the middle to put the TV support. Luckily , for the frame, we found an old wooden frame which had a nice size for our glass. We only had to paint it and drill four holes for the sensors.

TV

I used an LG 24″ TV
Back supportBack support
Raspberry Pi

Raspberry Pi

The software runs on a Raspberry Pi. It can run on the RPi2 and the 3.
Developing

Developing

I developed all the software on the Raspberry Pi itself. Using geany. After all it’s only Javascript, HTML, CSS and some Python.
News app

News app

kz41y6v

Smartphone remote app

Shows all the apps that are installed on the mirror and lets you launch them. Also lets you use the phones microphone for voice control.
Smartphone remote app

Leave a Reply

Your email address will not be published. Required fields are marked *